I've had this happen at our town's McDonalds. They re-did the lot and made a loop around the building, so you can use the back entrance, drive around the front of the building and get in the proper drive through line. But when the speaker is empty and there is no line, you can come in the back entrance, and just drive right up to the speaker.
Not too long ago, I was next in line for the speaker. A car pulled up through the back entrance, stopped and put on their turn signal. When the car in front of me moved ahead, that car then quickly swooped in front of me and cut in line. I made a "what the hell?" motion, but didn't escalate it, since my kid was in the car with me.
Not much you can do. I just shake my head at the laziness of some drivers.