joint US/Canadian government border travelers card

Funniest border story
Just after my first flight to the uS, my wife and i decided to go to Seattle for the weekend.
i was working afternoons so by the time I got home, changed, ate and such we didn't hit the border until about 3AM.
We drive up to the booth and no one is there, we wait for about 5 minutes nothing.
We don't know what to do, we can't move forward, can't move back.
Then we see a door open inside the building and a head pop out, the door then closes.
The door opens back up and the guy is adjusting his belt as he walks through the front door.
Comes to the passenger side window, asks us what we are doing, then he tells us to go ahead.
He then turns around and heads back in, my wife swears he was unbuckling his belt again before he hit the door

We think we caught him short