As others have mentioned, feeling unable to make a mad dash for an exit and escape is what scares me. It never used to scare me until one day, maybe ten years ago, when I was on a high floor in a government building and the fire alarm went off. We piled into the stairwell and then probably waited for a good twenty minutes as everyone in all the floors below us piled into the stairwell and exited in front of us. Interestingly, when I mentioned this incident years later to another coworker who had been standing in the stairwell behind me, he didn’t even remember it. So it clearly didn’t bother him at all.