Half-in-the-bag, a very fun movie review show commented on this trope, calling it, “The No-look look”. It’s where someone interrupts someone and they shut them down by answering without looking. Honestly, they filmed their own version and it makes it clear.
Watch from 8 minutes in to about 8:25 to get what he means.
8:16 is their actual reenactment.
This has to have been documented already, but what is it called on the TVtropes web site?