I’m not a lawyer, but, I believe that a legal standard in the US is that whoever puts anything into harms way is usually liable. Also, the seat is an attractive nuisance, i.e., you want to get your butt down onto the seat, and he knows about it. You are his guest, in unfamiliar surroundings. Additionally, the driver is in total control of his own vehicle, IIRC. The driver’s jerk attitude about “not doing…” doesn’t absolve him of looking out for his own property, on his own property, etc…his habits do not create liability.
OTOH, you are in the wrong because you didn’t grab his glasses, and toss them onto HIS seat. Or, pocket them, and take them into the restaurant and trash them. When he came discovered his loss, you could have said that you aren’t in the habit of watching out for other people’s property.
hh