As others have mentioned upthread, some less-than-fresh buffet foods are easy to spot – dried/shrunken chicken, crusty rice, wilted veggies, and so on. In other cases such as a separated sauce, the food just doesn’t look good, so I really don’t care whether it’s been sitting a long time or not – I’m still not eating it.
Of course, one of the nicest looking chicken pot pies I’ve ever eaten gave me a three-day encounter with salmonella, so you definitely can’t tell by looking all the time.