This is a little different than a car sale. In the case of a FSBO the agent doesn’t have time to drive around town looking for signs in yards of houses you might potentially like and FSBO’s are usually not very well advertised. It’s not like he wasn’t doing his best to find you a house.
Housing is such a subjective thing that he could find 25 houses that are perfect for you and you just don’t care for them for whatever emotional reasons.
He spent a fair amount of time and money for gas helping you look at houses it’s very considerate of you to want to compensate him. Of course you’re not obligated unless you have a buyer agency agreement, and if he didn’t get one of those that’s his own fault. (depending on the rules in your state). He also should have explained to you what to do if you did see a FSBO house.