I’ve contributed to that count a number of times myself!
My thoughts exactly. If I were the OP, I wouldn’t have asked, but if I were the brother, I would’ve paid it back and would’ve been mortified if I had to be reminded. I guess the brother’s conduct is weirder.
Also consider there may have been an equalizing transaction conducted between the wives.
MY WAG
While on a outing wife 1 borrows cash from W2 to make a secret purchase, let’s say it’s a gift for the OP 
. W1 promises to reimburse W2 later at home.
Later W2 when asked by husband for some tipping cash, she directs him to W1 for payment. Who whips out the cash without delay. Thereby confusing the husband. Who later confuses his brother.
Just wait and see, You’ll get your 40 back.
Oddly enough, this was a recent topic in my business class (e.g., difference between contracts and agreements) of course in this scenario, the connotation is slightly different. It’s just interesting to see that an unspoken agreement can lead to false assumptions and miscommunication (now this is in my behavioral science class - which in this case, is more relevant!)
The social dynamics between humans are weird. From a purely technical standpoint, you are not in the wrong because technically you’re entitled to that money. But from a social/very-stretched moral perspective, it can be considered rude that you had asked in the first place knowing what he had willingly done for you and for your family.
If you want to dive deeper for an answer - you can actually read up on autism. Because there is a spectrum in autism that lacks most of the basic social cues (i.e., giving someone a token of gratitude in exchange of a favor) and this is somewhat akin to that.
Of course, I don’t know what culture you and your family are from - or what ethnicities you are (this plays a big role) so I’m just going off a limb.
tl;dr - there’s no direct answer because there are too many factors in play!