A few years back I was ‘suffering’ from this problem – as the only childless woman in a department of five, it was simply assumed that I would pick up the slack and work extra hours whenever needed. I grumbled and whined to anyone who would listen, which of course didn’t solve anything.
I stumbled onto a solution by accident. The company I worked for had a benefit where if you took courses that would make you a more valuable/productive employee, they would reimburse you for tuition & books, so long as you got at least a B. Well, there were skills I regretted not having, and I found a course at a relatively closeby college that met M T W evenings that looked almost perfect – except that I couldn’t quite make it to the course if I left at 5p.m., at least not reliably.
So I went to personnel and talked to them about it, and they agreed that the course would benefit the company (eventually) and so they wrote up a special work agreement for me that I would come into work 15 minutes earlier each day and thus I could leave work 15 minutes earlier. After this was hashed out, my supervisor was brought in, and was agreeable. (Even complimented me for my ambition.)
I made sure I told everyone else in the department how excited I was at taking the course, how hard I expect it to be to go back to school, that I was nervous about keeping up with ‘real’ college students, on and on. And from then on, I simply wound up my work and left the building at 4:45. (With all my work done, of course.) I had a ‘commitment’ that the company agreed was valuable, and so I was good to go.
It was astonishing how little friction there was over it. A lot of that was because they would normally only realize that there was still work that needed doing at 5pm – and I was already long gone. 
A couple of times at the beginning of the new regime one of the other workers would try to guilt me over leaving instead of covering for “X, who really needs to go to her son’s recital”, but really, what could they say? In essence I was already committed to ‘working’ extra hours to improve my value to the company (just at a different location), and I had it in writing that the company felt this was a Good Thing.
So what happened when Mother A left early to attend to some child needs? Why, Mothers B and C and D had to fight it out between them who would stay to finish. Or else the supervisor had to step in and finish…
The end result was that the mothers as a group started not taking as many extra hours off. Because the ‘culture’ changed. When there was an automatic fill-in (me) the mothers would smile benignly when this one or that one wanted to leave for some non-urgent matter. Why not? It didn’t hurt them. But now they knew that if SHE left early, one or more of the rest of them would have to stay late, causing all sorts of problems of timing…
Anyway… the point is, I found an excuse for leaving work at my appointed time that was considered equally valid to excuses of caring for a child. If you’re being dumped on unfairly, maybe you can do the same. Is there a course you can take? How about volunteering to work at some socially valued activity that just happens to require that you be there promptly at (whatever) time. Helping at a food banks? Working on a Hot line? Volunteering at a Library? Driving for Meals on Wheels? Providing assistance to Elderly or blind people? Cleaning cages at a animal shelter? Your duty stretches don’t have to be long, they just have to begin at the right hour.
Yes, you’ll still be using up at least a few hours you might prefer to spend socializing with your friends, but at least you will no longer feel you are being taking advantage of at work, and you’ll probably garner gold stars and approval for being a good citizen instead of being treated as a cold-hearted slacker.