I think other people have done a real good job of saying most of what I’d want, but I wanted to add two points.
In the OP, you ask about feeling anger and compassion at the same time. Personally, I think the ability to feel both is a pretty necessary rite of passage to becoming a rational human being. If you let anger overwhelm everything else, you’ve reduced another person to something less than fully human. If you let compassion overwhelm everything, you fail to take the necessary responses to solve problems and protect yourself. Ultimately, I think we get the best outcomes when we weigh the two together. In your case, I think this means acknowledging that his past contributes to whatever needs he’s addressing - understanding him as a person - but it doesn’t prevent you from insisting that certain behaviors stop.
Also, in terms of looking at family history and how it leads to infidelity for him, I think a key is to understand that it’s not a single-step process leading directly from “bad family” to “infidelity.” Instead, it’s more like “bad family” leads to “feelings that are unpleasant/unfulfilled” leads to “a thought process about how to address those feelings” which then ends in “infidelity.” It is impossible to change the bad family. It difficult to change the feelings that arise from the bad family, but therapy might make that possible. However, how those feelings are addressed is 100% under his control, and infidelity is never going to be the only option. If he’s honest with you, his therapist and himself, there should be a way to identify other ways to handle it that don’t involve deceiving and hurting you. For some people, merely confessing those feelings (to you, a priest, therapist, sponsor, whatever) may provide the necessary outlet to keep them from turning into a destructive behavior. Even if he concludes that infidelity is a necessary part of his life that he’s not going to give up, at least he could be honest about that and let you either agree to some kind of open marriage or move on to find a marriage in which you have what you need.