If it were only possible to make a great big stick out of common sense and compassion, we could beat the jerks in this case about the head and shoulders with it.
The guy should have to pay the support. He stuck around for three kids so he obviously was acting in the role of a father. This isn’t a case where he found out the children weren’t his while they were newborns. He should have legal recourse against the biological father for half the amount paid, as well as the opportunity to sue his wife for fraud after the children reach majority. After all, his assumption of this debt is owed to her misrepresentations of the facts. This would make everybody pay for their misdeeds/own up to their responsibilities.
(Of course, this only represents what I think is fair, and in no way approximates our legal system.)
The visitation issue is wholly separate. I don’t know how much of an ass this guy is in real life, but he certainly sounds like one judging by the OP. If the children’s best interest isn’t served by visitation, then he doesn’t get to visit. Support payments aren’t visitation fees.