IANAL; I think that argument could be made, but it would be a stretch, and (probably) wouldn’t succeed. If I were a lawyer, and arguing on behalf of Congress, I think I would say something to the effect that “punishment” means something like “putting you in prison” or even “taking away money you already have”–that is, a fine. We have a right to liberty and property, and those things can’t be taken away except by due process. No one has a right to a Presidential pension–former Presidents didn’t even get pensions until after Harry Truman left office–and I think Congress could probably just eliminate all of them (without violating the constitution) as a cost-cutting measure. (As a former labor union member, I personally think you ought not mess with someone’s pension after the fact–after they’ve already worked for you for 8 years, or after they’ve quit their job, and can no longer decide to seek other employment/to have sought other employment. Which is different from telling all new employees when they’re being hired “Sorry, we can’t afford to give you a pension like we’re doing with those old-timers”. But I really, really doubt SCOTUS would uphold this as a principle if Congress just did away with the Presidential pensions of Jimmy Carter, Bill Clinton, George W. Bush, Barack Obama, and Donald Trump.)
Taking away the pension of one specific President (not “going forward”, but after the fact) does seem punitive, and frankly it would be punitive. Against that, though, SCOTUS has ruled that, for example, sex offender registries (which I understand can be quite burdensome) are not “punishment” and are therefore constitutional, even when applied ex post facto. It’s possible that precedent will eventually be overturned, though.
Bottom line: If Congress revises the Former Presidents Act to take away the pension and other benefits from former Presidents who were impeached and convicted, even if they’d already left office, Trump a former President who hypothetically had been impeached and convicted after he left office (but who had been President before the hypothetical revision to the Former Presidents Act was enacted) might very well challenge that in court on ex post facto grounds. My bet would be that he would lose, though.