This belief is so common that it needs to be singled out, stuffed into a leaky submarine, and sunk to the bottom of the ocean, unable forever to return.
Airplanes are probably the best example. At the very moment in history that people were supposedly saying that man was never meant to fly, hot-air balloons had a history of more than a century and the hydrogen dirigible was forecast to be the people mover of the next century. That’s because people understood the problem. Understood it all too well, in fact. Lighter-than-air crafts were known to be possible and powerful. Heavier-than-air crafts were impossible - but solely because there was no known engine that was both sufficiently light enough and powerful enough for the forces necessary. Then the gasoline engine was developed and got better and better.
See how this is a totally different usage of the word impossible? Heavier-than-air flight was an engineering impossibility. The physics was well known, which meant that success would occur as soon as the engineering exceeded the physical constraints.
For perpetual motion, the physics says explicitly that the constraints can never be exceeded. Cannot even ever be matched. You can’t simply improve your engineering sufficiently. There’s no such thing, even by using unobtainium.
Science is filled with almost impossible tasks that are nevertheless theoretically possible. Wormholes could be built with exotic matter that has negative pressure. Don’t ask me to explain that. There probably is no such stuff and if there were there’s probably no way to gather and manipulate and stabilize such stuff. But if there were it becomes an engineering problem. No real scientist would say that’s it’s impossible in theory, because we know the theory.
Perpetual motion is impossible in theory - because we also know that theory. Which says that no amount of engineering will suffice.
If your argument then becomes, but what if we come up with an entirely new theory that encompasses all physics and allows for perpetual motion? Strictly speaking, that argument can’t be refuted. Strictly speaking, I also can’t refute the argument that tomorrow the laws of physics will change and I’ll be able to flap my arms and fly to Betelgeuse. Neither one’s very likely, though, and science can be forgiven for not paying attention to either in the interim.