What causes that mass to eject out the back and thus transfer momentum? A push. With guns and rockets, that push is typically gas pressure.
Yes, it is different than a conventional push, where you push off an object. Take jumping off the ground. Conventionally, you push with your legs and then leave the ground. Alternately, you could throw a really big mass toward the ground and that would make you go up by momentum. The trick there is (a) supporting the really big mass before you throw it, and (b) throwing the mass rather than just relying on gravity to bring it down when you remove support.
Both use a push to make you move. One pushes on the stable, solid ground; the other pushes on a big mass that moves away from you*.
Rockets push on a smaller mass that moves away from you a lot faster. Still means momentum.
- Technically, the Earth is just a big mass that moves away from you, but it is such a much larger mass than you that it moves an insignificant amount compared to how much you move. But it’s the same thing, really.