Basically, yeah.
Rockets carry their propellant with them, and guns leave the propellant with the launcher.
Recoilless launchers may be rifled or smooth-bore, but whichever they are, the propellant stays with the launcher while the munition leaves the launcher, and all associated mass, behind.
OTOH, when a rocket’s propellant is exhausted, it must carry the remaining mass of it’s propulsion system (rocket case) with it, or have some means of jettisoning the waste mass. With single-stage rockets, there’s no real point to dumping the motor case, as it’s already up-to-speed, and you’re not going to get much benefit from a lighter rocket after the motor has burned out. Multi-stage rockets, of course, always want to dump as much weight as possible before lighting the next stage.
Guns give much higher innitial velocity, at the cost of down-range performance. Rockets tend to have lower initial velocity, but accelerate, or at least maintain their velocity, all the way to the limits of the range. An obvious exception to this are RPGs, Bazookas, and the like. In those cases, the rocket burns its entire fuel grain very rapidly, almost explosively. By the time the projectile has left the launcher, it’s a ballistic projectile, having already burned out. This gives no-recoil launch, and saves the operator from getting a rocket-exhaust facial, but comes at a serious range penalty. Some anti-tank launchers in the past have used longer-burning motors, and had shields attached to protect the operator’s face and head. For example, the WWII German Raketenpanzerbüchse 54, seen here, has an obvious faceshield. The operators were also given special protective clothing.
As for the advantages of one over the other?
Recoilless guns generally fire heavier shells, while rockets use much lighter, more simple, launchers. For the most part, the recoilless rifle has been reduced by advancing rocket & missile technology to a specialty item, though some examples, like the old reliable Carl Gustav launcher, reamain viable weapons systems.