See, the problem is that most people don’t have a good mental model for what’s going on with orbits.
It’s not that you’ll float a little bit away. We have an intuitive sense from our experience and evolutionary history of living on the ground, that if you move something it moves a little bit, and then stops. If there’s nothing under you, you fall. And so on.
Except, when you change the environment, those things turn out not to be true.
Things stop moving here on Earth because of friction. So you throw a baseball, and it flies through the air, and then hits the ground and stops moving. But with no ground to hit, it’s going to keep moving. With no air to slow it down, it keeps moving exactly as fast as it did before.
So if you’re up in orbit, you absolutely can climb out of your spaceship and sit on the nose-cone. And you can’t fall off and plummet to your death, because you’re already falling at exactly the same rate as the spaceship. You and the spaceship are actually moving very fast, but since you’re both moving at the same rate you don’t move away from each other.
But suppose you’re sitting on the nose cone and give yourself a little bump, and start floating away. Well, if that happens on Earth you expect to fall back pretty fast to the surface, and you’re back sitting on the ship, or falling to the ground. Except, since you’re both in orbit, that won’t happen. You give yourself a nudge away from the ship, and you keep moving away. Newtonian physics. If you nudge yourself away at 1 ft/sec, you keep moving away. So after a minute you’re 60 feet away, after an hour you’re 3600 feet away.
But of course, you and the ship aren’t really stationary to begin with, you were just stationary with respect to each other. You’re actually both orbiting the Earth. There’s still plenty of gravity up there, and if you were stationary you’d plummet back to Earth. But to get into the orbit, you’ve given yourself a sideways vector. You’re actually falling towards the earth at the exact same speed as if you were stationary. You fall towards the Earth, but you keep missing.
Think about how it would be if you were just sitting up in space at the height of a space shuttle orbit. There’s the Earth below you. You start to fall. Pretty soon you’re going to smack into the Earth, although it’s going to be a long fall (Project Excelsior - Wikipedia). OK, but suppose you were going at some speed–you’d be falling towards the Earth at some speed, but if you had a fast enough sideways speed the falling and the sidewaysing would be equal, and after falling for 1 second you’d still be the same height above the earth. This is called an orbit. This is why the Moon goes around the Earth, and the Earth goes around the Sun.
So if you’re sitting on the nose cone of your spaceship, and push away gently, you don’t drift away forever, because what you’ve really done is put yourself in a slightly different orbit around the Earth. Your orbit is elliptical, which means that no other forces act on you, you’ll return to the exact same spot above the Earth after one orbit. Of course, the Earth will be rotating below you, but that doesn’t matter, the Earth and you and the spaceship will be orbiting the sun, but that doesn’t matter because you and Earth have the same orbit around the sun, and the Solar System will be orbiting the galactic core, but that doesn’t matter because you and the Earth and the Sun have the same orbit. What matters is that you’ve got a different orbit than the spaceship. And when you return to the same spot after orbiting the Earth once, the spaceship will also be orbiting the Earth. It will return to exactly the same spot–except at a different time.