If you were previously stationary relative to the asteroid, and you then “stop (relative to another body) and change direction” then you are no longer stationary relative to the asteroid.
If you accelerate in any direction, then your velocity relative to any body will have changed. You might be going slower relative to asteroid X but faster relative to asteroid Y; that doesn’t matter - all that matters is that you have accelerated.
Picture this scenario: you’re heading away from the sun at 1,000km/h. Directly ahead of you out of the windscreen of your spaceship is asteroid A stationary relative to the sun (and so, relatively speaking, moving towards you at 1,000km/h). Directly behind you is asteroid B, heading away from the sun at 1,000km/h (and so stationary relative to you).
You now decelerate to zero, relative to the sun. You feel a “g-force” pushing you towards the windscreen, due to the deceleration from 1,000km/h away from the sun to 0km/h.
From an observer on asteroid A, you have gone from moving at 1,000km/h towards the asteroid, to stationary. Again, the g-force is the same.
From an observer on asteroid B, you have gone from stationary, to moving at 1,000km/h “backwards” towards the asteroid (because the asteroid is approaching the rear of your craft at that velocity). Again the g-force is the same (imagine reversing fast in your car from a standstill - the “g-force” you feel is towards the windscreen, the same as it is when braking hard while travelling forwards).