Former bomb-dropper here. Various folks have introduced a bunch of valid ideas, but also a bunch of red herrings for the specific case at hand.
When dropping bombs from high altitude and a level delivery, like every B-17 or B-29 footage you’ve ever watched …
The bomb(s) will continue to travel forward after dropping, albeit slowing their forward progress as they fall. The airplane can often gain some speed from the reduction in weight or drag, but this is negligible over the time scale we care about.
From the ground’s point of view, the bomb(s) will carry forward along the pre-release flight path some distance before impact. For WWII altitudes, speeds, and bomb designs this might be 2-ish miles from release to impact.
From the airplane’s POV, the bomb will begin to trail the aircraft and the trail will be getting larger faster all the way to bomb impact. For WWII altitudes & speeds, this might be 1/4 mile at impact.
For non-nuclear bombs and WWII high altitude deliveries, the bomb explosion is of no concern to the aircraft. Pretty to watch, but neither the shock wave nor the shrapnel, nor even secondaries if you’re bombing an ammo plant have any potential to get you.
Now for nuclear deliveries, escaping your own weapons effects was a big deal in 1945 and remains a big deal today.
As described by others above, for the high altitude level delivery, turning aggressively away was the best way to put distance between the bomber and the bomb. And at high altitude near the limit of the B-29’s capability, they needed to make a descending turn to be able to maintain flying speed in an aggressive (for a B-29) manuever. They lost a few thousand feet in altitude but gained a couple extra miles of separation. Net, net, that’s a winner. So that’s what they did and why they did it.
Now for bombing missions other than Hiroshima / Nagasaki, both before and since, there are dozens of additional tactical considerations and hundreds of variations on altitude, aircraft, tactics, weapons etc. This thread Could You "Slim Pickens" a Bomb In? - Factual Questions - Straight Dope Message Board includes a writeup by me on a different set of nuclear delivery tactics invented later.
The applied practical physics of unguided bombing is actually a pretty interesting topic. It’s a bit difficult to explain or understand without diagrams, which is why I’ve shied away from a complete treatment in the various threads we’ve had.