what's wrong with my iPhone compass?

I just got a new (to me) iPhone 5, which I’m happy with. Except the compass shows north being in various random directions.

I thought the compass app used GPS positioning rather than magnetism, which is why you have to “roll the ball around” a few times to get a reading in a new location. Maps can find me just fine, making me think the GPS module works fine (though it might be more reliant on cell tower positions than I realize).

If ancient seafarers could have a north-pointing device in their pockets, why can’t I?

No, it uses a magnetometer, and is very sensitive to local magnetic fields.
I had to give up a nice case, because it used magnets to hold the case shut.

If it’s like some GPS devices I’ve used, you might have to move for some distance for it to be able to give you your heading as well as your location (and of course, you could always carry one of these in your pocket.)

GPS-based compasses exist; they’re called satellite compasses, and as you might expect, they’re immune to magnetic disturbances, but they also work everywhere on the earth, whereas magnetic compasses get flaky near the magnetic poles. But satellite compasses are large things with multiple GPS receivers separated by a distance of a couple of feet, and AFAIK their primary application is for boats, for which position and orientation are important during docking/undocking.

As has been noted, your phone’s compass senses the earth’s magnetic field.

Right, GPS is based on the time delay from each satellite to the receiver. That won’t change for different orientations of any single receiver, so you can’t use it to find directions. If you have multiple receivers, like Machine Elf said, then you can use the position of both to find the direction between them.

One advantage that you could get from a compass plus GPS vs. a compass alone is that the GPS could be built with a database of magnetic declination at various points on the Earth, so that once you determine magnetic north, you can instantly correct it to true north.

No. The iPhone compass is a magnetometer. It senses the earth’s magnetic field directly without needing to be moving.