Basically, a compass works because it has a little magnet inside that aligns itself to the local direction of the magnetic field. On the Earth’s surface, this direction is pretty closely aligned with the direction towards the magnetic north pole, but once you get up into space the magnetic field does all sorts of wonky things. You can see a picture here; one of those spherical compasses would align itself with the black lines in the picture.
Once you get out far enough, the Sun’s magnetic field becomes larger than the Earth’s; this distance is (according to the above page) about 64 000 km on the day side of the Earth and more than ten times longer on the night side. (For comparison, the height of a geosynchronous orbit - usually the highest orbit that we put satellites in - is only about 36 000 km.) In a low Earth orbit, such as those used by the Space Shuttles or the ISS, I would be surprised if a compass didn’t point north, at least to some approximation; you’re only about 500 km above the Earth’s surface there. So whether or not a compass would point towards the magnetic north pole depends on where you are in space.
If it weren’t for the sun, the earth’s magnetic field would be approximately a “dipole”, for which the strength decreases with the inverse cube of the distance. So, if you were orbiting one earth’s radius above the earth, you’d experience a field about 1/8 of the value here at the surface. I’m not sure how small it would have to be for the friction in the compass would be larger than the magnetic force (that depends on how cheap your compass is, anyway). But, as MikeS pointed out, the sun’s field becomes important at about 10 earth radii, where the earth’s field is down by 1/1000.
Google “magnetosphere” and you’ll find lots of links.