My brain is wired like yours. I think my brain can not process numerical characters with the same speed and accuracy of text. My tendency is try to convert the numbers to text, for me to make sense of them mentally. Although converting them to text helps me stores them in memory, when it comes to doing mental math, I’m a mess. It’s like I can’t “see” them so that I can manipulate them properly.
When I write them down, I’m fine though.
I have a very good sense of direction, but it evaporates as soon as I drink alcohol in any quantity. So I have experienced both worlds - both having an automatic compass in my head and completely lacking one. I like to think it gives me some empathy for folks in your situation because ordinarily knowing my cardinal directions is as easy as knowing right-left-front-back-up-down. I’ve heard some people argue this is training, but I’ve had it since I was a young child. Training might improve what skill you have, but if you aren’t wired that way I don’t think training will help much, if at all.
Me, I have math issues. I’m one of those people who can’t remember more than 3-4 numbers in a sequence. If I don’t write down a phone number I will simply not be able to remember it. Do math in my head? Can you wait a half an hour for me to laboriously trudge through a simple sum? It’s [not lack of knowledge, or logic, or inability to formulate the problem - I can tell you how to figure a percentage, no problem - it difficulty in actually doing the math. I need a pencil and paper. Or a calculator. Give me a calculator I do just fine, as I know how to plug the numbers into a formula, but don’t ever ask me to do math in my head. The results are just ugly. I just can’t imagine what’s it like to be able to do math in one’s head with any sort of ease, or effortlessly remember numbers without hours or days of mind-numbing rote learning.
When I was in England, I had an American friend who would always make me figure out the restaurant tips because she said she couldn’t do the math. In England, at least at that time, accepted tip was 10%. It totally boggled my mind.
I still don’t know whether she really was that bad at math, or whether she had just decided she couldn’t do tip calculations and so had never bothered to try to figure out what a 10% tip was. (She did know that tip was 10%.)
Somehow, this calls to mind the quote by computer pioneer Charles Babbage:
I don’t teach them but I work with them every day, trying to figure out what they want me to program. When I ask, “Which do do you want me to do first, A or B?” the invariable answer is “Whoa, whoa, I’m not a programmer! That’s your job.” After 25 years I still haven’t figured out a good way to reply. It’s like asking a blind person to pick a color they like.
I had the dumbest problem with that yesterday…
In Barcelona’s Eixample, I was approached by a couple of tourists, who showed me a jpg of a map on a cellphone and asked me how to get to a point on it. Their map was oriented the way you describe, the usual way.
Only, maps of Barcelona printed locally have a different orientation, with the axis formed by Las Ramblas/Major de Gràcia in the middle and the squares of L’Eixample parallel to the edges of the map. The Paral.lel and Meridiana show at angles: in a normally-oriented map these are the only streets to be horizontal and vertical, respectively.
I wasn’t able to place us until I’d turned the map to the position I’m used to. I’ve been to that area thousands of times; the address they were looking for is less than 300m away from my grandmother’s. But it was just lines and blobs until I’d managed to position the map just-so.
Giving directions to another pair of tourists (from Passeig de Gràcia Train Station to their hotel), we were underground and I wasn’t sure whether the exit they had to take would leave them in a position where they had to go left or right, so I used the local direction: down. Well, downslope. Then I had to explain that yeah, I know it sounds strange the first time you hear it, but Passeig de Gràcia slopes and they had to walk “down” it.
And I am very good at remembering strings of numbers. I know all of my bank account numbers, phone numbers, social security number, my license number, without any issue. Mix it with letters and I have a harder time - thankfully my license plate is pretty easy. It’s three letters and the four numbers are both consecutive multiples of 15. When I told my SO this was how I remembered it, he looked at me blankly and said “That’s not a very easy way to remember it!” I was like…yes it is. 
Tellingly, I have a harder time keeping the letters straight in my head.
Faces, though? I am terrible at remembering faces. I know it’s partly because I don’t care to, but even when I try, I have a difficult time.
I’m OK at giving directions because I’m very factual. “Go three lights, make a right. Go down the street 3/4 of a mile and it’s on your left.” I have a very hard time taking directions if they are not given like this, however.
“go downtown and take the exit to south pearl street, it doesn’t say south pearl street, but it’s the one for Arbor Hill, you can’t miss it, then follow that road around for three miles and then you’ll see the armory garage, when you see the garage…”
blah blah blah. At this point I’ve stopped listening.
I also take the SAME route, or at most, one of two, to wherever I am going. I rarely vary. One day I told my SO this and he was trying to describe to me some other, SOOPER FAST route. I finally said, “You know I’m just going to nod and smile and then just drive the same way I always did, right?” I don’t have that compulsion to save 2.3 minutes off my time. I’d rather get there with no doubt.
My wife is just as bad. In many ways worse. In our own home she is completely unable to tell what room is either below or above where she’s standing.
We had a leaky pipe in our basement a few days ago and I could tell it was directly below the bay window in our kitchen. I didn’t expect her to know the exact location like I did but I thought for sure she knew we were under the kitchen. She asked me if maybe a toilet was leaking and if there was a toilet above that spot. The two bathrooms we have are one on top of the other and clear on the other side of the house.
The technical name for that style of navigation is “pilotage”, or navigating by landmarks.
If the person says “travel north for 3/4 of a mile then turn west” that’s dead reckoning.
Sometimes, a person is very much in turn with one of those two methods but not the other. When I give directions I try to use instructions that incorporate both so as to maximize the chances of being understood.
^ And that’s just confused and incoherent. 
I have an excellent sense of direction, and yet I feel exactly the same way - often not worth saving two extra minutes, which could easily be wiped out by a traffic jam anyhow.
I’m def. a pilotage type of person then. If someone told me the other I’d pull the same thing - smile, nod, and then get my own directions! First I have to stop and think about which way north is. Yes, I can figure it out - IF I know the town.
As to the confused and incoherent directions, I used to have a coworker who PRIDED herself on giving directions and always gave directions like that. I got to know her well and was eventually able to forbid her from giving directions to any of our volunteers. I told her why, too - EVERYONE GETS LOST.
ETA: I find this sort of thing fascinating - the way people navigate, I mean. We’re all so different!
In the same situation, I would quickly vizualize a map (of western Europe, to be precise) and that’s how I would know where west is.
“Left” is more complicated. It’s the side opposite to where the hand I use to write is situated. As a kid, I had to mimic writing to know where my right and my left where. It evolved over time (for instance, later, I would just think of writing) and now I know where my right hand is instantly. But I have to think of this hand to know where is my right and where is my left. It’s a quick process but not instantaneous, so as a result, I’m prone to say “then turn left” while pointing to the right when, for instance, I give directions. The pointing is always the correct part, and the words “left” and “right” the mistake.
I also had issues with prices since the creation of the Euro because I always visualize prices instantly on a kind of scale that makes turns, goes up or down, etc… And besides, stuff appears also on this scale according to their price. For instance, a cheap car is somewhere on an horizontal line going from 10 000 to 100000 and is stuck there. That was a valid position in French Francs, it is not in Euros. Furthermore, although I somehow adapt to normal inflation (prices of small appartments in Paris moved significantly to the right and even could went up a bit and to the left during the last 20 years), significant variations of prices over a short period aren’t taken into account. For instance computers are still in a much higher position in my mind than they are now in real life (and that position is still French Francs dependant, so I have to do the maths anyway)
I’m sure the previous paragraph makes no sense to most people but it’s the way my brain works.
My mother and I had a very strange discussion once, about how to describe where something is, geographically. If you asked me where Kansas is, I’d picture a map in my head, and say it’s in the center of the continental U.S., between Colorado and Missouri, or I’d say it’s south of Nebraska or north of Oklahoma. I’d always describe it as if I were looking at a map.
My mother would say it’s one of the states you go through on your way to California . . . or half-way to the west coast. She’d describe it from the point of view of driving in a car. I tried to get her to visualize a map, and to describe it in relation to other states, and she kept saying things like you’ll hit Kansas if you drive south from Nebraska.
And:
ME: Mansfield is south-west of here.
HER: Mansfield is on the way to Columbus.
I also noticed that, when following a map, as a passenger in the car, she always turned the map as the car turned. I always read a map with north on top.
This brings up an interesting problem. You establish radio contact with an alien civilization.
How do you tell them what “left” is?
I don’t remember the details, but I read somewhere that it would be possible to use the parity violation of beta decay in Cobalt-60 (the Lee-Yang postulate and the Wu-Ambler experiments of 1956, published in 1957). That would unambiguously determine “left” and “right”.
I understand that it would be something like “set up this particular experimental apparatus, and look at the spin of the electrons being ejected. The ones that are more prevalent will be (either right or left, depending on how you do the setup)”.
Interestingly, this wouldn’t work if the aliens and their environment are made of anti-matter, but that’s another question 
I didn’t even realise that the local maps were rotated! I assumed that the sea was to the south.
But yes, I know exactly what you mean by “up” and “down”. That slope made the city pretty easy for me to navigate (well…except for those little twisty streets in el Gòtic…)
Wouldn’t it be a lot easier to crystalize table sugar or glucose and talk about it in terms of the light rotation? The instrumentation seems simpler…
Thanks! There is a Kindle version, so I put it on my list.
The problem there, as far as I understand it, is that you can only tell them “one type of crystal rotates light one way, the other another way”, but still can’t meaningfully make a distinction between left and right. You cannot describe “clockwise” or “counterclockwise” without having earlier agreed on what is left and what is right.
The beta decay experiment, on the other hand, definitely shows a very clear distinction between left and right that doesn’t depend on defining “left” and “right” beforehand, and you can use it to make an unambiguous definition.
Can’t you also have a magnetic field going from up to down, fire electrons away from you and see which direction they turn?