Then there’s the reverse question: Why did they use to spell “Esthonia” and “Thibet” with the digraph ⟨th⟩ when they were never pronounced with [θ] or anything but plain [t]? It torments my mind not knowing whether there had once been a rational explanation now forgotten, or there never was a rational explanation in the first place and shit just happens randomly.
There was a rebel sect of Islam in the 9th century CE, the Qarmati rebellion. The Arabic plural name of the group is al-Qarāmitah. However, French Orientalists spelled the name “Carmathian” and that was formerly copied by English writers. This does have an intelligible reason for the h. In French, the word ending spelled -tian is always pronounced “-sian.” Inserting the h preserved the plosive [t] sound for French speakers. I cite this as an example of a deeply buried rational reason for the spelling even if it looks weird to us.
Another example is the Mohawk saint, Kateri Tekakwitha. In Mohawk, stop phonemes like /k/ and /t/ are voiced to [g] and [d], unless otherwise indicated. That’s what the h is for: to preserve the unvoiced sound of /t/. So her name is pronounced Gaderi Degagwita. It took some deep diving into Mohawk phonology to elucidate this h.
I can’t find any rational reason for “Esthonia” or “Thibet” no matter how you slice it.