Not arbitrary. My Year 7 teacher explained it thus. The root Latin word for “differ” and hence “different” is essentially the same as that for diverge, deviate (as a verb), etc. it contains a sense of separation, of moving away. Logically, you can’t move away “to” something that you’re moving away “from”. It’d be like saying you’re separating “to” your wife, or that two cars on the highway diverged “to” each other.
Data point-Australia, 1971.
Perhaps the “to” form is an elision of phrases of the form “X is different compared to Y”. While some might prefer “with” in that usage, it is not illogical to use “to” as a preposition to follow comparison, because comparison requires bringing things together, conceptually at least. But if you ditch the word “compared”, perhaps out of a sense that it is unnecessarily verbose, you get “different to”.
That’s interesting. It seems to me that this suggests a reasonable account of what may be happening.
In the case of diverge, it can still literally mean “move away” or “move apart” in modern English. And I think that we would surely never say divergent to when using it to mean physically moving away/apart. But divergent to is still common, at about 10% the frequency of divergent from according to google ngram, presumably when it means “not the same as”.
Unlike divergent, in modern English different no longer retains the sense of its etymological root. And I don’t think its etymology is in the forefront of most people’s minds, i.e. the original metaphor of “moving apart” is lost - in modern English it’s now just a word that means “not the same”. So perhaps that explains why historically it was almost exclusively different from, but some dialects seem to be becoming more permissive of the different to variant.