It’s funny, because according to this article, it was first proposed to be called “Georgium Sidus” by William Herschel, the astronomer credited with discovering it. King George III had sent a large pension to Herschel following his discovery, but it was pointed out that naming celestial bodies after celebrities never “stuck”, and also “sidus” would refer to a star and not a planet. Herschel then started calling it “The Georgian Planet” which yeah, sounded pretty awful.
Others suggested that the planet be named “Neptune”, but that didn’t happen obviously, because that name went to the next planet instead. One other popular idea that took hold was to just name it “Herschel” after the man who discovered it, and it was referred to that name by many astronomers for years. (Could you imagine if the planets were named Mercury, Venus, Earth, Mars, Jupiter, Saturn, Neptune, and HERSCHEL?!)
Johan Elert Bode, who later would be the namesake of “Bode’s Law”, was the person who suggested the name “Uranus”. Uranus in Greek mythology was the father of Cronus (who the Romans named “Saturn”), and Cronus was a titan who gave birth to Zeus (“Jupiter” in Roman mythology). And that made a certain kind of sense that appealed to astronomers… Heading out from Earth, you have Mars (Ares in Greek mythology), whose father was Jupiter, whose father was Saturn, whose father was Uranus. And for that matter, Mercury and Venus were also children of Jupiter. (You have to skip Earth in this metaphor, but of course, it’s not like astronomers discovered the planet everybody was already standing on.) It kind of turned the solar system into a family tree, where each planet’s namesake god was descended from the next one.
The controversy over the name lasted a long time. For years the name of the planet was variously referred to as “Georgian Planet”, “Herschel”, and “Uranus”. In particular, the English were very attached to “Georgian” (after all, he was their king) and the French were fond of “Herschel”.
It wasn’t until after the discovery of Neptune in 1846 that the scientific community fully settled on Uranus, so the naming controversy lasted almost 70 years. (Note that the element “Uranium” was named in support of Bode’s naming of the planet Uranus, clearly this was a major political thing in the scientific community of the day.)
As to why Uranus and not Caelus? The answer is kind of silly (and maybe a bit apocryphal) but apparently Bode didn’t know that Caelus was the Roman name of the deity. Note that technically, Uranus is Ouranos in Greek, and “Uranus” was the Romanization of the name. Bode apparently thought that the Romanization of the name was the actual Roman name of the god, and thus made it consistent with other planetary names. You’d think that over the 70 years they debated the name, someone would have pointed out the error, but whatever.