I think the question needs to be broken down a bit. Why gold is valuable for ornamentation is a different question from why gold is valuable for industry is a different question from why gold is valuable as money.
Gold is valuable for ornamentation, I believe, primarily because it’s shiny and rare. The shininess argument, I think, tends to get short shrift, but this has a lot to do with it. There’s plenty of stuff as rare or more rare than gold, but that no one cares about because it’s ugly or it looks just like stuff that’s common. Gold stands out because of its shine and rarity, and thus, pretty much no matter what it is you’re looking at, you’re not going to turn your head away from it quite as quickly if it’s covered in gold.
Gold is valuable for industry, I believe, because of its conductivity and malleability, or its unique properties more generally. However, it’s expensive, as you’ve noted, and so most of the time, if something else will work, you just go with that, leaving gold only for when nothing else will do. Silver, for example is more conductive than copper, and so, silver would be a better choice for electrical wiring, except that it’s not that much more conductive than copper while costing about a hundred times as much. But in photography, silver is essential for developing film, and so you have no choice but to use that. Gold is similar except that examples spring to mind less easily.
Gold is valuable as money, or as a store of value, for many reasons, most of them linked to the ephemeral aspects of the idea of value. In order to be useful as money, it has to be something that everyone wants, at least most of the time. (Dying of thirst in the desert makes gold useless to you, but anything but water is going to be worthless to you in this situation.) So, the question becomes: Why is gold something that everyone wants? There’s a bit of a vicious circle thing going on here, which I think is what gets to most people. That is, if say 90% of the population of a given area likes gold, that makes it valuable to the other 10% as well since they know they can trade it with others for things they want and need. To see how the cycle begins, I think we need to look at the ornamentation bit again. Who gets to do the most decorating? The wealthy. Versailles looks a hell of a lot better than he hoses of the townspeople who lived nearby. Thus since the wealthy were the only ones who had the time and the energy and the means to decorate lavishly, gold is associated with the wealthy, and so, I believe, gold took on the idea of being something associated wealthy people, and over time, something associated with wealthiness, and then, over time, with wealth itself. So, the idea that gold is valuable because it has few purposes other than ornamentation makes a bit of sense to me. The fact that it doesn’t become nongold and can be split and combined but can’t be created essentially at will is part of the draw, of course, but that doesn’t explain the beginning of the vicious circle that makes it what it is. This tends to mean, in the end, that gold is valuable because it’s shiny and rare, as strange as that may seem.
I have no cites, but this is my reasoned guess.