Once upon a time, rank insignia was worn on epaulettes.
Company officers wore relatively plain epaulettes:
alone for a 2nd lieutenant,
with a bar for a 1st lieutenant,
with 2 bars for a captain.
Field officers wore fancier epaulettes:
alone for a major,
with a leaf for a lieutenant colonel,
with an eagle for a colonel.
Generals wore much fancier epaulettes with 1 to 4 stars.
Note that the 2nd lieutenant and the major both wore unadorned epaulettes. The shape of the epaulette itself was sufficient to distinguish them.
Some branches wore gold insignia on silver epaulettes, some wore silver insignia on gold epaulettes. Generals’ epaulettes were always gold, so their insignia has always been silver. At some point, the army switched everyone to gold epaulettes. So everyone’s insignia became silver.
In the mid-1800s, epaulettes were restricted to full dress uniforms. For everyday duty, the insignia was worn on rectangular shoulder straps like you see in Western movies and Civil War movies. Now there was a problem: Since the shoulder strap was identical for all commissioned ranks, there was no way to distinguish a major from a 2nd lieutenant. So they took the lieutenant colonel’s leaf and changed the color. Now the major had a gold leaf, and the 2nd lieutenand had a plain shoulder strap. Now the notion of silver outranking gold was semi-official.
When the army adopted khaki and olive drab field uniforms in the late 1800s, they dispensed with the big shoulder strap, and simply pinned the insignia to the collar or the shoulder. Now there was a problem: since the uniform was almost identical for all ranks, there was no way to distinguish a 2nd lieutenant from a buck private. So they took the 1st lieutenant’s bar, and changed the color. Hence the “butterbar”. Now the notion of silver outranking gold was firmly established.
The official rank insignia may be silver, but most of the other decorations on the uniform are gold (brass). The higher the rank, the more decorations.