OP got it right off the bat – it’s a mixed bag depending on the political situation of the specific entity.
Since many National Olympic Committees are independent NGOs or quasi-public entities rather than government offices, many athletes do not even have to march/stand under the legal colors/anthem of their government, so each NOC tells the OC what is it they want raised/played at awards time.
Puerto Rico, for instance, ever since 1952 uses La Borinqueña, which was adopted as the Commonwealth’s official anthem on that year. Every one of the US Non-Incorporated Territories by now has an officially-adopted anthem/state-song.
Before then, starting in the 30s we competed in Central American games, variously using the US flag (back pre-WW2 territories did not generally get flags of their own) or a banner of the Seal of Puerto Rico, and playing the US anthem if necessary. In Helsinki in 1948 we went under a “neutral” flag a-la Chinese-Taipei/Taiwan – a seal of PR under the Olympic rings – and would have used the Olympic anthem.
“Politically delicate” cases such as Chinese-Taipei (aka Taiwan in our real world), as mentioned by calm kiwi will go under a special neutral flag and Olympic Anthem. This was also the answer in '92 for the “Unified Team” of the CIS (former USSR).