The Taroko Gorge is well worth seeing. Catch the train and don’t drive the “highway” yourself, it’s pretty hair-raising. The little temple is lovely and the tunnels and gorge are spectacular. The human cost of building that road beggars belief. The local Polynesian (sic) culture is worth checking out, too. It was freaky to hear the locals singing in harmony exactly like Tongans.
The National Palace Museum is amazing. If you go, I’d love an answer to this question I asked here in 2002. That came up in searching for previous threads on this topic, which yielded this one, which is full of good stuff.