When I saw the thread title, I thought you meant songs that were about the band/singer’s own fans, or about being a fan of that artist, and I thought of The Kinks’ “Rock and Roll Fantasy” (“Dan is a fan and he lives for our music / It’s the only thing that gets him by”).
Reading the OP, I see you’re asking for “songs that are about other musicians, from the point of a fan.”
There are plenty of examples from this thread from February: Songs About Other Musicians. But that thread is broader in scope than this one (some, but certainly not all, songs about other musicians are from the point of view of a fan).