Cassie by Flyleaf
“Caroline” by Concrete Blonde (more complex than complimentary)
Jill Sobule had several–“I Kissed a Girl”; “Margaret”; “Karen by Night”. Again, more complex than complimentary.
That’s the first thing that popped into my mind when I read the thread title, but having read the OP, it’s not really honoring Jolene.
She introduced that song by thanking all the confused Katy Perry fans for buying her song.
Electricity by Joni Mitchell is about a woman who is trying to fix an electrical circuit. (I love this song, BTW. It’s in my top-five list of favorite Mitchell songs.)
Anneke van Giersbergen - Yalin; haunting song about a friend she lost to suicide.
“Damn I Wish I Was Your Lover”—Sophie B Hawkins
The first one that popped into my head was Adia by Sarah McLaughlin.
I also thought of one that a woman wrote about herself, Tough by Kellie Pickler. I think it fits with the idea of celebrating strength in women.
Any parent of daughters will appreciate Martina McBride’s country tearjerker “In My Daughter’s Eyes”: In My Daughter's Eyes - YouTube
Sara Bareilles’s “She Used To Be Mine,” from the musical Waitress, is a recent favorite of mine: Sara Bareilles - She Used To Be Mine (Official Video) - YouTube
Natalie Merchant wrote her song Wonder about a girl who was born with physical handicaps, whom Merchanthad met when she was a teenager, and worked at a day camp for special-needs children.
All great answers, friends. 
Hey, speaking of Patti Smith’s Horses and sisters, you remind me, the song “Kimberly” is addressed to Patti’s little sister, about holding her as a baby, with terrific tenderness and affection.
Stevie Nicks wrote a hell of a lot of them, comparatively speaking, for Fleetwood Mac. I cannot think of any other names as big as her who have so many songs written/sung by women, about other women.
Rhiannon
Sara
Gold Dust Woman
Gypsy
Stevie Nicks didn’t just sing them, she wrote them all.
I love Stevie. I had Sara played at my twin’s funeral. I don’t think there’s a more powerful female singer/songwriter than Stevie Nicks out there right now.
Mary Jane by Alanis Morissette is extremely moving. Sunny Came Home by Shawn Colvin is quite haunting too.
It’s not evident in the title, but “Red Bike” by Wendy and Lisa is about a woman.
“Ring of Keys” from Fun Home.
Other songs from the show, like “Changing my Major”
Well, I think it’s to honour women rather than a woman, but how about Hymn to Her by the Pretenders?
Surprising fact - assuming the Wiki entry is correct, this wasn't written by Chrissie Hynde.j
This was written by Nile Edwards and Nile Rodgers.
A couple of Indigo Girls songs stand out in this regard.
“Virginia Woolf” by Emily Saliers has the feeling I brought in the OP of being inspired by an illustrious woman.
“Nuevas Señoritas” by Amy Ray is a tender elegy to two indigenous women activists who’d recently been murdered, Ingrid Washinawatok (Menominee) and Marsha Gómez (Choctaw).
Joan Jett remade “Crimson and Clover” but didn’t change the pronouns so it is still about a woman.