Kid Rock has a female drummer.
Nuts… that was my contribution.
Well, there’s Elastica. Mostly female band but only Justine sang, I think. Same with Bangles and Go-Go’s.
Justine was in Suede and Bret Anderson was the singer.
The Fall featured Brix on guitar, Mark E. Smith is the vocalist.
Session players: the aforementioned Sara Lee is everywhere (she was even in the B-52’s “Love Shack” video, and of course the great Carol Kane, on a number of 60s and 70s US hits.
Cait O’Riorden was the original bassist for The Pogues, although she did sing on a few songs, most memorably “I’m a Man You Don’t Meet Everyday.”
From 1997 to 2006 the Northern Irish indie band Ash had Charlotte Hatherley (now a moderately successful solo performer) on guitar.
She also sang backing vocals though, the part of the OP that makes this that much more difficult. I was going to suggest The Subways, who have Charlotte Cooper on bass, but she also sings backing vocals.
In a couple of recent TV performances, Mika has had a female drummer. No idea if she’s his regular drummer, or just some eye candy for the TV though.
I don’t know enough about the Talking Heads to know if she did sing with them, but at least some tracks had female vocalists when they reformed as The Heads. “The Damage I’ve Done” is a really great song by them.
HippyHollow, Elastica is the opposite of what I’m asking about, isn’t it? It sure sounds like a female singer to me, anyway.
And MeShell NdegeOcello did vocals on John Mellencamp’s “Wild Night.”
The reason I asked this question is there are lots of indie bands with a woman in them, but the vast majority have her be the singer. Off the top of my head the Smashing Pumpkins was the only band I could think of that didn’t have the girls sing, so I thought I’d ask to see what I was missing.
Skankin’ Pickle had Lynette Knackstedt on guitar and is listed as “guitar and vocals” but she most certainly is NOT their lead singer. Everyone else in the band, including the horn players, is listed as “[instrument], vocals”
Yeah, Elastica isn’t quite the fit for your OP. Justine’s previous band, Suede, is, though.
I’m pretty sure there was a woman in Pulp, and I only recall hearing Jarvis Cocker as lead vocalist…
The Heads did “Damage I’ve Done” with Johnette Napolitano, formerly of Concrete Blonde. It is a bad ass song.
Yes, Pulp had Candida Doyle.
Justine Frischmann was in Suede before they really hit the big time.
The Dandy Warhols have Zia McCabe on keyboards.
David Bowie’s bass player is Gail Ann Dorsey… I don’t think she sings on his recordings, but at his concerts she fills in Freddy Mercury’s part on “Under Pressure” and sounds amazing. I dont’ think she sings much otherwise for him, but I could be wrong.
Hippy Hollow
I already mentioned Carol Kaye in posting #16. (I misspelled her name as Kay instead of Kaye).
Carol Kane is someone else entirely.
I think Mika’s bass guitarist (or is it drummer) is a rockin’ British chick.
You always see his band when he performs live. 
The E. Street Band has featured two female violinists over time, Suki Lahav in the very old days and Soozie Tyrell quite recently.
She is credited with backing vocals (not that there are any obvious female vocals on anything I’ve heard.)
Not sure if any of thee have been posted yet, but…
Snowpony - Deb Googe (bass)
Imperial Teen - Jone Stebbins and Lyn Perko on bass and drums, respectively
Velvet Underground - Maureen “Mo” Tucker (drums) (Nico wasn’t really in the band, right?)
Butthole Surfers - Theresa Nervosa (drums), Cabbage (drums), Kathleen Lynch (dancer)
Beat Happening - Heather Lewis (drums)
Mock Turtles - Joann Gent (keyboards)
The Germs - Lorna Doom (bass)., Dottie Danger (Belinda Carlisle) early drummer
The Adverts - Gaye Advert (bass)
The Beach Boys - Carol Kaye (bass - “Pet Sounds” LP, "Good Vibrations " 45)
The Legendary Pink Dots - April Iliffe/Aradia (keyboard, glockenspiel)
The Rentals - Petra Haden (violin/synth), Cherielyn Westrich (synth)
Jejune - Arabella Harrison (bass)
Black Flag - Kira Roessler (bass - 1983-1985)
Dos - Kira Roessler (bass)
The Waitresses - Tracy Wormworth (bass)
The Evens - Amy Farina (drums
The Warmers - Amy Farina (drums)
X-Ray Spex - Lora Logic (Saxophone)
I think a few of these ladies may have done backing vox.
My BF named half of these, so if any are wrong, it’s his fault 
The Duchess - AKA Norma-Jean Wofford, Bo Diddley’s guitar player from 1961 -1966, as well as his previous guitar player, Peggy Jones.
Also, Nancy Wilson of Heart probably should count, even though she sings lead on a few songs and backup, she’s a well respected guitar player in her own right.
Mika’s drummer’s name is Cherisse Osei.
Meg sings on “In The Cold Cold Night”
Janis Tanaka, bassist for Fireball Ministry
Paz Lenchantin, bassist for A Perfect Circle, Zwan
They might have done some back-up vocals, but AFAIK neither did lead vocals.
Wendy and Lisa sang backup, didn’t they? And Jill Jones sang on the song “1999.”
Another likely candidate is Lori Goldston. Seems Creager played the European leg of the tour, but Goldston played the US leg and the MTV show.