If you’re a chess player, wouldn’t you file Gladys Knight under N? 
In personal filing systems the best rule is to put stuff where you will look for it later. I would file GKatP under K but I would also file the Plastic Ono Band under L, Ian Anderson under J, and John Cougar under M.
Like others have said, list things so you can find them. Whatever works most easily for you. If you want to put Gladys Knight and the Pips under “K,” go for it. It’s your system.
I identify mine the same way as Ximenean. I drop the “The” from band names, and sort individual artists by first name. So mine would look like:
Aerosmith
Beatles
Bee Gees
Billy Joel
Bruce Springsteen
Cars
Chicago
David Bowie
Eagles
Eddie Money
just to give an example. But that’s just me. It’s your music, so just do it however you prefer.
It is your system of alphabetization, therefore you may do whatever you like within that system, however wrong it is. That is my brief nod to relativism.
It is one of my pet peeves with computers that we humans have to adapt our systems to their stupidity.
Case in point - your above list should be alphabetized as follows -
Aerosmith
The Beatles
The Bee Gees
David Bowie
The Cars
Chicago
The Eagles
Billy Joel
Eddie Money
Bruce Springsteen
but our organizing tool can’t handle that. So, rather like changing the rules to suit the stupidest kid in the class, we have to go through this kind of shit -
Aerosmith
Beatles, The
Bee Gees, The
Bowie, David
Cars, The
Chicago
Eagles, The
Joel, Billy
Money, Eddie
Springsteen, Bruce
or put up with it being incorrect. Why can’t we have the names correct and be able to tell the computer which part of the name to sort by? Other than an institutionalized stupidity is almost impossible to change…
I freely admit to being a pedant, but if someone working under me alphabetized anything by first name, they would be fired.
And that is one of many reasons why I hate the CD database…
Sort everything by the first letter of the first name, individual names included, the the exceptions of ‘A’, ‘An’, and "The’ as first words.
You’re welcome.
I use itunes. and sometimes WMP
I don’t really care how they decide to order the music. If I want to hear Gladys Knight And The Pips I select the artist heading and start typing “Glad” in the search box before I even get to the “ys” I have all their albums listed. If it is a song I am looking for I do the same with the song heading highlighted. Just tap out a few keys. Works with WMP and Itunes probably a bunch more. I find shit fast that way. It is really the only way I look for music on my computer. No real need for me to organize my real CDs after they are digitized. So off in a pile somewere or sold/given away they go.
Wouldn’t that be “Knight, Gladys and Pips, the” using the OP’s naming scheme? 
I drop ALL "The"s but I’d keep the “The” in “Gladys Knight & The Pips” for absolutely no good reason, “&” instead of “and” just to keep the filename shorter.
My MP3 player doesn’t understand that when I want to just listen to Elvis Costello I also want to hear Elvis Costello & The Attractions. So The Attractions get no love from me.
Gary T’s “Eric Burdon and the Animals” would just be “Animals” (using my system) works for me. Since I rarely care what album a song is from I put “w Eric Burdon” in the album field. This works really nice when it’s a band with a special guest sitting in, say “(The) Specials” but with Desmond Dekker doing the lead vocals. This especially comes up with jazz recordings where it’s X’s quartet but with Y on Z instrument.
CMC fnord!
Ya know you can turn that [del]fucking annoying shit[/del] useful feature off, right? WMP>Tools>Options>Library>un-check the ones in the bottom box.