With this way, you can “thumb” through your LPs “A to Z” while using the cover image as your visual clue. The on-edge lettering is formatted like a book, so that way goes L to R.
I think it comes from the days when people had piles of LPs leaning against the wall. Later we became “sophisticated” and put them onto shelving.
Just a WAG.