Reply
 
Thread Tools Display Modes
  #1  
Old 05-15-2020, 06:52 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359

Why is a truly random "shuffle" musical function so difficult to execute?


**Feel free to split hairs about if I mean "random" or if I mean "varied."**

Since at least 2011, I've always been flabbergasted at the inability of any of my music devices being able to generate a truly random play list.

Lately I use a music playing app and listen to music through my phone. I have some odd 200 bands, some with more tracks than others, but several thousand songs.

All this week I've put my entire track list on shuffle. I am aware of the shuffle/turn off shuffle/turn shuffle back on trick to get your shuffle playlist out of a loop.

My playlist is indeed different every day, but it gravitates to the same few bands and tracks. There are at least 5 songs that I hear every time I shuffle during the first 20 minutes or so. Likewise, it will play one band's full album in a random order before playing the second half of my library.

Why is it so hard to really shuffle a play list? Is there a trick I don't know?
  #2  
Old 05-15-2020, 07:02 PM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
First of all, don't think you need "truly random" numbers here, just a pseudorandom function that mixes things up.

Beyond that, it is by no means hard to shuffle a list, so the question is whether turning shuffle off and then on really results in a re-shuffle. I would test it a few times. If you have and the same songs are at the beginning each time, either it is a bug (certainly possible), or else the "shuffle" function is not a random shuffle at all, but rather some ad hoc algorithm that does things like keep the tracks of an album together.

Last edited by DPRK; 05-15-2020 at 07:04 PM.
  #3  
Old 05-15-2020, 08:12 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359
Quote:
Originally Posted by DPRK View Post
First of all, don't think you need "truly random" numbers here, just a pseudorandom function that mixes things up.

Beyond that, it is by no means hard to shuffle a list, so the question is whether turning shuffle off and then on really results in a re-shuffle. I would test it a few times. If you have and the same songs are at the beginning each time, either it is a bug (certainly possible), or else the "shuffle" function is not a random shuffle at all, but rather some ad hoc algorithm that does things like keep the tracks of an album together.
I used to be able to "fool" iTunes back when that was my main way of listening: I'd sort songs by track length then "shuffle" that list, it tended to un-bunch a lot of bunched-up data points and would result in a more natural listening experience. With my app, Pi Music Player, the shuffle all tracks function is a little hit-and-miss.

But you're right. I don't need to hear one of every band before any band repeats or anything of that nature, but hearing 4 tracks off one album within an hour when I have thousands of hours of music, the illusion of randomness gets lost.
  #4  
Old 05-15-2020, 10:29 PM
Voyager's Avatar
Voyager is offline
Charter Member
 
Join Date: Aug 2002
Location: Deep Space
Posts: 47,732
Quote:
Originally Posted by dontbesojumpy View Post
I used to be able to "fool" iTunes back when that was my main way of listening: I'd sort songs by track length then "shuffle" that list, it tended to un-bunch a lot of bunched-up data points and would result in a more natural listening experience. With my app, Pi Music Player, the shuffle all tracks function is a little hit-and-miss.

But you're right. I don't need to hear one of every band before any band repeats or anything of that nature, but hearing 4 tracks off one album within an hour when I have thousands of hours of music, the illusion of randomness gets lost.
I've had the same experience using an Android app. 4 songs from one album once I could see, but the standard pattern was to concentrate on a few albums with a few other songs interspersed and then move to some new albums to play a few songs from. I could move it to new albums by manually playing a song from one.
I think there are some old threads about this very problem.
  #5  
Old 05-15-2020, 11:18 PM
Thudlow Boink's Avatar
Thudlow Boink is offline
Charter Member
 
Join Date: May 2000
Location: Lincoln, IL
Posts: 29,024
Quote:
Originally Posted by DPRK View Post
it is by no means hard to shuffle a list
WAG: it's not hard to shuffle a list, but the problem is that your music collection isn't stored in the form of a one-dimensional list, but in a more complex structure. So there more involved in randomly picking a song than just (pseudo)randomly picking a number from 1 to n.
  #6  
Old 05-15-2020, 11:38 PM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
Quote:
Originally Posted by Thudlow Boink View Post
WAG: it's not hard to shuffle a list, but the problem is that your music collection isn't stored in the form of a one-dimensional list, but in a more complex structure. So there more involved in randomly picking a song than just (pseudo)randomly picking a number from 1 to n.
Quite true, but "shuffle", as opposed to "random play", is supposed to start by building a list, which could be your entire collection or a subset like everything by a given artist or in a given genre, and then shuffle that playlist like a deck of cards. Either way ISTM a one-dimensional list gets constructed as the first step, unless I have misunderstood the meaning of "shuffle".
  #7  
Old 05-15-2020, 09:32 PM
suranyi is offline
Charter Member
 
Join Date: Sep 2000
Posts: 8,030
I donít know, my iPod touch 6th generation seems to work pretty well. I have thousands of songs in it. Iíve been using shuffle a lot lately and havenít noticed excessive repetition.
__________________
Right now, itís Girlsí Generation. Tomorrow, itís Girlsí Generation. Forever, itís Girlsí Generation!
  #8  
Old 05-15-2020, 09:59 PM
Jackmannii's Avatar
Jackmannii is offline
Guest
 
Join Date: Oct 2000
Location: the extreme center
Posts: 33,400
As a veteran of iPod shuffle listening, I've always been struck by the relative non-randomness of the shuffle setting. The iPod gremlins are way overly fond of certain of my albums (which are not sorted according to a rating system).

I mildly enjoy a two-disc album of Hispanic versions of rock hits ("Los Nuggetz"), but I don't need to hear a song off that album for every hour or two of listening, when a truly random shuffle would have one such song materializing at best every couple of days.
  #9  
Old 05-15-2020, 10:05 PM
SmartAleq's Avatar
SmartAleq is offline
Guest
 
Join Date: Sep 2005
Location: PDXLNT
Posts: 5,843
This is why I've bought about six sequential Sony car decks in a row--they actually have a really good shuffle algorithm. I usually have either a 32 or a 64GB USB drive plugged in with the whole thing set on shuffle > device and I hardly ever get repeats, as one would expect with several thousand tracks to work through. Anyway, it's pretty good.

On my phone I recently installed Folder Player and it's also got a decent shuffle to it. Good enough for government work, anway.

FooBar does good shuffle too.
  #10  
Old 05-16-2020, 07:59 AM
Bumbershoot is offline
Guest
 
Join Date: May 2007
Location: New Mexico
Posts: 3,653
Here's what I've been doing lately to get a more or less random selection- I go to my alphabetical song list (~4000 songs) and pick a letter, maybe G or R or whatever. Then pick one of the tunes and hit Play. Shuffle is turned off. The only thing the songs have in common is they all start with G (until they all play, then H comes up). It works pretty well for me.
  #11  
Old 05-16-2020, 09:19 AM
Chronos's Avatar
Chronos is offline
Charter Member
Moderator
 
Join Date: Jan 2000
Location: The Land of Cleves
Posts: 88,635
It's really, really easy to shuffle a list randomly (or at least, randomly enough). The problem is, that's really unpopular with customers, many of whom will complain that it's "not random enough". So the makers of music players instead come up with algorithms that are designed to give what most customers want when they say "random", and that's a much harder problem (for starters, you have to figure out what customers actually want, when you can't rely on what they say).
  #12  
Old 05-16-2020, 10:19 AM
Jet Jaguar is offline
Guest
 
Join Date: Jan 2001
Location: Monster Island
Posts: 1,659
Yes, I recall reading about how much work went into the shuffle feature of some online streaming service (Pandora IIRC, but it might have been Spotify). The problem is that humans are really good at finding patterns in randomness, and the challenge is for the computer to look for those patterns and preemptively break them up.

My own workaround is that I use a player on my phone (Rocket Player) that has dynamic, rule-based playlists. I usually add a rule to only play songs that haven't been played in the last 7-30 days, depending on the size of the playlist, and once a song is played it drops off the list for the duration. I get a constant turnover of music that way.

Last edited by Jet Jaguar; 05-16-2020 at 10:24 AM.
  #13  
Old 05-16-2020, 10:57 AM
DesertDog is offline
Charter Member
 
Join Date: Oct 2002
Location: Mesa, Ariz.
Posts: 6,484
Quote:
Originally Posted by Chronos View Post
It's really, really easy to shuffle a list randomly (or at least, randomly enough). The problem is, that's really unpopular with customers, many of whom will complain that it's "not random enough". So the makers of music players instead come up with algorithms that are designed to give what most customers want when they say "random", and that's a much harder problem (for starters, you have to figure out what customers actually want, when you can't rely on what they say).
Well, I hope they do a better job of it than the folks at I -heart- radio do. If I bring up, say, The Beatles channel it will play one (1) Beatles tune then start throwing up songs at random that go farther and farther afield and play another Beatles tune an hour later, if I'm lucky. Some bands have a rather thin catalog and doing so would be more understandable in such cases, but The Beatles? You're not even trying.

They keep asking my to subscribe and give them money. Hell, no, I'll stick with Alexa instead.
  #14  
Old 05-16-2020, 12:17 PM
Mark Finn is offline
Guest
 
Join Date: Dec 2019
Posts: 101
I sort by title. That's about as random as you can get. Songs with digits in the title (e.g. "500 Miles") come up first.
  #15  
Old 05-16-2020, 08:53 PM
Deeg is offline
Member
 
Join Date: Jun 2008
Posts: 3,707
Quote:
Originally Posted by DesertDog View Post
Well, I hope they do a better job of it than the folks at I -heart- radio do. If I bring up, say, The Beatles channel it will play one (1) Beatles tune then start throwing up songs at random that go farther and farther afield and play another Beatles tune an hour later, if I'm lucky. Some bands have a rather thin catalog and doing so would be more understandable in such cases, but The Beatles? You're not even trying.
I don't know anything about I-heart-Radio but I've worked on vaguely similar projects. I assume in their case they don't want to be random. I'm guessing that some songs (like any from The Beatles' catalog) are more expensive than others. They have to balance playing the songs that you like as cheaply as possible, thus The Beatles get played less than you like. What happens if you select a less popular (and possibly cheaper) band?
  #16  
Old 05-16-2020, 05:21 PM
Mangetout's Avatar
Mangetout is offline
Member
 
Join Date: May 2001
Location: England
Posts: 58,242
Quote:
Originally Posted by Chronos View Post
It's really, really easy to shuffle a list randomly (or at least, randomly enough). The problem is, that's really unpopular with customers, many of whom will complain that it's "not random enough". So the makers of music players instead come up with algorithms that are designed to give what most customers want when they say "random", and that's a much harder problem (for starters, you have to figure out what customers actually want, when you can't rely on what they say).
Yep, this is one of those problems where the customer thinks they are being specific when they describe what they want as 'not like that'.
  #17  
Old 05-16-2020, 10:28 AM
Deeg is offline
Member
 
Join Date: Jun 2008
Posts: 3,707
I'm a programmer and it's very easy to randomly shuffle a list. The problem is that humans have a terrible sense of what is truly random.

I know a professor who makes his students flip a coin 50 times and write down the results (HTHHH...). He then tells them to make up another sample of 50 flips and try to make it random. He can then look at the two samples and immediately tell which one represents the real coin flips and he is almost always right. His trick: he looks for the sample with the most consecutive heads or tails. In a random sample of 50 coin flips it is very unlikely that there will NOT be a run of 6 (?) or more consecutive values. Students, however, think that 6 in a row is not really random and almost never have such a sample.

When you're listening to your shuffled list of 1000+ songs you will eventually come to a section equivalent to 6 heads in a row and in the moment you'll think the random shuffle sucks but it's just an expected outcome of truly random shuffles.

Coming up with an algorithm that humans <think> is random is much harder because it's a psychological problem, not a mathematical/programming problem. You mentioned "varied" in the OP but what does that mean? If you have two songs by the same band in a row, one a rocker and one a ballad, is that varied? What about about the same song by two different bands? It's an interesting problem but likely has no solution that most people would agree on.
  #18  
Old 05-16-2020, 06:00 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359
Quote:
Originally Posted by Deeg View Post
I'm a programmer and it's very easy to randomly shuffle a list. The problem is that humans have a terrible sense of what is truly random.

I know a professor who makes his students flip a coin 50 times and write down the results (HTHHH...). He then tells them to make up another sample of 50 flips and try to make it random. He can then look at the two samples and immediately tell which one represents the real coin flips and he is almost always right. His trick: he looks for the sample with the most consecutive heads or tails. In a random sample of 50 coin flips it is very unlikely that there will NOT be a run of 6 (?) or more consecutive values. Students, however, think that 6 in a row is not really random and almost never have such a sample.

When you're listening to your shuffled list of 1000+ songs you will eventually come to a section equivalent to 6 heads in a row and in the moment you'll think the random shuffle sucks but it's just an expected outcome of truly random shuffles.

Coming up with an algorithm that humans <think> is random is much harder because it's a psychological problem, not a mathematical/programming problem. You mentioned "varied" in the OP but what does that mean? If you have two songs by the same band in a row, one a rocker and one a ballad, is that varied? What about about the same song by two different bands? It's an interesting problem but likely has no solution that most people would agree on.

This doesn't really apply directly because flipping a coin has two possible outcomes vs a randomized playlist should have millions if not billions of variable outcomes. I guess based on the pure semantics of it, I could "randomly" pick every song in a row as they appear on each album in sequential order. This is tantamount to mining the raw ore and metals necessary to create a car, putting all the raw ingredients in a big bag, shaking it until they "randomly" combine to create exactly an Oldsmobile Cutlass Supreme, just by coincidence.

Replaying the same few songs within the first hour of every single "shuffle" session isn't random, it's a buggy code one would think.

The problem is this has been an ongoing issue across all media platforms and players for just about everyone I've ever talked to on the topic. By pooling picks of the same artists/albums early and often in a shuffle, it means other bands will bottleneck into the same kind of pools towards the end of the shuffle resulting in clusters of similar songs.

I do understand the issue has to do with "what truly random means" vs "what human expectations of random are vis-ŗ-vis listening experience, but there has to be a happy middle ground??

Last edited by dontbesojumpy; 05-16-2020 at 06:03 PM.
  #19  
Old 05-16-2020, 09:01 PM
Deeg is offline
Member
 
Join Date: Jun 2008
Posts: 3,707
Quote:
Originally Posted by dontbesojumpy View Post
This doesn't really apply directly because flipping a coin has two possible outcomes vs a randomized playlist should have millions if not billions of variable outcomes. I guess based on the pure semantics of it, I could "randomly" pick every song in a row as they appear on each album in sequential order. This is tantamount to mining the raw ore and metals necessary to create a car, putting all the raw ingredients in a big bag, shaking it until they "randomly" combine to create exactly an Oldsmobile Cutlass Supreme, just by coincidence.
I think you're vastly underestimating the chances of duplicate songs and likely overestimating (via confirmation bias) how often it happens. What music device are you using? It would be interesting to see if they publish their shuffle algorithm.

Quote:
Originally Posted by dontbesojumpy View Post
The problem is this has been an ongoing issue across all media platforms and players for just about everyone I've ever talked to on the topic. By pooling picks of the same artists/albums early and often in a shuffle, it means other bands will bottleneck into the same kind of pools towards the end of the shuffle resulting in clusters of similar songs.
I think this is proof that it's not the shuffle algorithms but human perception. What's more likely, that every shuffle algorithm has the same king of bug? Or that we're just bad at judging what is truly random?
  #20  
Old 05-16-2020, 10:03 PM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
Quote:
Originally Posted by Deeg View Post
I think you're vastly underestimating the chances of duplicate songs and likely overestimating (via confirmation bias) how often it happens. What music device are you using? It would be interesting to see if they publish their shuffle algorithm.


I think this is proof that it's not the shuffle algorithms but human perception. What's more likely, that every shuffle algorithm has the same king of bug? Or that we're just bad at judging what is truly random?
He or she said that, out of thousands of songs, 5 of them invariably appear during the first 20 minutes. That does not sound like a random shuffle.
  #21  
Old 05-17-2020, 10:46 AM
pulykamell is offline
Member
 
Join Date: May 2000
Location: SW Side, Chicago
Posts: 49,456
Quote:
Originally Posted by DPRK View Post
He or she said that, out of thousands of songs, 5 of them invariably appear during the first 20 minutes. That does not sound like a random shuffle.
But that could be confirmation bias. My first thought was exactly Deeg's that humans are just bad at judging what is "random."
  #22  
Old 05-17-2020, 04:05 PM
Ike Witt's Avatar
Ike Witt is offline
Charter Member
Charter Member
 
Join Date: Aug 1999
Location: Lost in the mists of time
Posts: 15,574
Quote:
Originally Posted by Deeg View Post
I'm a programmer and it's very easy to randomly shuffle a list. The problem is that humans have a terrible sense of what is truly random.
This article agrees with you.
Quote:
Five thousand songs, and it brings up three tracks by the same artist in quick succession? Come on now, that can't be right. This is supposed to be random!

Except, of course, it is random. Our brains just don't like it.
  #23  
Old 05-17-2020, 05:29 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359
Quote:
Originally Posted by Ike Witt View Post
This article agrees with you.
Again, it's well established that, technically speaking, having a band's full album play songs 1-10 in a row out of a giant library is semantically truly random. It's just utterly useless as a shuffle function for a music player.

In my case, having the same few bands repeat up to 4 times within the first 50 tracks is TRULY RANDOM, but also total garbage as a "shuffle" function. My sentience allowing me to notice this isn't really the problem. It's a lousy shuffle.
  #24  
Old 05-16-2020, 12:25 PM
Chronos's Avatar
Chronos is offline
Charter Member
Moderator
 
Join Date: Jan 2000
Location: The Land of Cleves
Posts: 88,635
It's understandable how that happens: Most I Heart Radio stations themed on a particular band have songs by that band, and songs that are similar to songs by that band. But the Beatles have songs that are similar to everything, so anything is similar to some Beatles song or another, and so a Beatles channel can include nearly any song at all.

While that might be a reasonable approach for most bands, they should maybe consider a different approach for that particular band. I mean, if any band deserves special treatment, it's the Beatles.
  #25  
Old 05-16-2020, 01:03 PM
Triskadecamus is offline
Charter Member
 
Join Date: Oct 1999
Location: I'm coming back, now.
Posts: 7,680
I did a "playlist" of songs alphabetically by title, and then did shuffles on that. Problem is, I don't like all the songs on my collection. Still end up hitting next. Not hitting next is the goal, right? Telepathy is the only answer. But who wants Apple to be able to do that?
  #26  
Old 05-16-2020, 03:34 PM
Chronos's Avatar
Chronos is offline
Charter Member
Moderator
 
Join Date: Jan 2000
Location: The Land of Cleves
Posts: 88,635
If you don't like some songs in your collection, surely the solution is to take them out of your collection? Or if, for some reason, you want to keep it and just never listen to it, make a playlist that's all of the music you own except for the few you don't like?
  #27  
Old 05-16-2020, 10:46 PM
Snarky_Kong is offline
Guest
 
Join Date: Oct 2004
Posts: 8,610
I think it's significantly more likely to be confirmation bias than it is that Apple can't randomize a list.
  #28  
Old 05-17-2020, 08:33 AM
Chronos's Avatar
Chronos is offline
Charter Member
Moderator
 
Join Date: Jan 2000
Location: The Land of Cleves
Posts: 88,635
Quote:
Quoth dontbesojumpy:

I do understand the issue has to do with "what truly random means" vs "what human expectations of random are vis-ŗ-vis listening experience, but there has to be a happy middle ground??
The happy middle ground between "what people actually like" and "truly random" is right exactly on "what people actually like". Any middle ground actually between those two would inherently not be the "happy ground".
  #29  
Old 05-17-2020, 08:57 AM
Ludovic is offline
Charter Member
 
Join Date: Jul 2000
Location: America's Wing
Posts: 31,199
People aren't monolithic. For instance, some people might enjoy a shuffle concentrating heavily on tracks the user has already played heavily while some people might enjoy more true randomness because if they wanted to play their most beloved tracks they wouldn't use shuffle.
  #30  
Old 05-17-2020, 10:02 AM
jjakucyk is offline
Guest
 
Join Date: May 2016
Location: Cincinnati
Posts: 551
My understanding of the way Apple has always programmed iTunes, iPods, and iPhones is that they would take the full playlist and shuffle the order that would play through until the end, at which point the whole list would start over with a new shuffle order. That limits calculation to one time and eliminates the possibility of duplicate songs being played back-to-back or very close to one another, which could happen if a truly random selection was made every time the next song came up. This may not apply to the newer iterations of the Music app and streaming services though.

So I would argue that while it's not a truly random selection, it feels more random like in Deeg's example about flipping coins because there's no possibility of repeats, and you're also guaranteed to hear everything. Where it can break down is when you get to the end of the list. While the chance of a repeat or recently played song coming up again is much smaller, it can still happen. Once I was listening on an older iPod Shuffle, and after skipping one or two songs it kept coming back to the same ones over and over. It took a moment to realize I had come to the end of my playlist and it didn't have a chance yet to build the new shuffled order, so it was kind of stuck. Those Shuffles don't have a whole lot more processing power than what's needed to play the music, so it couldn't build the new playlist order in just a couple seconds.
  #31  
Old 05-17-2020, 10:59 AM
markn+ is offline
Guest
 
Join Date: Feb 2015
Location: unknown; Speed: exactly 0
Posts: 3,318
Even playing short pop songs, there's only time to play seven or eight in 20 minutes. If "at least 5" of them are always the same, that would indeed be stunningly nonrandom. But perhaps there is a bit of hyperbole in that report.
  #32  
Old 05-17-2020, 12:00 PM
Mijin's Avatar
Mijin is offline
Guest
 
Join Date: Feb 2006
Location: Shanghai
Posts: 9,712
I used to work in video game development, and lots of "random" numbers had to be fudged somewhat to *feel* random to a human. If the player got a bad result two or three times in a row, they may feel the system is cheating. So I also put it down to human psychology / confirmation bias here.

------------------------------

OTOH it's possible there's a flaw in the shuffle algorithm. But they would really have had to drop the ball to screw up one of the most basic elements of a music player. And the OP asks why such a music function is badly implemented on any music device, to which I'd dispute the premise. I have a number of music devices (I'm oldskool), that shuffle great.

Last edited by Mijin; 05-17-2020 at 12:01 PM.
  #33  
Old 05-17-2020, 12:25 PM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
Quote:
Originally Posted by Mijin View Post
I used to work in video game development, and lots of "random" numbers had to be fudged somewhat to *feel* random to a human. If the player got a bad result two or three times in a row, they may feel the system is cheating.
It sounds as though none of your playtesters ever walked into a casino...
  #34  
Old 05-17-2020, 12:48 PM
Mijin's Avatar
Mijin is offline
Guest
 
Join Date: Feb 2006
Location: Shanghai
Posts: 9,712
It's a very common feature in games though.

A triple-A game is unlikely to play all its random numbers "straight". Because people are wont to rage quit as soon as two 90% shots miss in a row.
Yes it's irrational, but OTOH look at it this way: why assume that the most fun experience is always going to be the most mathematically fair one? Designers obviously aim for the former.

Last edited by Mijin; 05-17-2020 at 12:49 PM.
  #35  
Old 05-17-2020, 01:48 PM
vonespy is offline
Guest
 
Join Date: Jul 2013
Posts: 98
Some of the perception of non-randomness can be explained by the Gambler's Fallacy. I think part of the issue is how most devices come up with a "random" playlist. IIRC, the playlists are generated in a stateless manner to where they are re-generated on a regular basis for various reasons. If the playlists were more stateful (similar to what jjakucyk describes) and stuck in the device until every piece of music was played, then things would seem more random. Instead the playlists get re-generated frequently which means the odds of hearing, say, Afternoon Delight twice in the same week are reasonable. And once we hear Afternoon Delight a couple of times in a week's time, we start doubting our sanity ... er... the randomness of not-so-random devices.

Not to derail but an interesting, just barely tangent to this would be Benford's Law. It completely blew my mind when I first heard about it.

Last edited by vonespy; 05-17-2020 at 01:52 PM. Reason: Added jjakucyk credit
  #36  
Old 05-17-2020, 01:05 PM
AnalogSignal is offline
Member
 
Join Date: Feb 2010
Posts: 1,739
This topic was covered in detail in Steven Levy's book about the iPod The Perfect Thing.

The author was convinced the iPod shuffle algorithm wasn't working properly with the same songs being repeated and "Aretha Franklin forever waiting in the wings" never getting a chance to sing. In this book from 2006, the author complained to Steve Jobs who in turn checked with the Apple programmers to see if the shuffle was implemented properly. The conclusion was that shuffle was working correctly and it was dumb luck that certain songs were being repeated and others were never played.
  #37  
Old 05-17-2020, 02:03 PM
Chronos's Avatar
Chronos is offline
Charter Member
Moderator
 
Join Date: Jan 2000
Location: The Land of Cleves
Posts: 88,635
The way I think of Benford's Law: First, suppose that I pick a (uniform) random number between 1 and 9. What are the odds that the first digit is a 1? 1 in 9. What are the odds that it's a 9? The same 1 in 9.

But now suppose that I pick a random number between 1 and 20. What are the odds now that the first digit is is a 1? Much higher, now: 11 in 20. But the odds that the first digit is a 9 are much lower: 1 in 20.

Or what if I pick a random number between 1 and 50? Both of those decrease now, 11 in 50 for 1, 1 in 50 for 9. But 1 is still much more likely than 9.

Pick from 1 to 99? Now it's exactly balanced again, 11 possibilities for each... for that one specific case. But from 1 to 98 or 1 to 100 would again favor 1 over 9.

And now, what if I pick a random number between 1 and n, but you don't know what n is? Well, if 1 is sometimes exactly as common as 9, and sometimes more common, but never less common, then you can say that the first digit should be 1 more often than it is 9.
  #38  
Old 05-17-2020, 02:28 PM
vonespy is offline
Guest
 
Join Date: Jul 2013
Posts: 98
Chronos, wow, excellent explanation!
  #39  
Old 05-17-2020, 02:51 PM
Kimble's Avatar
Kimble is offline
Guest
 
Join Date: Jun 2001
Location: Nashville, TN
Posts: 553
I don't think this is what's happening to dontbesojumpy, but I offer this as a documented example of bad shuffling:

I told my Echo Dot, "Alexa, play the Indie Dance Essentials playlist from Apple Music on shuffle mode." This is an Apple-created playlist with 100 tracks.

It started with track 1. The next tracks were 4, 5, 7, 10, 6, 2, 11, 3, 9, 8.
After that: 15, 13, 21, 19, 14, 17, 20, 18, 12, 16.
Then: 30, 25 ... and I stopped after that. That is very not random.

I couldn't get playing my own playlists to work on my Echo Dot. I don't have Amazon Music, so I don't know if Alexa does a better job with her native music ecosystem.

Hitting Shuffle and then play in iTunes in Windows on an Apple Music playlist started with 68, 36, 25, 84, 98, 83, 56, 59, 87, 29, 18, 11, 53, 21, 60. That seems random enough to me.
  #40  
Old 05-17-2020, 03:18 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359
Is there an app or simple way to log songs and play counts? I used to scrobble to last.fm but I don't think that works. I'd like to bump my observations up from anecdotal to statistical.

When I'm working--the time when I listen to shuffle the most--I don't really have time to make notes on what's playing. So anything that might track that could be helpful.

I just opened my player to a new shuffle. In the first 8 tracks played, 2 were off the same album (Barkmarket, L. Ron).

Track 12 was Superdrag, as was 15 and 25 *same album*.

Track 23 was off Gleemer's Anymore, as was 35 and 38.

Tracks 30 and 37 were off the same album. There were other pairs (many other pairs, or trios) in the first 50 songs played, but you get the point.

I re-sorted by song title and fired up a new shuffle.

Tracks 1 and 4 were by the same band.

Track 8, 10 were by the same band, Cloudkicker,

This is what I mean. Hearing the same band/same album a few songs apart regularly. It seems to tend to bunch up.

New shuffle:

Tracks 7 and 9 are off the same album.

I could do this all day.

Again, this is where we devolve into the semantics of "what random means" vs what someone wants in a shuffle. This is why I brought up the term "varied," because when some people say they want "random" they really mean "a varied experience."

In my experience, once a band is played, my player seems to want to clump that band or that album together within a span of 20 songs or so, out of thousand and thousands of songs.

I'm using Pi Music Player v3.0.8.1

There's a "persist shuffle" setting that doesn't reshuffle after exiting the app. Not sure if that would help.

For reference I have over 108 hours of music to shuffle from.
  #41  
Old 05-18-2020, 05:13 AM
Asuka is offline
Guest
 
Join Date: Feb 2016
Posts: 1,638
I have a SanDisk Clip MP3 player with over 250 different songs I play on random as I jog.

In my 30 minutes of jogging despite being on completely random with 250 choices there's about 3 songs that have a 50% chance of showing up during those 30 minutes of jogging every single time.
  #42  
Old 05-18-2020, 05:37 AM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
Quote:
Originally Posted by Asuka View Post
I have a SanDisk Clip MP3 player with over 250 different songs I play on random as I jog.

In my 30 minutes of jogging despite being on completely random with 250 choices there's about 3 songs that have a 50% chance of showing up during those 30 minutes of jogging every single time.
If it is on "completely random", then each song played has 247/250 chances of not being one of the 3. If you have time for, say, 10 songs, that makes about an 11% chance of one of those songs showing up. If it's really 50% (not just a bad estimate), then, sure, there is a problem.
  #43  
Old 05-18-2020, 05:52 AM
Melbourne is offline
Guest
 
Join Date: Nov 2009
Posts: 6,313
As I recall, a standard card shuffle does NOT shuffle the cards into a good random order, and that it was well known there were predictable results. So maybe "shuffle" is a good description of OP's phone does ....

Anyway, unless somebody has credible, reliable, independent documentation, 'random shuffle' could mean anything. If the device is doing a "random walk" from the starting point, (which is one kind of reasonable approach that might please some people), then repetition and centering is highly probable.
  #44  
Old 05-18-2020, 06:34 AM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
Quote:
Originally Posted by Melbourne View Post
As I recall, a standard card shuffle does NOT shuffle the cards into a good random order, and that it was well known there were predictable results.
Correct, this is well-known among card players, and the folklore I always heard was to shuffle seven times for a thorough mix.

Quote:
So maybe "shuffle" is a good description of OP's phone does ....

Anyway, unless somebody has credible, reliable, independent documentation, 'random shuffle' could mean anything. If the device is doing a "random walk" from the starting point, (which is one kind of reasonable approach that might please some people), then repetition and centering is highly probable.
I consider it poor design if I do not have control over the parameters of the "random walk" and its underlying links, or if this functionality is not documented.
  #45  
Old 05-18-2020, 09:13 AM
Omar Little's Avatar
Omar Little is offline
Guest
 
Join Date: Apr 2000
Location: Within
Posts: 13,816
Quote:
Originally Posted by dontbesojumpy View Post
There are at least 5 songs that I hear every time I shuffle during the first 20 minutes or so.
What is the average length of your songs? Mine are typically 3-4 minutes. Based upon your description above, every time you shuffle play your library, the same 5 songs are played first or within the first 6-7 songs that are played. That doesn't sound correct.
  #46  
Old 05-18-2020, 08:36 PM
dontbesojumpy is offline
Guest
 
Join Date: Oct 2009
Posts: 1,359
I ran a new shuffle today and counted the first 30 tracks played. The approximate runtime for the first 30 tracks was a total of around 120 minutes.


Tracks 11, 13, 17, 18, 22 and 28 were all the same band.

Tracks 6, 10, 23, and 26 were all the same band (Pavement)

Tracks 8, 16 and 27 all the same band.

There are around 200 different bands to pick from, and almost half of the first 30 slots were occupied by only 3 different bands.
  #47  
Old 05-18-2020, 09:07 PM
Voyager's Avatar
Voyager is offline
Charter Member
 
Join Date: Aug 2002
Location: Deep Space
Posts: 47,732
I understand randomness. I was reading Knuth before most of you were born.
Obviously some bands or artists, which I have a lot of songs from, show up more often than others and often repeat. No problem there. Albums with lots of tracks (like Variations) show up more often than albums with few tracks. No problem there either.
But a consistent experience of several albums showing up again and again (similar to dontbesojumpy's experience,) shows something is not random. This is a shuffle, btw - tracks never repeat unless I have more than one of them stored.
It seems more likely that for this player they decided that when you hear one track from an album you are primed to hear another, and so distort the shuffle to provide it.
I have hundreds of albums and thousands of tracks, so while it happening once in a while is reasonable, it is not reasonable for it to happen consistently.
  #48  
Old 05-18-2020, 09:27 PM
GMANCANADA is offline
Guest
 
Join Date: Mar 2003
Location: Toronto
Posts: 581
This may be incorrect, but I was never under the impression that iTunes "shuffle" was random. Otherwise they would call it "random" play, not "shuffle" play.

I was under the impression that is analyses the songs using some sort of algorithm (as other posters have argued), maybe tempo, genre etc and plays them in groups based on that, siding between groups slowly so there is no jarring change from song to song.

My workout playlist on my iPhone has very wide range of music about +1000 songs, from 1950's rock to modern day with some classical thrown in. When I do a "shuffle" it tends to keep things in their genres and similar tempos. I assumed that was how it was supposed to work.

I'll get many songs in a row by similar artists (or even the same artist) in similar genres, then it will start to shift me to other artists and genres. I never get "Ride of the Valkyries" followed by Sex Pistols "God Save the Queen" followed by "You Can't Hurry Love." etc.
  #49  
Old 05-18-2020, 09:45 PM
Thudlow Boink's Avatar
Thudlow Boink is offline
Charter Member
 
Join Date: May 2000
Location: Lincoln, IL
Posts: 29,024
Quote:
Originally Posted by GMANCANADA View Post
This may be incorrect, but I was never under the impression that iTunes "shuffle" was random. Otherwise they would call it "random" play, not "shuffle" play.
One interpretation is that "shuffle" plays through all the songs on a CD or playlist once each, in a random order (like shuffling a deck of cards), while "random" picks songs randomly without worrying about whether or not they're being repeated. But I don't claim that this is how these terms are universally used.


Anyway, I found this article that says the same thing that Ike Witt's does: that Spotify changed their shuffle algorithm to be less random so that it would seem more random.

Last edited by Thudlow Boink; 05-18-2020 at 09:45 PM.
  #50  
Old 05-18-2020, 10:25 PM
DPRK is offline
Guest
 
Join Date: May 2016
Posts: 5,120
So, presumably, what the OP wants to know now is how to get a genuinely randomly (deck of cards) shuffled playlist in Pi Music Player...
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 05:00 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.

Send questions for Cecil Adams to: cecil@straightdope.com

Send comments about this website to: webmaster@straightdope.com

Terms of Use / Privacy Policy

Advertise on the Straight Dope!
(Your direct line to thousands of the smartest, hippest people on the planet, plus a few total dipsticks.)

Copyright © 2019 STM Reader, LLC.

 
Copyright © 2017