# Why is a truly random "shuffle" musical function so difficult to execute? **URL:** \ **Category:** Factual Questions\ **Created:** [May 15, 2020, 11:52pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892 "2020-05-15T23:52:46Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Dr.Colossus](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.colossus/32/9423_2.png) [@Dr.Colossus](https://boards.straightdope.com/u/Dr.Colossus)\ **Post date:** [May 16, 2020, 11:00pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/21 "2020-05-16T23:00:12Z")
> [@Deeg](#): > > I’m a programmer and it’s very easy to randomly shuffle a list. The problem is that humans have a terrible sense of what is truly random. > > I know a professor who makes his students flip a coin 50 times and write down the results (HTHHH…). He then tells them to make up another sample of 50 flips and try to make it random. He can then look at the two samples and immediately tell which one represents the real coin flips and he is almost always right. His trick: he looks for the sample with the most consecutive heads or tails. In a random sample of 50 coin flips it is very unlikely that there will NOT be a run of 6 (?) or more consecutive values. Students, however, think that 6 in a row is not really random and almost never have such a sample. > > When you’re listening to your shuffled list of 1000+ songs you will eventually come to a section equivalent to 6 heads in a row and in the moment you’ll think the random shuffle sucks but it’s just an expected outcome of truly random shuffles. > > Coming up with an algorithm that humans \ is random is much harder because it’s a psychological problem, not a mathematical/programming problem. You mentioned “varied” in the OP but what does that mean? If you have two songs by the same band in a row, one a rocker and one a ballad, is that varied? What about about the same song by two different bands? It’s an interesting problem but likely has no solution that most people would agree on. This doesn’t really apply directly because flipping a coin has two possible outcomes vs a randomized playlist should have millions if not billions of variable outcomes. I guess based on the pure semantics of it, I could “randomly” pick every song in a row as they appear on each album in sequential order. This is tantamount to mining the raw ore and metals necessary to create a car, putting all the raw ingredients in a big bag, shaking it until they “randomly” combine to create exactly an Oldsmobile Cutlass Supreme, just by coincidence. Replaying the same few songs within the first hour of every single “shuffle” session isn’t random, it’s a buggy code one would think. The problem is this has been an ongoing issue across all media platforms and players for just about everyone I’ve ever talked to on the topic. By pooling picks of the same artists/albums early and often in a shuffle, it means other bands will bottleneck into the same kind of pools towards the end of the shuffle resulting in clusters of similar songs. I do understand the issue has to do with “what truly random means” vs "what human expectations of random are vis-à-vis listening experience, but there has to be a happy middle ground?? ---
**Author:** ![Deeg](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/deeg/32/2955_2.png) [@Deeg](https://boards.straightdope.com/u/Deeg)\ **Post date:** [May 17, 2020, 1:53am UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/22 "2020-05-17T01:53:27Z")
> [@DesertDog](#): > > Well, I hope they do a better job of it than the folks at I -heart- radio do. If I bring up, say, The Beatles channel it will play one (1) Beatles tune then start throwing up songs at random that go farther and farther afield and play _another_ Beatles tune an hour later, if I’m lucky. Some bands have a rather thin catalog and doing so would be more understandable in such cases, but _The Beatles_? You’re not even trying. I don’t know anything about I-heart-Radio but I’ve worked on vaguely similar projects. I assume in their case they don’t want to be random. I’m guessing that some songs (like any from The Beatles’ catalog) are more expensive than others. They have to balance playing the songs that you like as cheaply as possible, thus The Beatles get played less than you like. What happens if you select a less popular (and possibly cheaper) band? ---
**Author:** ![Deeg](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/deeg/32/2955_2.png) [@Deeg](https://boards.straightdope.com/u/Deeg)\ **Post date:** [May 17, 2020, 2:01am UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/23 "2020-05-17T02:01:12Z")
> [@Dr.Colossus](#): > > This doesn’t really apply directly because flipping a coin has two possible outcomes vs a randomized playlist should have millions if not billions of variable outcomes. I guess based on the pure semantics of it, I could “randomly” pick every song in a row as they appear on each album in sequential order. This is tantamount to mining the raw ore and metals necessary to create a car, putting all the raw ingredients in a big bag, shaking it until they “randomly” combine to create exactly an Oldsmobile Cutlass Supreme, just by coincidence. I think you’re vastly underestimating the chances of duplicate songs and likely overestimating (via confirmation bias) how often it happens. What music device are you using? It would be interesting to see if they publish their shuffle algorithm. > [@Dr.Colossus](#): > > The problem is this has been an ongoing issue across all media platforms and players for just about everyone I’ve ever talked to on the topic. By pooling picks of the same artists/albums early and often in a shuffle, it means other bands will bottleneck into the same kind of pools towards the end of the shuffle resulting in clusters of similar songs. I think this is proof that it’s not the shuffle algorithms but human perception. What’s more likely, that every shuffle algorithm has the same king of bug? Or that we’re just bad at judging what is truly random? ---
**Author:** ![DPRK](https://avatars.discourse-cdn.com/v4/letter/d/4491bb/32.png) [@DPRK](https://boards.straightdope.com/u/DPRK)\ **Post date:** [May 17, 2020, 3:03am UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/24 "2020-05-17T03:03:30Z")
> [@Deeg](#): > > I think you’re vastly underestimating the chances of duplicate songs and likely overestimating (via confirmation bias) how often it happens. What music device are you using? It would be interesting to see if they publish their shuffle algorithm. > I think this is proof that it’s not the shuffle algorithms but human perception. What’s more likely, that every shuffle algorithm has the same king of bug? Or that we’re just bad at judging what is truly random? He or she said that, out of thousands of songs, 5 of them invariably appear during the first 20 minutes. That does not sound like a random shuffle. ---
**Author:** ![Snarky\_Kong](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@Snarky\_Kong](https://boards.straightdope.com/u/Snarky_Kong)\ **Post date:** [May 17, 2020, 3:46am UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/25 "2020-05-17T03:46:05Z")
I think it’s significantly more likely to be confirmation bias than it is that Apple can’t randomize a list. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 17, 2020, 1:33pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/26 "2020-05-17T13:33:55Z")
> [@](#): > > Quoth **dontbesojumpy** : > > I do understand the issue has to do with “what truly random means” vs "what human expectations of random are vis-à-vis listening experience, but there has to be a happy middle ground?? The happy middle ground between “what people actually like” and “truly random” is right exactly on “what people actually like”. Any middle ground actually _between_ those two would inherently not be the “happy ground”. ---
**Author:** ![Ludovic](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Ludovic](https://boards.straightdope.com/u/Ludovic)\ **Post date:** [May 17, 2020, 1:57pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/27 "2020-05-17T13:57:00Z")
People aren’t monolithic. For instance, some people might enjoy a shuffle concentrating heavily on tracks the user has already played heavily while some people might enjoy more true randomness because if they wanted to play their most beloved tracks they wouldn’t use shuffle. ---
**Author:** ![jjakucyk](https://avatars.discourse-cdn.com/v4/letter/j/3d9bf3/32.png) [@jjakucyk](https://boards.straightdope.com/u/jjakucyk)\ **Post date:** [May 17, 2020, 3:02pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/28 "2020-05-17T15:02:34Z")
My understanding of the way Apple has always programmed iTunes, iPods, and iPhones is that they would take the full playlist and shuffle the order that would play through until the end, at which point the whole list would start over with a new shuffle order. That limits calculation to one time and eliminates the possibility of duplicate songs being played back-to-back or very close to one another, which could happen if a truly random selection was made every time the next song came up. This may not apply to the newer iterations of the Music app and streaming services though. So I would argue that while it’s not a truly random selection, it feels more random like in Deeg’s example about flipping coins because there’s no possibility of repeats, and you’re also guaranteed to hear everything. Where it can break down is when you get to the end of the list. While the chance of a repeat or recently played song coming up again is much smaller, it can still happen. Once I was listening on an older iPod Shuffle, and after skipping one or two songs it kept coming back to the same ones over and over. It took a moment to realize I had come to the end of my playlist and it didn’t have a chance yet to build the new shuffled order, so it was kind of stuck. Those Shuffles don’t have a whole lot more processing power than what’s needed to play the music, so it couldn’t build the new playlist order in just a couple seconds. ---
**Author:** ![pulykamell](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pulykamell/32/3166_2.png) [@pulykamell](https://boards.straightdope.com/u/pulykamell)\ **Post date:** [May 17, 2020, 3:46pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/29 "2020-05-17T15:46:15Z")
> [@DPRK](#): > > He or she said that, out of thousands of songs, 5 of them invariably appear during the first 20 minutes. That does not sound like a random shuffle. But that could be confirmation bias. My first thought was exactly **Deeg** ’s that humans are just bad at judging what is “random.” ---
**Author:** ![markn\_1](https://avatars.discourse-cdn.com/v4/letter/m/f9ae1b/32.png) [@markn\_1](https://boards.straightdope.com/u/markn_1)\ **Post date:** [May 17, 2020, 3:59pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/30 "2020-05-17T15:59:30Z")
Even playing short pop songs, there’s only time to play seven or eight in 20 minutes. If “at least 5” of them are always the same, that would indeed be stunningly nonrandom. But perhaps there is a bit of hyperbole in that report. ---
**Author:** ![Mijin](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mijin/32/9369_2.png) [@Mijin](https://boards.straightdope.com/u/Mijin)\ **Post date:** [May 17, 2020, 5:00pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/31 "2020-05-17T17:00:15Z")
I used to work in video game development, and lots of “random” numbers had to be fudged somewhat to _feel_ random to a human. If the player got a bad result two or three times in a row, they may feel the system is cheating. So I also put it down to human psychology / confirmation bias here. * * * OTOH it’s \*possible \*there’s a flaw in the shuffle algorithm. But they would really have had to drop the ball to screw up one of the most basic elements of a music player. And the OP asks why such a music function is badly implemented on \*any \*music device, to which I’d dispute the premise. I have a number of music devices (I’m oldskool), that shuffle great. ---
**Author:** ![DPRK](https://avatars.discourse-cdn.com/v4/letter/d/4491bb/32.png) [@DPRK](https://boards.straightdope.com/u/DPRK)\ **Post date:** [May 17, 2020, 5:25pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/32 "2020-05-17T17:25:47Z")
> [@Mijin](#): > > I used to work in video game development, and lots of “random” numbers had to be fudged somewhat to _feel_ random to a human. If the player got a bad result two or three times in a row, they may feel the system is cheating. It sounds as though none of your playtesters ever walked into a casino… 🙂 ---
**Author:** ![Mijin](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mijin/32/9369_2.png) [@Mijin](https://boards.straightdope.com/u/Mijin)\ **Post date:** [May 17, 2020, 5:48pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/33 "2020-05-17T17:48:53Z")
It’s a very common feature in games though. A triple-A game is unlikely to play all its random numbers “straight”. Because people are wont to rage quit as soon as two 90% shots miss in a row. Yes it’s irrational, but OTOH look at it this way: why assume that the most fun experience is always going to be the most mathematically fair one? Designers obviously aim for the former. ---
**Author:** ![AnalogSignal](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/analogsignal/32/1085_2.png) [@AnalogSignal](https://boards.straightdope.com/u/AnalogSignal)\ **Post date:** [May 17, 2020, 6:05pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/34 "2020-05-17T18:05:04Z")
This topic was covered in detail in Steven Levy’s book about the iPod [The Perfect Thing](https://en.wikipedia.org/wiki/The_Perfect_Thing). The author was convinced the iPod shuffle algorithm wasn’t working properly with the same songs being repeated and “Aretha Franklin forever waiting in the wings” never getting a chance to sing. In this book from 2006, the author complained to Steve Jobs who in turn checked with the Apple programmers to see if the shuffle was implemented properly. The conclusion was that shuffle was working correctly and it was dumb luck that certain songs were being repeated and others were never played. ---
**Author:** ![vonespy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/vonespy/32/6204_2.png) [@vonespy](https://boards.straightdope.com/u/vonespy)\ **Post date:** [May 17, 2020, 6:48pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/35 "2020-05-17T18:48:48Z")
Some of the perception of non-randomness can be explained by [the Gambler’s Fallacy](https://en.wikipedia.org/wiki/Gambler%27s_fallacy). I think part of the issue is how most devices come up with a “random” playlist. IIRC, the playlists are generated in a stateless manner to where they are re-generated on a regular basis for various reasons. If the playlists were more stateful (similar to what **jjakucyk** describes) and stuck in the device until every piece of music was played, then things would seem more random. Instead the playlists get re-generated frequently which means the odds of hearing, say, Afternoon Delight twice in the same week are reasonable. And once we hear Afternoon Delight a couple of times in a week’s time, we start doubting our sanity … er… the randomness of not-so-random devices. Not to derail but an interesting, just barely tangent to this would be [Benford’s Law](https://en.wikipedia.org/wiki/Benford%27s_law). It completely blew my mind when I first heard about it. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 17, 2020, 7:03pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/36 "2020-05-17T19:03:33Z")
The way I think of Benford’s Law: First, suppose that I pick a (uniform) random number between 1 and 9. What are the odds that the first digit is a 1? 1 in 9. What are the odds that it’s a 9? The same 1 in 9. But now suppose that I pick a random number between 1 and 20. What are the odds now that the first digit is is a 1? Much higher, now: 11 in 20. But the odds that the first digit is a 9 are much lower: 1 in 20. Or what if I pick a random number between 1 and 50? Both of those decrease now, 11 in 50 for 1, 1 in 50 for 9. But 1 is still much more likely than 9. Pick from 1 to 99? Now it’s exactly balanced again, 11 possibilities for each… for that one specific case. But from 1 to 98 or 1 to 100 would again favor 1 over 9. And now, what if I pick a random number between 1 and _n_, but you don’t know what _n_ is? Well, if 1 is sometimes exactly as common as 9, and sometimes more common, but never less common, then you can say that the first digit should be 1 more often than it is 9. ---
**Author:** ![vonespy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/vonespy/32/6204_2.png) [@vonespy](https://boards.straightdope.com/u/vonespy)\ **Post date:** [May 17, 2020, 7:28pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/37 "2020-05-17T19:28:38Z")
**Chronos** , wow, excellent explanation! ---
**Author:** ![Kimble](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/kimble/32/19185_2.png) [@Kimble](https://boards.straightdope.com/u/Kimble)\ **Post date:** [May 17, 2020, 7:51pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/38 "2020-05-17T19:51:27Z")
I don’t think this is what’s happening to **dontbesojumpy** , but I offer this as a documented example of bad shuffling: I told my Echo Dot, “Alexa, play the Indie Dance Essentials playlist from Apple Music on shuffle mode.” This is an Apple-created playlist with 100 tracks. It started with track 1. The next tracks were 4, 5, 7, 10, 6, 2, 11, 3, 9, 8. After that: 15, 13, 21, 19, 14, 17, 20, 18, 12, 16. Then: 30, 25 … and I stopped after that. That is very not random. I couldn’t get playing my own playlists to work on my Echo Dot. I don’t have Amazon Music, so I don’t know if Alexa does a better job with her native music ecosystem. Hitting Shuffle and then play in iTunes in Windows on an Apple Music playlist started with 68, 36, 25, 84, 98, 83, 56, 59, 87, 29, 18, 11, 53, 21, 60. That seems random enough to me. ---
**Author:** ![Dr.Colossus](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.colossus/32/9423_2.png) [@Dr.Colossus](https://boards.straightdope.com/u/Dr.Colossus)\ **Post date:** [May 17, 2020, 8:18pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/39 "2020-05-17T20:18:31Z")
Is there an app or simple way to log songs and play counts? I used to scrobble to last.fm but I don’t think that works. I’d like to bump my observations up from anecdotal to statistical. When I’m working–the time when I listen to shuffle the most–I don’t really have time to make notes on what’s playing. So anything that might track that could be helpful. I just opened my player to a new shuffle. In the first 8 tracks played, 2 were off the same album (Barkmarket, L. Ron). Track 12 was Superdrag, as was 15 and 25 _same album_. Track 23 was off Gleemer’s Anymore, as was 35 and 38. Tracks 30 and 37 were off the same album. There were other pairs (many other pairs, or trios) in the first 50 songs played, but you get the point. I re-sorted by song title and fired up a new shuffle. Tracks 1 and 4 were by the same band. Track 8, 10 were by the same band, Cloudkicker, This is what I mean. Hearing the same band/same album a few songs apart regularly. It seems to tend to bunch up. New shuffle: Tracks 7 and 9 are off the same album. I could do this all day. Again, this is where we devolve into the semantics of “what random means” vs what someone wants in a shuffle. This is why I brought up the term “varied,” because when some people say they want “random” they really mean “a varied experience.” In my experience, once a band is played, my player seems to want to clump that band or that album together within a span of 20 songs or so, out of thousand and thousands of songs. I’m using Pi Music Player v3.0.8.1 There’s a “persist shuffle” setting that doesn’t reshuffle after exiting the app. Not sure if that would help. For reference I have over 108 hours of music to shuffle from. ---
**Author:** ![Ike\_Witt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ike_witt/32/283_2.png) [@Ike\_Witt](https://boards.straightdope.com/u/Ike_Witt)\ **Post date:** [May 17, 2020, 9:05pm UTC](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892/40 "2020-05-17T21:05:50Z")
> [@Deeg](#): > > I’m a programmer and it’s very easy to randomly shuffle a list. The problem is that humans have a terrible sense of what is truly random. [This article](https://www.bbc.com/news/technology-31302312) agrees with you. > [@](#): > > Five thousand songs, and it brings up three tracks by the same artist in quick succession? Come on now, that can’t be right. This is supposed to be random! > > Except, of course, it is random. Our brains just don’t like it. [Previous page](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892.md?page=1) [Next page](https://boards.straightdope.com/t/why-is-a-truly-random-shuffle-musical-function-so-difficult-to-execute/853892.md?page=3)