Patterns In Randomness: The Bob Dylan Edition
    By Barry Leiba | December 15th 2011 02:29 PM | 11 comments | Print | E-mail | Track Comments

    The human brain is very good — quite excellent, really — at finding patterns. We delight in puzzles that involve pattern recognition... consider word-search puzzles, the “Where’s Waldo” stuff, and the game Set. We’re also great at giving patterns amusing interpretations, as we do when we fancy that clouds look like ducks or castles — or when we claim to see images of Jesus in Irish hillsides, pieces of wood, paper towels, and store receipts. Remember the cheese sandwich with the Virgin Mary on it, which sold on eBay for $28,000 in 2004? Miraculous, indeed.

    It’s with the knowledge that we find apparent patterns in randomness that I approach this puzzling aspect of the “random play” feature of my car stereo. I’ve stuck in a microSD card that has about 4000 songs on it. I’ve put it on random play. And it appears to be playing songs in random order.

    But it sure seems to be playing a lot of Dylan.

    Bob, not Thomas. I like Bob Dylan, of course; that’s why I have quite a bit of him on the microSD card. But, for instance, on one set of local errands, it played two Dylan songs, something else, another Dylan, two other songs, then another Dylan. Four out of seven? Seems a bit odd.

    Now, I know that if you ask a typical person which sequence is more likely to come up in a lottery drawing, 1-2-3-4-5, or 57-12-31-46-9, he will say not only that the latter is more likely, but that if the former came up he’d be sure something was amiss. In fact, they’re equally likely, and are as likely as any other pre-determined five-number sequence, but the one that looks like a pattern is one we think “can’t be random.” Similarly, it’s certainly possible to randomly pick four Dylan songs out of seven — or even four in a row, for that matter. And if there’s a bug in the algorithm that the audio system uses, why would it opt for Dylan, and not, say, Eric Clapton or the Beatles, both of which I also have plenty of on the chip?

    So I played around with some numbers. Let’s make some simplifying assumptions, just to test the general question. Assume I have 20 songs from each artist, and a total of 4000 songs (and, so, 200 artists). If I play seven songs, how likely is it that two will be by the same artist?

    It’s easier to figure out how likely it is that there won’t be repetitions. The first song can be anything. The likelihood that the second will be of a different artist than the first is (4000-20)/3999, about 99.5%. The likelihood that the third will differ from both of those is (4000-40)/3998. Repeat that four more times and multiply the probabilities: there’s a 90.4% chance of seven different artists in seven songs... meaning that there’s about a 9.6% chance of at least one repetition. Probably more likely than we might think.

    Let’s look at Dylan, specifically. I have about 120 of his songs on there (3% of the total; maybe I should delete some, but that’s a separate question). What are the chances of having no Dylan in seven songs? No Dylan for the first is 3880/4000, 97% (makes sense: 3% chance of Dylan in any one selection). Continuing, no Dylan, still, for the second is 3879/3999. Repeat five more times and multiply: 71.3% chance of no Dylan, so there’s a 28.7% chance of at least one Dylan song if we play seven.

    What about the chances of at least two Bob Dylan songs... a repetition of Dylan? Well, we figured out no Dylan above. Let’s figure out exactly one, and then add them. For the first to be Dylan and none of the others, we have 120/4000 * 3880/3999 * 3879/3998 * 3878/3997 * 3877/3996 * 3876/3995 * 3875/3994. About 2.5%. It’s the same for one Dylan in any other position — the numerators and denominators can be mixed about. So the chances of exactly one Dylan song out of seven is 2.5 * 7, or 17.5%. Add that to the chances of zero, 71.3 + 17.5 = 88.8%, so there’s an 11.2% chance of at least two Dylan songs in a mix of seven songs.

    In other words, it’s a better than one in four chance that I’ll hear at least one Bob Dylan song, and a better than one in ten chance that I’ll hear at least two of them every time I take a 20- or 30-minute ride. Thrown in some confirmation bias, where I forget about the trips that had Clapton and the Beatles and Billy Joel and Carole King, but no Dylan, and I guess the system is working the way it’s supposed to.

    But, damn, it plays a lot of Bob Dylan!


    My phone seems to randomly pick albums(artist?) and then plays a bunch of them, then switches to a different artist.
    I'll also note, computer random number generators aren't.
    Never is a long time.

    I appreciate your math, but why in the world would you pollute the sample with non-Dylan songs. Bob should be 100%. :-)

    I have 500 Dylan songs on an iTunes playlist. Starting from zero, how many random (shuffled) plays will it take to hear every song at least once? Why does Catfish play 15 times and Restless Farewell never? Does God play dice with Dylan songs? :-)

    All the best, - Fabe

    HA! I was going to answer "Lucky You, Don't count, just be grateful!"to Mr. Leiba- but your answer gave me a laugh- 100% indeed. I can't get enough of that man.
    I have my ipod docked in a clock radio and he is the first thing I hear every morning... and sometimes the last thing at night.

    hmmm... I wonder if that's why I'm unattached. LOL

    Huh... 500 eh? Step up to the big leagues and call me when you hit 1200...

    There is no such thing as a random number.

    This reminds me of a math class I once had. The instructor assigned the task of flipping a coin 10000 times and recording the results. He could tell when someone cheated and just wrote their own sequence of "H" and "T" without flipping the coin (or more intelligently simulating the task by computer). A person just writing the sequence will typically never write 8 H's in a row. However, in 10000 real flips, it's almost a certainty that such a sequence will occur. People think they know what "random" looks like, but for the most part our intuitions about randomness are pretty bad.

    WHAT? Only 120 Dylan songs out of 4000? Something is amiss and the "random" incident is trying to tell you: Add More Bob!

    loved the post- numbers, random and otherwise are always fascinating even with my less than rudimentary understanding.

    OH GOD- I'm so sorry about the three posts in a row- I kept getting a message that I was putting some information in wrong so I redid it got same info-redid it again with some additional- and now I see three have shown up... is this random?

    Oddly, the comment made directly to you, the author didn't come up. Now I wonder if I should post it again?

    "In a book that nobody can write."

    Nice post that stimulates the brain cells this early in the morning.
    First time I've visited here, got here through Expecting Rain.

    I'm a systems guy and a songwriter, so it goes without saying I love brain exercises, puzzles, patterns etc.
    If you are interested in hearing some of my music, go to my music web page.
    I'd recommend They're Gone, Dad I Miss You and The Old Man of the Mountain (Yes, the one from NH).

    Keep up the good work.

    Phil T.

    Barry, your anti spam efforts are appreciated (learned about them through your science 2.0 bio)...'Enjoyed your article and here's a link to my favorite Dylan song with lyrics:
    Thanks for all the comments, everyone (and it's amusing to see all the fellow Dylan fans; quick response to that: I love Dylan, but I love a lot of other music as well).

    I've just posted an update to this. The frequency of “B” artists in the mix was due to more than an error in randomness.