# Amazon's book recommendation engine is dumb -- any better ones? **URL:** \ **Category:** Cafe Society\ **Created:** [May 19, 2009, 7:02pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998 "2009-05-19T19:02:45Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Gadarene](https://avatars.discourse-cdn.com/v4/letter/g/e47c2d/32.png) [@Gadarene](https://boards.straightdope.com/u/Gadarene)\ **Post date:** [May 20, 2009, 10:05pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/21 "2009-05-20T22:05:01Z")
> [@Eleanor\_of\_Aquitaine](#): > > By the way, Connie Willis is working on a new novel in her time-travel universe called _All Clear_, about the London Blitz. She is supposedly in the editing phase. Feels like she’s been in the editing phase for years now. 🙂 ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 20, 2009, 11:33pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/22 "2009-05-20T23:33:05Z")
> [@](#): > > Let’s take a hypothetical bestseller called Slice of Pi. Say that forty percent of the reading public owns a copy of Slice of Pi, but only twenty-five percent of the people who own a copy of Bellwether also own a copy of Slice of Pi. (Let’s also say, for simplicity’s sake, that if you own a copy of a book, that means that you like it.) This strongly suggests that people who own a copy of Bellwether are significantly less likely to like Slice of Pi than people who don’t own a copy of Bellwether. Yet as long as that 25 percent is still a large enough chunk of the Bellwether-owning public, Slice of Pi will show up on the list of items that people who purchased Bellwether also purchased…even if the other 75 percent of the people who own Bellwether would think that Slice of Pi is bland, allegorical crap. OK, let’s consider a hypothetical other book to compare to. Suppose that there’s also a book titled _Foo_, that only 1% of the general public liked, but that 10% of the people who liked _Bellwether_ also liked. Clearly, then, there’s a significant connection between liking _Bellwether_ and liking _Foo_, right? And I’m guessing that you think that _Foo_ should be recommended instead of _Slice of Pi_. Here’s the thing, though: If I’m a person that liked _Bellwether_, then there’s a 25% chance that I’ll like _Slice of Pi_, and a 10% chance that I’ll like _Foo_. _Foo_ may be more similar to _Bellwether_, but that doesn’t change the fact that I’m more likely to like _Slice of Pi_ than I am to like _Foo_, and that it’s therefore a better recommendation. Put another way, suppose that there were yet another book, _Bar_, that’s loved by _everyone_. If I were the one person on the planet who hadn’t read it yet, surely you’d recommend that I pick it up, right? ---
**Author:** ![Thudlow\_Boink](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/thudlow_boink/32/320_2.png) [@Thudlow\_Boink](https://boards.straightdope.com/u/Thudlow_Boink)\ **Post date:** [May 21, 2009, 12:37am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/23 "2009-05-21T00:37:35Z")
This is becoming an interesting exercise in probability. It looks to me as though **Chronos** wants to read book S (_Slice of Pi_) if P(S|B) is large, while **Gadarene** wants to read book S if P(S|B) \> P(S). That is, **Chronos** wants to read _Slice of Pi_ if there is a large chance that he will like it given that he liked _Bellwether_. **Gadarene** wants to read _Slice of Pi_ if the fact that he/she liked _Bellwether_ makes it more likely that he/she will like _Slice of Pi_ than it would have been otherwise. ---
**Author:** ![Thudlow\_Boink](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/thudlow_boink/32/320_2.png) [@Thudlow\_Boink](https://boards.straightdope.com/u/Thudlow_Boink)\ **Post date:** [May 21, 2009, 12:41am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/24 "2009-05-21T00:41:49Z")
> [@Gadarene](#): > > (Let’s also say, for simplicity’s sake, that if you own a copy of a book, that means that you like it.) The problem with this assumption is that, the majority of the time, people buy books that they have not yet read (they buy them in order to read them). If you want to know what people who **liked** this book also **like** (as opposed to knowing what people who **bought** this book also bought), I suppose you could look through the customer reviews, find the ones that rated the book highly for reasons similar to why you liked the book, and check that reviewer’s other reviews to see what other books they rated highly. ---
**Author:** ![Harriet\_the\_Spry](https://avatars.discourse-cdn.com/v4/letter/h/e56c9b/32.png) [@Harriet\_the\_Spry](https://boards.straightdope.com/u/Harriet_the_Spry)\ **Post date:** [May 21, 2009, 1:15am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/25 "2009-05-21T01:15:01Z")
Perhaps this is obvious, but Amazon also has another way to generate recommendations - by rating books recommended based on your purchases. By rating a lot of books and putting books on my wishlist (both their recommendations and other stuff I run across that I want to read), I’ve got this doing a pretty good job of recommending stuff to me that it at least makes sense to look at. It does an especially decent job keeping up with new books by authors I like and new books about my professional interests. The professional thing is probably because it’s a fairly small, consistent number of people even paying attention to those books. It does have the tedious habits of recommending every edition, recommending books I own, and making really whacked out recommendations every time I order a baby shower gift (this can be turned off by telling it not to use an item to make recommendations). My library had a service for a while (maybe they still do) where you could send them a list of 10 books you liked and they would have a trained human librarian make 10 recommendations. I got a reasonable list from that, too. ---
**Author:** ![smurfslappa](https://avatars.discourse-cdn.com/v4/letter/s/5f9b8f/32.png) [@smurfslappa](https://boards.straightdope.com/u/smurfslappa)\ **Post date:** [May 21, 2009, 1:25am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/26 "2009-05-21T01:25:05Z")
It keeps offering me Twilight. What a load of crap. ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [May 21, 2009, 4:03am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/27 "2009-05-21T04:03:21Z")
> [@Gadarene](#): > > Your conclusion that Book A will be a good recommendation for someone who likes Book B as long as x percent of people who own Book B also own Book A just isn’t logically sound. It’s a good recommendation if it gets people to buy it, and it apparently works well enough for Amazon to rely on the system to turn a profit. Remember, Amazon doesn’t care about how well their recommendations work for any individual customer; they only care about how well their recommendation system works on average. As long as people are making purchases based off their recommendations, they’re happy. You also have to consider that Amazon has millions of items and millions of customers, which means that they need something fast in order to be able to make recommendations at all. There’s a definite trade-off between how sophisticated your recommendations can be and how much time you need to compute them. It’s possible that they could do something more sophisticated, but they have to be clever about implementing it (see [Dynamo](http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf) for an idea of how clever) and it’s not clear that it’ll have a great impact on their bottom line. ---
**Author:** ![Markxxx](https://avatars.discourse-cdn.com/v4/letter/m/5daacb/32.png) [@Markxxx](https://boards.straightdope.com/u/Markxxx)\ **Post date:** [May 21, 2009, 4:36pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/28 "2009-05-21T16:36:44Z")
I would try Google Group’s archive of Usenet. In it’s day Usenet used to have a lot of book groups and though it’s not used much anymore, you can get a lot of older books you probably haven’t read yet. And since they’re old they are probably at your library ---
**Author:** ![phreesh](https://avatars.discourse-cdn.com/v4/letter/p/9fc29f/32.png) [@phreesh](https://boards.straightdope.com/u/phreesh)\ **Post date:** [May 21, 2009, 5:19pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/29 "2009-05-21T17:19:24Z")
I’m surprised that nobody has chimed in to recommend that you do some work to let Amazon know: a. What you already own (as noted, this can be a pain, as you have to tell it you own the hardback, the paperback, the tenth anniversary edition, the large print version, etc) b. What you like (star rankings) c. What you aren’t interested in I have spent a number of hours rating the stuff I own (telling Amazon that I both already own and what star ranking I give it) and checking off ‘not interested’. I have also gone through the genres I generally like and ranked maybe a couple hundred books/movies that I’ve read or seen but don’t own. That invested time has resulted in generally excellent recommendations. Amazon has quite a lot of ways of helping to improve its recommendations if you look into it. The downside is that it takes time and by the end Amazon will know quite a lot about you, which is a bit scary. ---
**Author:** ![Sigmagirl](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@Sigmagirl](https://boards.straightdope.com/u/Sigmagirl)\ **Post date:** [May 21, 2009, 6:40pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/30 "2009-05-21T18:40:02Z")
OK, I’m on my recommendations now, and can anybody tell me what a “shopping list” is as opposed to a wish list? A shopping cart will add up items right now for purchase. A wish list is stuff I think I want, but am putting on the back burner for future purchases or for somebody to buy me, maybe, as a gift. A shopping list? What would be helpful is a third selection box, after “I own it” and “Not interested,” that says “Interested.” It would bookmark and compile those items I’ve checked into a separate list that I can come back to later and maybe narrow down. That may be the purposed of a “shopping list,” but the way it is set up now, I have to click back through several pages to return to the recommendations page. Too much trouble. ---
**Author:** ![wonderlust](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wonderlust/32/7237_2.png) [@wonderlust](https://boards.straightdope.com/u/wonderlust)\ **Post date:** [May 21, 2009, 6:56pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/31 "2009-05-21T18:56:24Z")
[LibraryThing](http://www.librarything.com/) has a number of different ways to get suggestions, and even has an Unsuggest feature. The more books you enter from your library, the better your results are, because it compares you library to similar libraries. ---
**Author:** ![Harriet\_the\_Spry](https://avatars.discourse-cdn.com/v4/letter/h/e56c9b/32.png) [@Harriet\_the\_Spry](https://boards.straightdope.com/u/Harriet_the_Spry)\ **Post date:** [May 21, 2009, 8:46pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/32 "2009-05-21T20:46:32Z")
> [@Sigmagirl](#): > > What would be helpful is a third selection box, after “I own it” and “Not interested,” that says “Interested.” I use my wish list that way. If it shows up on recommendations and I think I want it eventually, I move it to the wish list. My wish list is full of no end of goodness now! The shopping list automatically includes items you buy routinely. I don’t usually shop Amazon for that kind of stuff, so I ignore the shopping list. ---
**Author:** ![jackdavinci](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/jackdavinci/32/3259_2.png) [@jackdavinci](https://boards.straightdope.com/u/jackdavinci)\ **Post date:** [May 22, 2009, 6:32am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/33 "2009-05-22T06:32:11Z")
Supposedly this sort of thing is very hard to engineer and there’s even been reward money offered to anyone who can do it right, so you might be stuck. I will point out though that there are other ways to get recommendation ideas from Amazon: 1. The keywords and category sections. Clicking on the keyword or category that is best associated with the quality you like about the book will often lead you to other books with similar qualities. 2. User’s book lists. These are often more helpful since they are specifically created by users, and have useful titles like “25 weird animated DVDs” or “subversive kid’s fiction” etc. If your book is on somebody’s list, you might like some of the other stuff on the list. 3. Reader reviews. Often reader reviews will compare the book favorably, unfavorably, or neutrally to other books. If you really love a particular book and are desperate to find similar things you can ask for suggestions there or here. I would argue that creating an algorithm to suggest books based purely on a list of books you’ve bought or rated is inherently troubled. It’s kind of like asking your wife what you want for dinner. Well, you tend to like pizza, she says (this is like suggesting an author you’ve already read). Or you and my friend Frannie both like pizza and she recently found this cool new Spanish restaurant (like suggesting books popular with other buyers of this book). People’s book tastes are just too eclectic to make suggestions in a broad based way. You may love hard sci-fi, but are also buying parenting books because you just had kids, and gardening books because you recently got into that, but when you go to the beach you always bring a trashy bodice ripper. There’s no way for a computer to relate these things together into some kind of generalized personal preference psychological profile. It’s much better for you to just go looking specifically for sci-fi books if that’s what you’re in the mood for, which amazon and other sources are perfectly good at, or sci-fi books in a similar style to a particular author, which google and wiki have lots of examples of or can be sussed out using amazon keywords or the like. Just asking for simply books you might like based on all the books you’ve previously liked is 1) too inspecific a query for there to be a useful answer and 2) prevents truly useful suggestions that are outside your normal range of reading. ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [May 22, 2009, 4:48pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/34 "2009-05-22T16:48:12Z")
> [@jackdavinci](#): > > Supposedly this sort of thing is very hard to engineer and there’s even been reward money offered to anyone who can do it right, so you might be stuck. You’re thinking of the [Netflix Prize](http://www.netflixprize.com/). Technically they’re trying to solve a slightly different problem (estimating a user’s ratings of a movie rather than trying to compute a list of recommendations), but you’re basically right. (For the more technical minded here, you would think that being able to estimate whether a user will rate movie A higher than movie B would allow you to put together a ranked list of recommendations, but it turns out that computing the list which minimizes the disagreement with those estimates is NP-hard. Different algorithms are needed.) > [@](#): > > People’s book tastes are just too eclectic to make suggestions in a broad based way. You may love hard sci-fi, but are also buying parenting books because you just had kids, and gardening books because you recently got into that, but when you go to the beach you always bring a trashy bodice ripper. There’s no way for a computer to relate these things together into some kind of generalized personal preference psychological profile. It’s much better for you to just go looking specifically for sci-fi books if that’s what you’re in the mood for, which amazon and other sources are perfectly good at, or sci-fi books in a similar style to a particular author, which google and wiki have lots of examples of or can be sussed out using amazon keywords or the like. Just asking for simply books you might like based on all the books you’ve previously liked is 1) too inspecific a query for there to be a useful answer and 2) prevents truly useful suggestions that are outside your normal range of reading. It’s not necessarily impossible, but as I mentioned above, Amazon is constrained by their computational resources. Better algorithms and better implementations are needed. ---
**Author:** ![fiddlesticks](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/fiddlesticks/32/18129_2.png) [@fiddlesticks](https://boards.straightdope.com/u/fiddlesticks)\ **Post date:** [May 22, 2009, 6:12pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/35 "2009-05-22T18:12:47Z")
Of course, you can tell Amazon that “I already own this book” and “I’m not interested in this book” via various links on those pages. While I don’t think it improves things, at least you won’t see books you have/aren’t interested in any longer. ---
**Author:** ![Lynn\_Bodoni](https://avatars.discourse-cdn.com/v4/letter/l/e47c2d/32.png) [@Lynn\_Bodoni](https://boards.straightdope.com/u/Lynn_Bodoni)\ **Post date:** [May 22, 2009, 8:59pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/36 "2009-05-22T20:59:41Z")
> [@fiddlesticks](#): > > Of course, you can tell Amazon that “I already own this book” and “I’m not interested in this book” via various links on those pages. While I don’t think it improves things, at least you won’t see books you have/aren’t interested in any longer. Yes, actually, you will. Often, Amazon will recommend a book that I BOUGHT FROM AMAZON. Sometimes I will buy copies of books that I’ve read for friends, but not very often. Also, as someone noted upthread, Amazon regards every edition as a separate book, rather than listing it in one entry with choices of hardback, paperback, trade edition, and large print in HB and PB. If I don’t want a book in paperback, I’m very unlikely to want it in hardback. I have to admit, I use Amazon recommendations in a way they probably didn’t intend. I notice which authors and books they keep showing me, and if I see these authors at a used book store, then I will pick up one or two. Sometimes this works out very well (I found out that I LOVE John Scalzi, for instance) and sometimes I don’t like the author, but at least I’m not out as much money. So, Amazon might or might not sell me some books by John Scalzi. At least their recommendation engine worked well that time. ---
**Author:** ![Khadaji](https://avatars.discourse-cdn.com/v4/letter/k/9e8a1a/32.png) [@Khadaji](https://boards.straightdope.com/u/Khadaji)\ **Post date:** [May 22, 2009, 10:27pm UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/37 "2009-05-22T22:27:43Z")
One of the things that annoys me is when I rate a book as “I hate it” and Amazon recommends another book - giving the reason that I rated the first. You would think a very bad rating would mean I don’t want others like it, but Amazon’s engine just sees that I rated it (apparently.) ---
**Author:** ![lucky\_henry](https://avatars.discourse-cdn.com/v4/letter/l/bc8723/32.png) [@lucky\_henry](https://boards.straightdope.com/u/lucky_henry)\ **Post date:** [May 23, 2009, 2:55am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/38 "2009-05-23T02:55:57Z")
> [@Gadarene](#): > > …If anyone has any suggestions for a book recommendation engine that doesn’t just give me the same old stuff, I’d be interested to check it out. I’ve found [Literature Map](http://literature-map.com/) to be quite helpful. You enter the name of an author you like and it returns a spacial abstract with the author you named in the center, and other authors arranged around the center with similarity based on proximity…it’s too hard to explain, just check it out! ---
**Author:** ![Lynn\_Bodoni](https://avatars.discourse-cdn.com/v4/letter/l/e47c2d/32.png) [@Lynn\_Bodoni](https://boards.straightdope.com/u/Lynn_Bodoni)\ **Post date:** [May 23, 2009, 7:46am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/39 "2009-05-23T07:46:25Z")
That Literature Map is a fun toy. Bookmarked. Thanks! ---
**Author:** ![Khadaji](https://avatars.discourse-cdn.com/v4/letter/k/9e8a1a/32.png) [@Khadaji](https://boards.straightdope.com/u/Khadaji)\ **Post date:** [May 23, 2009, 11:55am UTC](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998/40 "2009-05-23T11:55:46Z")
> [@lucky\_henry](#): > > I’ve found [Literature Map](http://literature-map.com/) to be quite helpful. You enter the name of an author you like and it returns a spacial abstract with the author you named in the center, and other authors arranged around the center with similarity based on proximity…it’s too hard to explain, just check it out! Pretty interesting. I wonder how they choose proximity. I checked Jim Butcher and I have to say I would not have put Tanya Huff right up against him, and then have Simon R. Green and Glen Cook so far away. But it still looks like a great source. Thanks. [Previous page](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998.md?page=1) [Next page](https://boards.straightdope.com/t/amazons-book-recommendation-engine-is-dumb-any-better-ones/496998.md?page=3)