When Netflix first came out with their movie recommendations, I thought it was a great idea. I started rating movies I’d seen — good and bad — confident that the brain behind it all will do its magic and recommend some hidden movie gems that will, you know, change my life. Well, I’m still waiting for those movies. And to be honest, I’ve become a little bit frustrated with the whole thing.

Describing the latest example I encountered will reveal how much I liked a movie that I probably have no business liking, but I’m willing to sacrifice a little bit of my reputation in the name of science, or whatever this is…

The first problem I encountered is a pure UI issue, and has to do with how Netflix shows the star movie ratings on their pages. As an example, this is what I see for the movie August Rush in my queue:

You would assume that the customer average rating is just over the 3-mark, right? Well, looking at it closer, it turns out that Netflix shows you a rating they call “Our best guess” (3.4 in this case), instead of showing you the customer average (4.1 in this case):

Here’s the problem. I loved this movie. I’m giving it 4 stars. But since Netflix doesn’t know that I have a soft spot for modern musicals (despite how highly I rated the movie “Once”), the “Netflix brain” didn’t think I would like this movie as much as the average customer.

This is a problem you see often on sites where the UI does not give proper user feedback about what it’s showing you.  It took me a few weeks to realize they’re showing me “Our best guess” in search results, and not the true customer average. Now I have to mouse over to see the true average every time. Why? Because I don’t trust the brain any more. (By the way, this is just one example, but as I’ve looked into it more, I realized it’s a systemic problem for me — Netflix’s best guess is rarely in line with my tastes).

Incidentally, on Amazon.com, the average user rating is 4.5 out of 5 stars. Pretty good. So this is the problem then. There is such a wide range of tastes out there that it’s hard to know who to trust. This is the problem Netflix is trying to solve — let’s look at “users like you” and then show you that average instead of the overall average. You’re therefore initially more inclined to believe the “best guess” rating provided by Netflix, than the average consensus provided by all users. It’s a good idea, but the implementation doesn’t seem to be there yet.  (The discussion about the validity of 5-star ratings in general is a separate and very interesting discussion).

I say all this to make a simple point — it appears that the collective wisdom of all users does a better job of predicting if I will like a movie than the recommendation engine provided by Netflix. The question is whether it would ever be possible for recommendation engines to get to know you well enough based on your preferences. Maybe if it takes into account not only your movie interests, but also music, books, online activity, etc.? Yes it sounds creepy, but how else would Netflix know how much I like strange modern musicals?