The Percent Match Score is a fraud.
Last year, Netflix has caused a lot of turmoil by changing its recommendation system.
https://media.netflix.com/en/company-blog/goodbye-stars-hello-thumbs
Immediately, a lot of members were upset that Netflix had replaced the stars rating system with a less nuanced thumbs rating system. It took a little longer to realize that the most dramatic change was in fact replacing the predicted rating with a percentage. Netflix calls it the ?Percent Match Score?, and it?s very important as it determines which shows will be displayed prominently. The problem is that nobody understands how it works. There has been a lot of debates on the internet, for example this one:
Like the man from the video, when you see the percentage given to a show, you assume that it represents how much you?ll like the show. The higher the percentage, the more you?ll like it. That?s what I thought too, and all the persons around me thought the same. After all, that?s how the predicted rating has always worked. Bad news, we?re all wrong, it?s not at all what the percentage means. A percent match score isn?t the same as a predicted rating. It?s a completely different thing, and the man from the video above is comparing apples and oranges, hence the weird results.
Here?s what Netflix says about the Percent Match Score:
?This score indicates how likely we think you are to like that title?.
As you can see, it?s not about how much you?ll like the title, it?s about how likely you are to like it. In other words, it?s the probability that you?ll give it a thumbs up. If you turn the definition around, the lower the percentage, the more likely you are to not like the show. For example, saying that you?re 70% likely to like a show also means that you?re 30% likely to not like it. It?s very far from being a predicted rating! Not only you don?t know if the show will be excellent, or just very good, or just good (goodbye stars, hello thumbs), but when the percentage is lower than 100%, there?s also a possibility that the show will be bad. Thank you Netflix, we?d be lost without your precious help!
The closer to 100%, the more Netflix is confident that you?ll like the show. Conversely, a percentage lower than 100% reveals an uncertainty. For example, a show with 80% match score means that Netflix recommends you to watch that show, but there?s 20% chance that it?s an error to recommend it to you. You want to know how much you?ll like a show, but instead Netflix tells you how likely its prediction could be wrong!? It?s so idiotic that there has to be a problem with the definition given by Netflix.
That doesn?t make sense? It?s not as if Netflix was trying to accurately predict how much you?ll like a show. Netflix is only trying to predict if you?ll like it (even a little) or not. For such a binary prediction, and assuming that Netflix is at least a little good at understanding your tastes, there should be many cases where there?s little doubt whether you?ll like or not. If Netflix understood your tastes perfectly, there would be only match scores of 100% (Netflix is sure you?ll like) and match scores of 0% (Netflix is sure you won?t like). How far the percentage is from 100% can be interpreted as a measure of how badly Netflix understands your tastes.
We can observe percentages as low as 55%, which would indicate a terrible understanding of your tastes. For example, I?m surprised that Netflix is only 73% confident that I?ll like the Silence of the Lambs, one of my all time favorite movies. The old system was able to predict that I?ll LOVE that movie, but the new system isn?t even sure that I?ll LIKE it? Seriously? Considering how Netflix has proven in the past that it could be very good at understanding our tastes, the only remaining explanation is that the percentages must represent something else, but what?
The fact that 55% is the lowest percentage given to a show seems to confirm that this thing really is a percentage of probability. It never goes as low as 50% probability because it would be as accurate as flipping a coin (1 chance out of 2 that you?ll like). By the way, not showing any percentage lower than 55% is a serious issue with this new system, because you can?t tell if a show doesn?t have a percent match score because it?s too bad or because the system wasn?t able to make a prediction. A percentage lower than 50% could have warned you that you?re more likely to dislike a show rather than like it, but Netflix has chosen to hide all the low percentages. No more bad shows, the new system is so corrupted that the bad shows aren?t shown as bad anymore. That?s certainly better for the Amy Schumer stand-up special!
Furthermore, everybody noticed that the Netflix Originals get high percentages and as a consequence they monopolize the top spots. At least we have a proof that the percentages are not completely random ? In fact, the more we look at it, the less the percentages look random. Netflix seems to have the ability to manipulate the percentages to promote the shows it wants. Some excellent movies are given unfair low scores, while some average Netflix Originals are given undeservedly high scores.
I?ve read dozens of discussions on the subject, and apparently nobody in the world understands what those percentages mean. It?s ironical considering that one of the reasons that Netflix gave to justify the change was precisely that users didn?t understand the previous prediction system.
What we know: – The percentages are different from the predicted ratings. – The percentages probably are percentages of probability. – The percentages are boosting Netflix Originals. – The percentages can?t be what Netflix say they are.
What we don?t know: – What the percentages really are.
A clarification from Netflix would be much welcome!
Netflix recommendations are broken.
With the less accurate thumbs rating system, Netflix can only have a less accurate understanding of our tastes, it?s mathematical. But it?s replacing ?how much we?ll like? by ?how likely we?ll like? that definitively ruined the recommendation system. We don?t want some weird percentage of probability. We want to know how much we?ll like a show. We want a predicted rating!
At best, the percentages will help you to find shows that you?ll like, but in no way they can help you to find the best possible shows for you, those that you?ll love rather than just like. You?re plunged in an ocean of average shows, with little chance of finding the pearls hidden in the depths.
There?s a theory which suggests that it?s exactly what Netflix wants. Indeed, if you know that you?ve already watched all the best shows, you can be tempted to unsubscribe. On the contrary, if you?re always hopeful to make a great discovery, then you can stay subscribed indefinitely. In my opinion, some people might indeed unsubscribe if the recommendation system is too good, but certainly many more will unsubscribe if the recommendation system is broken.
Netflix has run some tests and found that people rate more with the thumbs than with the stars. Ok, with the thumbs you get more data, but if that data is less accurate, you gain nothing. Especially when analyzing movie tastes, the nuances are essential. It?s very different to LIKE a movie and to LOVE it. I don?t think that behavior analysis can compensate the loss, because our behavior is easily misinterpreted.
For example, I watch all the Marvel movies, but I never give them great ratings. I?m always super excited when I see a new Marvel movie trailer, but the hard truth is that they?re not my favorite movies, they?re always kind of a disappointment. Until now, that hasn?t stopped me from watching them all, but I wouldn?t like Netflix to conclude, based on my behavior, that they?re the best movies for me. Instead, Netflix should warn me when a show will be disappointing, and I?ll decide if I?m curious enough to watch it anyway.
In a radical move, Netflix won?t trust your ratings anymore because (I quote) ?there?s a difference between what you rate highly and what you actually like watching?. As if we were children, Netflix now decides for us what we should watch? with a strong bias towards its own shows. Netflix is clearly messing with the percent match scores, and the personalized recommendations have been replaced by marketing tactics.
Netflix was a precursor in movie recommendations, and for a long time it?s been the undisputed leader. But today it?s probably heartbreaking for the Netflix engineers to see their advances going down the drain like that. Nonetheless, we shouldn?t be too angry at Netflix management. Netflix spends BILLIONS of dollars each year to create and promote their original contents. Those huge investments could quickly turn into a loss if nobody watched them.
The change happened more than a year ago and Netflix is stronger than ever. The initial uproar didn?t last very long. Even if it?s hard to admit, Netflix probably took the best decision for their business. Do Netflix a favor, watch their original contents. Many of them are worth your time, it?s just a shame that it?s become harder to sort the good from the bad.
An alternative recommendation system.
Would you like Netflix to go back and revert to their old recommendation system? Don?t hold your breath, it?ll probably never happen. The dumb rating system is here to stay, but there?s an alternative?
Let me introduce Coollector Movie Database, a free Mac/Windows app:
https://www.coollector.com
It?s not an alternative to Netflix itself, there aren?t any shows to watch. But it?s an alternative recommendation system that can help you to decide what to watch on Netflix.
It?s almost like switching back to the good old Netflix recommendation system: no more thumbs, you can give accurate ratings to the shows (even more nuanced than with the 5 stars rating if you want). No more percent match score, the system predicts your rating. A big difference is that you can rate and get recommendations for any show, not only those available on Netflix. For example, the app can help you to decide what to watch on TV, in theaters, on Hulu? The Netflix catalog is much easier to browse, and you can filter by genre, by year, by country, by running time, etc? or you can order by IMDb popularity, by year, etc… The app supports the Netflix catalogs from 25 countries, updated daily. As a bonus, you can mark a show as ?not interested?, another useful feature that Netflix removed a long time ago.
Did I mention it?s free? Don?t hesitate to give it a try!
https://www.coollector.com
