The Netflix prize

In 2006, Netflix announced that it would award a $1,000,000 prize to the best recommender algorithm submitted that could outperform their own algorithm. Two years later, the prize was awarded to a team called BellKor for their Pragmatic Chaos system. Netflix never used the prize-winning Pragmatic Chaos system, explaining that a production version would be too expensive to implement. That prizewinner turned out to be a mix of over 100 different methods. Meanwhile, some of the top competitors went on to extend and market their own recommender systems. Some of the resulting algorithms have been patented.

The competition was open to anyone who registered. Data for testing proposed algorithms was provided by Netflix. The main dataset was a list of 100,480,507 triples: a user ID number, a movie ID number, and a rating number from 1 to 5. The data included over 480,000 customer IDs and over 17,000 movie IDs. That's a very large utility matrix, which is also very sparse: about 99...