Who cares about coin flips

Who cares about coin flips? Well, virtually no one. However, (a) coin flips are a great simple application to get the hang of Bayesian analysis; (b) the kinds of problems that a beta prior and a binomial likelihood function solve go way beyond assessing the fairness of coin flips. We are now going to apply the same technique to a real life problem that I actually came across in my work.

For my job, I had to create a career recommendation system that asked the user a few questions about their preferences and spat out some careers they may be interested in. After a few hours, I had a working prototype. In order to justify putting more resources into improving the project, I had to prove that I was on to something and that my current recommendations performed better than chance.

In order to test this, we got 40 people together, asked them the questions, and presented them with two sets of recommendations. One was the true set of recommendations that I came up with,...