We'll go over the concepts of Thompson sampling and Bayesian control problems at a high level, but they are well worth exploring further as an extension of the topics covered in this book.
Thompson sampling and the Bayesian control rule
Thompson sampling
Essentially, Thompson sampling has us believing what the prior probability distribution is and updating it every time we get new information about the environment. Eventually, our updated belief will coincide with the true probability distribution. This approach is fundamentally Bayesian. This is because it treats the probability distribution as our current lack of knowledge about the environment and updates it according to the new information we get.
In Thompson sampling...
