A patient takes a special cancer test that has an accuracy of test_accuracy=99.9%—if the result is positive, then 99.9% of the patients tested will suffer from that particular type of cancer. Conversely, 99.9% of the patients with a negative result will not suffer from that particular cancer.

Suppose that a patient is tested and the result is positive. What is the probability of that patient suffering from that particular type of cancer?

We will use Bayes' theorem to ascertain the probability of the patient having cancer:

To ascertain the prior probability that a patient has cancer, we have to find out how frequently cancer...

In this section, we will state and prove both Bayes' theorem and its extension.

Bayes' theorem states the following:

We can prove this theorem by using elementary set theory on the probability spaces of the events A and B. In other words, here, a probability event will be defined as a set of the possible outcomes in the probability space:

As you can see in the preceding diagram, we can state the following relationships:

Rearranging these relationships...

Suppose we are given the following table of data. This tells us whether or not our friend will play a game of chess with us outside in the park, based on a number of weather-related conditions:

We would like to establish, by using Bayes' theorem, whether our friend would like to play a game of chess with us in the park given that the Temperature is Warm, the Wind is Strong, and it is Sunny.

In this section, we will implement a program calculating the probability of a data item belonging to a certain class by using Bayes' theorem:

# source_code/2/naive_bayes.py 
# A program that reads the CSV file with the data and returns
# the Bayesian probability for the unknown value denoted by ? to
# belong to a certain class.
# An input CSV file should be of the following format:
# 1. items in a row should be separated by a comma ','
# 2. the first row should be a heading - should contain a name for each
# column of the data.
# 3. the remaining rows should contain the data itself - rows with
# complete and rows with the incomplete data.
# A row with complete data is the row that has a non-empty and
# non-question mark value for each column. A row with incomplete data is
# the row that has the last column with the value...

Suppose that we would like to find out whether our friend would like to play chess with us in a park in Cambridge, UK. But, this time, we are given different input data:

So, now we are wondering how the answer to whether our friend would like to play in a park in Cambridge, UK, will change with this different data in regard to the Temperature being Warm, the Wind being Strong, and the Season being Spring.

So far, we have been given a probability event that belonged to one of a finite number of classes, for example, a temperature was classified as cold, warm, or hot. But how would we calculate the posterior probability if we were given the temperature in °C instead?

For this example, we are given the heights of five men and five women, as shown in the following table:

Suppose that the next person has a height of 172 cm. What gender is that person more likely to be, and with what probability?

In this chapter, we learned about Naive Bayes and how it can be applied in different ways for different purposes.

Bayes' theorem states the following:

Here, P(A|B) is the conditional probability of A being true, given that B is true. It is used to update the value of the probability that A is true given the new observations about other probabilistic events. This theorem can be extended to a statement with multiple random variables:

The random variables B1,...,Bn have to be conditionally independent given A. The random variables can be discrete or continuous and follow a probability distribution, for example, normal (Gaussian) distribution.

We also studied the discrete random variable. We learned that it is best to ensure that you have a data item for each value of a discrete random variable given any...