Naive Bayes

Naive Bayes is a classification algorithm based on Bayes' probability theorem and conditional independence hypothesis on the features. Given a set of m features, , and a set of labels (classes) , the probability of having label c (also given the feature set x_i) is expressed by Bayes' theorem:

Here:

• is called the likelihood distribution
• is the posteriori distribution
• is the prior distribution
• is called the evidence

The predicted class associated with the set of features will be the value p such that the probability is maximized:

However, the equation cannot be computed. So, an assumption is needed.

Using the rule on conditional probability , we can write the numerator of the previous formula as follows:

We now use the assumption that each feature x_i is conditionally independent given c (for example, to calculate the probability of x₁ given c, the knowledge of the label c makes the knowledge of the other feature x₀ redundant, ):

Under this assumption, the probability...