Pairwise measure
In this section, we will propose some measures of agreement between two classifiers. The intention is to fix the notions of agreement/disagreement for two classifiers and then take the concept to the overall classifiers of the ensemble in the next section. If 
and 
are classifier models with predictions 
, we can then obtain a table that gives us the following:
- predicts
as 1; predicts it as 1
- predicts
as 1; predicts it as 0
- predicts as 0; predicts it as 1
- predicts as 0; predicts it as 0
The information across the N observations can be put in a tabular form, as follows:
|
M1 predicts 1 |
M1 predicts 0 | |
|---|---|---|
|
M2 predicts 1 |
n11 |
n10 |
|
M2 predicts 0 |
n01 |
n00 |
Table 2: Contingency table for two classifiers/raters
The diagonal elements of the preceding table show the agreement between the two models/classifiers, while the off-diagonal elements show the disagreement. The models are sometimes referred to as raters. The frequency table is also known as the contingency table. Using...
