Bayes factors and information criteria

We have already said that Bayes factors are more sensitive to priors than many people like. It is like having differences that are practically irrelevant when doing inference but that turn out to be important when computing Bayes factors. And this is one of the reasons many Bayesians do not like Bayes factors.

Now we are going to see an example that will help clarify what Bayes factors are doing and what information criteria are doing. Go back to the definition of the data for the coin flip example and now set 300 coins and 90 heads; this is the same proportion as before but we have 10 times more data. Then run each model separately:

with pm.Model() as model_BF_0:
    theta = pm.Beta('theta', 4, 8)
    y = pm.Bernoulli('y', theta, observed=y)

    trace_BF_0 = pm.sample(5000)
chain_BF_0 = trace_BF_0[500:]
pm.traceplot(trace_BF_0);

with pm.Model() as model_BF_1:
    theta = pm.Beta('theta', 8, 4)
    y = pm.Bernoulli('y', theta, observed=y)
    trace_BF_1...