Exercises

1. In the kernelized regression example, try changing the number of knots and the bandwidth (one at a time). What is the effect of those changes? Try also using a single knot; what do you observe?
2. Experiment with fitting other functions using kernelized regression. For example y = np.sin(x) + x**0.7 or y = x. Using these functions changes the number of data points and parameters like in Exercise 1
3. In the example where we sample from the GP prior increase the number or realizations, by replacing:

   plt.plot(test_points, stats.multivariate_normal.rvs(cov=cov, size=6).T)

   with

   plt.plot(test_points, stats.multivariate_normal.rvs(cov=cov, size=1000).T, alpha=0.05, color='b')

   How does the GP prior look? Do you see that f(x) is distributed as a Gaussian centered at 0 and standard deviation 1?

4. For the GP posterior using the Gaussian kernel, try defining test_points outside the interval [0, 10]. What happened outside the data interval? What does this tell us about extrapolating results (especially...