Lost objects and inactive PGs

This section of the chapter will cover the scenario where a number of OSDs may have gone offline in a short period of time, leaving some objects with no valid replica copies. It's important to note that there is a difference between an object that has no remaining copies and an object that has a remaining copy, but it is known that another copy has had more recent writes. The latter is normally seen when running the cluster with min_size set to 1.

To demonstrate how to recover an object that has an out-of-date copy of data, let's perform a series of steps to break the cluster:

1. First, let's set min_size to 1; hopefully by the end of this example, you will see why you don't ever want to do this in real life:

sudo ceph osd pool set rbd min_size 1

2. Create a test object that we will make later make Ceph believe is lost:

sudo rados -p rbd put lost_object logo.png...