Unlike the majority of Cephs features, which by default perform well for a large number of workloads, Cephs tiering functionality requires carefully configuration of its various parameters to ensure good performance. You should also have a basic understanding of your workloads I/O profile; tiering will only work well if your data has a small percentage of hot data. Workloads that are uniformly random or involve lots of sequential access patterns will either show no improvement or in some cases may actually be slower.
Tuning tiering
Flushing and eviction
The main tuning options should be looked at first are the ones that define the size limit to the top tier, when it should flush and when it should evict.
The following two configuration options configure the maximum...