Monitoring node metrics
Whether you are using JMX monitoring software or the nodetool
utility, it is important to know what statistics are worth watching. The names and locations of specific attributes can vary depending on the Cassandra version, but the key ideas remain the same.
The objective here is to give you an understanding of the available statistics so that you will know how to choose the proper monitoring targets. We will use nodetool
for this purpose, as its options tend to be more stable. You should find it straightforward to locate the equivalent JMX mbean.
Thread pools
Cassandra's design employs a staged event-driven architecture (SEDA), which essentially comprises of message queues (containing events) feeding into thread pools (or stages). The stages fire off messages to other stages via a messaging service. There are stages for handling a variety of tasks.
Running nodetool tpstats
provides detailed information about what's happening at each of the stages. A buildup of pending...