Anomaly detection in Amazon OpenSearch Service

Amazon OSS’s anomaly detection feature uses the Random Cut Forest (RCF) algorithm to detect anomalies in OpenSearch log data in real time. RCF is an unsupervised machine learning algorithm that models a sketch of your incoming data stream. It calculates an anomaly score and level of confidence for each incoming data point. Anomaly detection then uses these scores to distinguish abnormal data from normal variations. You can also utilize the pre-created anomaly detection features such as monitoring HTTP responses, monitoring e-commerce orders, and monitoring host health.

Let’s go ahead and configure anomaly detection for the container logs deployed from the sample application:

1. Navigate to Amazon OSS | OpenSearch Plugins | Anomaly detection | Create detector:

Figure 11.38 – Create detector

2. Let’s set the detector’s Name field to ContainerAppLogs, and for...