What are some LLM Observability Solutions?
There are niche players offering platform-based LLM observability solutions for enterprises, such as Arize and Aporia . Additionally, this year 2023, Datadog , a major player in the Observability field, introduced a beta version of a comprehensive LLM observability solution to their system. This solution gathers data from applications, models, and various integrations to expedite the identification and resolution of real-world application issues including model cost spikes, performance degradation, drift, hallucinations and many more, ultimately ensuring a positive end-user experience.
There is a popular open-source solution available called LangKit ( https://github.com/whylabs/langkit ). This open-source Python module offers the capability to extract relevant signals from...