Evidently AI

Evidently AI is an open source framework for evaluating, testing and monitoring AI systems. It covers LLM applications, RAG pipelines, AI agents and predictive machine learning models in a single library, released under the Apache 2.0 license. It is built for teams that need to check an AI system for hallucinations, edge cases, PII leaks, risky outputs, jailbreaks and data drift on every update. A library of more than 100 built in metrics covers adherence to guidelines and format, factuality, PII detection, retrieval quality and context relevance, and sentiment, toxicity and tone. Teams can combine rules, classifiers and LLM based judges, or write custom evaluations with any prompt, model or rule. Synthetic data generation creates realistic, edge case or adversarial inputs, and a live dashboard tracks results over time so drift and regressions show up early. Results can be explored in shareable visual reports. The framework is free to use under its open source license. The vendor also offers a hosted product, and its pricing is not published on the pages it shows, so the contact link is the route for enquiries.