Trulens

TruLens is an open source evaluation and tracing library for AI agents and retrieval augmented generation apps. It instruments an application with OpenTelemetry, records every step as a span with latency, inputs, outputs, tokens and cost, and then scores each step with LLM judges that explain their scores, so a poor answer can be traced to a specific cause. The tool is aimed at teams building agents and RAG pipelines who want metrics in place of guesswork. Built in metrics cover the RAG triad of context relevance, groundedness and answer relevance, along with agent focused checks. Judges can be tailored to a domain by adding a rubric, examples and a custom score range, and teams can write their own metrics in Python by selecting the parts of a record to score. Version 2.12 added tracking and evaluation of complete multi turn conversations. The project cites published benchmark results for its judges against human annotations, and it is designed to compare application versions so a team can decide which one to ship and where cost can be cut without losing quality. TruLens is free and open source, installed from the project's package and repository. The site has no pricing page and lists no paid plans.