TruLens

Founded by Anupam Datta in 2023

Evaluate and trace AI agents and retrieval pipelines with OpenTelemetry instrumentation

Pricing model

Free

Rating

★ People love it

Time to value

Quick Setup (< 1 hour)

You can use TruLens to inspect, evaluate, and monitor your AI agents and retrieval-augmented generation pipelines. It instruments your code with OpenTelemetry to track execution traces, latency, inputs, outputs, tokens, and inference costs at every step. You can apply automated judges to score outputs on groundedness, context relevance, answer relevance, and safety. TruLens lets you run evaluations live in production or across test datasets, compare multiple application versions side by side on a leaderboard, and detect exactly where your agents fail or produce inaccurate answers.

What TruLens does

Instrument agent code using OpenTelemetry decoratorsScore query responses on groundedness and relevanceTrace multi-turn conversational agent execution pathsCompare metric averages across different application versionsRun batch evaluation runs against DataFrame datasetsDefine custom scoring rubrics with few-shot examplesRecords latency, tokens, inputs, outputs, and costs per execution span using OpenTelemetryEvaluates retrieval-augmented generation using the RAG triad of groundedness, context relevance, and answer relevanceProvides benchmarked LLM judges that score execution steps and generate explanatory feedbackSupports custom evaluation rubrics with few-shot examples and configurable score rangesCompares app versions side by side on a leaderboard to track quality and cost tradeoffsRuns evaluations either live in production or asynchronously across offline datasetsAuto-instruments workflows built with LangGraph, LlamaIndex, or custom Python code