Open-source observability & evaluation for AI agents

Trace, evaluate, debug, and optimize AI applications and coding agents with OpenTelemetry.

⭐ Star     🚀 Quickstart     📚 Docs

Documentation · Quickstart · Examples · Join Slack

See what your AI agents are actually doing

AI applications are no longer just LLM calls.

A production agent can involve:

flowchart TD
    U([User]) --> A[AI Agent]
    A --> L[LLM calls]
    A --> T[Tool calls]
    A --> R[Retrieval]
    A --> M[Memory]
    A --> S[Sub-agents]
    A --> P[Prompts]
    A --> C[Code changes]
    L & T & R & M & S & P & C --> E{{Evaluation}}
    E --> O[["Cost / Quality / Errors"]]

    style U fill:#F97316,stroke:#7C2D12,color:#fff
    style A fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style E fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style O fill:#F97316,stroke:#7C2D12,color:#fff

OpenLIT gives you visibility across the entire workflow.

Trace every LLM call, tool invocation, prompt, agent step, token, cost, error, and evaluation — using OpenTelemetry.

⚡ Get started in 5 minutes

1. Start OpenLIT

git clone https://github.com/openlit/openlit.git
cd openlit

docker compose up -d

Open:

http://127.0.0.1:3000

2. Install the SDK

Python:

pip install openlit

TypeScript:

npm install openlit

3. Instrument your application

Python:

import openlit

openlit.init()

That's it.

OpenLIT automatically instruments supported LLM providers, frameworks, vector databases, and other AI infrastructure and exports OpenTelemetry traces and metrics.

4. Send telemetry

By default, configure the OTLP endpoint:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4318"

Or:

import openlit

openlit.init(
    otlp_endpoint="http://127.0.0.1:4318"
)

Open your dashboard and start exploring your AI application's traces, metrics, costs, and performance.

🤖 Observe Claude Code, Cursor & Codex

AI coding agents are powerful — but understanding what they actually did can be difficult.

OpenLIT gives you an OpenTelemetry-native view of coding-agent sessions.

Install the CLI:

macOS / Linux

curl -fsSL https://raw.githubusercontent.com/openlit/openlit/main/cli/scripts/install.sh | sh

Windows

iwr -useb https://raw.githubusercontent.com/openlit/openlit/main/cli/scripts/install.ps1 | iex

Configure OpenLIT:

openlit configure --endpoint http://127.0.0.1:4318

Install coding-agent instrumentation:

openlit coding install --vendor=all

Or install individual integrations:

openlit coding install --vendor=cursor
openlit coding install --vendor=claude-code
openlit coding install --vendor=codex

Check your installation:

openlit doctor

Now OpenLIT can capture:

flowchart LR
    S([Coding Agent Session]) --> P[User prompt]
    S --> L[LLM calls]
    S --> T[Tool calls]
    T --> T1[File reads]
    T --> T2[File edits]
    T --> T3[Shell commands]
    T --> T4[Search]
    S --> SA[Sub-agent activity]
    S --> TU[Token usage]
    S --> CO[Cost]
    S --> CI[Code impact]

    style S fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style T fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff

Explore the resulting sessions in the Coding Agents dashboard.

🔍 What OpenLIT gives you

Traces

Understand exactly what happened during an AI request.

All represented using OpenTelemetry.

💰 AI cost observability

Track the cost of your AI applications across:

Support custom pricing for custom and fine-tuned models.

🧪 AI evaluations

Automatically evaluate LLM and agent outputs using LLM-as-a-Judge evaluations.

Built-in evaluation types include:

Use evaluations to move from:

"The agent produced an answer."

to:

"The agent produced a good answer."

🐛 Debug production AI

Find the requests that matter.

Investigate:

Go from:

Something went wrong.

to a fully traced root cause:

flowchart TD
    A[Agent] --> P[Prompt] --> L1[LLM] --> T[Tool call] --> R[Retrieval] --> L2[LLM] --> E([Error])

    style E fill:#DC2626,stroke:#7F1D1D,color:#fff
    style A fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff

📸 See OpenLIT in action

Traces — full agent conversation, spans & cost	Evaluations — hallucination, bias & toxicity checks	Connectors — plug in ClickHouse, Tempo, Loki, Prometheus & more
Prompt Hub — versioned, centrally managed prompts	Rule Engine — conditional rules on trace attributes	Dashboards — cost, latency & usage charts at a glance

🧠 Prompt management

Use Prompt Hub to:

Example:

prompt = openlit.prompts.get(
    "customer-support"
)

Keep prompt management separate from application code while maintaining version control and observability.

⚙️ Rule Engine

Define runtime rules based on trace attributes.

Use rules to dynamically control:

Example:

IF
  environment = production
  AND
  model = expensive-model

THEN
  run cost evaluation
  + retrieve production prompt

🔌 OpenTelemetry-native

OpenLIT is built around OpenTelemetry, rather than creating a proprietary telemetry format.

Your telemetry can flow through the OpenTelemetry ecosystem:

flowchart TD
    A["AI App / AI Agent"] -->|OpenTelemetry| C[OpenTelemetry Collector]
    C --> B[OpenLIT Backend]
    C --> O["Other OTel backends<br/>(Datadog, Grafana, Honeycomb, ...)"]
    B --> D[OpenLIT Dashboard]

    style A fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style C fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style B fill:#F97316,stroke:#7C2D12,color:#fff
    style D fill:#F97316,stroke:#7C2D12,color:#fff

This means you can integrate OpenLIT into an existing OpenTelemetry architecture instead of replacing it.

🧩 70+ integrations

OpenLIT auto-instruments a growing ecosystem of AI providers, frameworks, vector databases, and GPU infrastructure with a single line of code. Click any badge to view its integration guide.

LLM Providers

Vector & Data Stores

AI Frameworks & Agents

Governance & Protocols

GPU Monitoring

See the complete integration list in the documentation.

🛠️ SDKs

OpenLIT provides OpenTelemetry-native SDKs for:

Python

pip install openlit

Python SDK →

TypeScript

npm install openlit

TypeScript SDK →

Go

Go SDK →

🏗️ Architecture

OpenLIT is designed to run in your infrastructure.

A typical deployment looks like:

flowchart TD
    subgraph App["Your application"]
        direction LR
        Agent --> LLM --> Tools --> RAG --> DB
    end
    App -->|OpenTelemetry| Collector[OpenTelemetry Collector]
    Collector --> CH[(ClickHouse)]
    CH --> Dash[OpenLIT Dashboard]

    style App fill:#1F2937,stroke:#F97316,stroke-width:2px,color:#fff
    style Collector fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style CH fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style Dash fill:#F97316,stroke:#7C2D12,color:#fff

🔐 Self-hosted by default

Run OpenLIT inside your own infrastructure using Docker or Kubernetes.

Your telemetry stays under your control.

docker compose up -d

For Kubernetes, see the installation documentation.

🚀 From trace to optimization

Observability is only the beginning.

OpenLIT is designed around a continuous AI engineering loop:

flowchart LR
    T[Trace] --> E[Evaluate] --> A[Analyze] --> O[Optimize] --> M[Manage] -.-> T

    style T fill:#F97316,stroke:#7C2D12,color:#fff
    style E fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style A fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style O fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style M fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff

The goal is simple:

Make AI systems observable, measurable, debuggable, and continuously improvable.

🌎 Community

OpenLIT is open source and built with the AI engineering community.

Join us:

• GitHub
• Documentation
• Slack
• X / Twitter

If OpenLIT is useful to you, please consider giving the repository a ⭐.

It helps other AI engineers discover the project.

🤝 Contributing

Contributions are welcome.

You can contribute by:

• fixing bugs
• adding integrations
• improving documentation
• creating examples
• improving SDKs
• adding evaluations
• building dashboards
• reporting issues
• sharing OpenLIT with other developers

Check the repository's issues for opportunities to contribute.

📄 License

OpenLIT is licensed under the Apache License 2.0.

See LICENSE for details.

🙇 Acknowledgments

This project is proudly supported by:

💻 Contributors