P1 · checkout-service.p99 > 2000ms for 3m
AI on-call for production incidents.
Tiravan investigates across your metrics, traces, logs, and deploys in one loop — and delivers a verified Causal Chain RCA before you finish reading the alert.
InvestigationINC-7194 · checkout-service p99 > 2000ms
Establish baseline p99 for the entrypoint service across the last hour, then confirm the elevation is real rather than a scrape gap.
prometheus_query_range200 · 89ms
histogram_quantile(0.99, sum by(le)(rate(http_request_duration_seconds_bucket{service="checkout"}[5m])))0ms1000ms2500ms
↑p99 climbed from 340ms → 2140ms at 14:32:00, sustained.
The p99 climbed sharply at 14:32:00 and stayed elevated. Fetch the trace tree at 14:32:12 to isolate the slowest downstream span.
otel_get_trace_tree200 · 142ms
trace_id=e2f8b1c9a04d · timestamp=2026-08-03T14:32:12Z
gateway2180ms
├─ auth-service40ms
├─ checkout-service2110ms
│ ├─ cart-service28ms
│ └─ payment-service1980ms
└─ notifications18ms
◉
payment-service is the slow leaf: 1980ms of the 2180ms wall.Pull logs from payment-service over ±30s of the slow span (14:31:42 → 14:32:42) filtered to error level to surface the fatal line.
loki_search_logs200 · 63ms · 47 hits
{service="payment-service", level="error"} | 14:31:42 → 14:32:4214:31:58WARN db.pool: acquiring connection took 812ms (pool_size=20, in_use=20)
14:32:04WARN db.pool: acquiring connection took 1180ms (pool_size=20, in_use=20)
14:32:11ERROR payments.charge: connection acquisition timeout after 2000ms · card_token=**** · idempotency_key=ord_7194
14:32:12ERROR payments.charge: connection acquisition timeout after 2000ms · card_token=**** · idempotency_key=ord_7195
14:32:12INFO deploy.observer: deploy sha=
bd42f1a to payment-service completed 14:31:47 (24s before first timeout)✕Fatal path:
connection acquisition timeout after 2000ms. Correlated with deploy bd42f1a, 24s prior.{ "summary": "checkout p99 elevated to 2.1s after payment-service deploy exhausted its DB connection pool.", "trigger": "deploy bd42f1a (payment-service) at 14:31:47", "effect": "checkout.p99 340ms → 2140ms; 47 charge timeouts in the window", "root_cause": "bd42f1a added a synchronous audit-log write on the charge path holding pool connections open for the full request; pool of 20 is now saturated at 3 rps.", "evidence_cell_ids": ["E-2180", "E-2185", "E-2191", "E-2197"], "suggested_action": "revert bd42f1a; move audit-log write off the request path (async)." }
Synthetic incident, real mechanism. The 3-turn pattern above (baseline → trace → logs) is the canonical investigation shape in §2 Method; the RCA payload's schema is machine-checkable and is what the API returns.
Investigates across every telemetry silo your on-call already reads.
- Prometheus
- OpenTelemetry
- Loki
- GitHub
Your on-call shouldn't need four browser tabs.
Every real incident has its root cause inside one telemetry silo — and every investigation stitches signals across four of them by hand. Tiravan does that traversal for you in one Claude-driven ReAct loop, per tenant, with credentials that never leave the plugin's stack frame.
Per-tenant tool injection
The agent's tool array is built per request from your tenant's flag-enabled MCP schemas. No global tool list. No cross-tenant leakage. Credentials decrypt inside the plugin's stack frame and never cache.
Causal Chain RCA
Every investigation ends in a machine-checkable payload:{summary, trigger, effect, root_cause, evidence_cell_ids}. Streamed live as it forms. Not a free-text summary; not a black-box classifier verdict.
Two named personas
Nakula wakes on the PagerDuty alert and posts a verified RCA to the incident thread. Sahadevasits in the workspace and answers the questions your dashboards don't. Twin brothers from the Mahabharata.
Symptom → Dependency → Isolation.
The canonical three-turn investigation pattern. Real tool calls, real data. Every RCA cites the evidence cells you can click through to.
Baseline the entrypoint. — Symptom
Nakula queries Prometheus for the service on the alert — pulls its p99 over the last hour, confirms the elevation is real, and marks the exact timestamp where it climbed.
METRICSprometheus_query_rangeWalk the trace tree. — Dependency
Fetches the OpenTelemetry trace at the slow timestamp, walks the span hierarchy, and mathematically isolates the slowest downstream leaf — the actual owner of the tail latency.
TRACESotel_get_trace_treegateway2180ms├─ auth-service40ms├─ checkout-service2110ms│ └─ payment-service1980msPull logs around the slow span. — Isolation
Loki search over ±30s of the slow span, filtered to errors, cross-referenced against the most recent deploy. Surfaces the fatal log line and correlates it to the change that shipped it.
LOGSloki_search_logs14:31:58 WARN db.pool: acquiring connection took 812ms14:32:11 ERROR payments.charge: connection acquisition timeout after 2000ms14:32:12 INFO deploy.observer: sha=bd42f1acompleted 24s prior
The loop terminates when a Pydantic-validated Causal Chain RCA object assembles. Median: three tool calls, twenty-two seconds, one verified answer posted to your incident thread.
Two modes of use, one loop, one brain.
Nakula
The alert-driven investigator — runs autonomously on the webhook.
Wakes on the webhook. Runs the ReAct loop end-to-end. Posts the assembled Causal Chain RCA to the incident thread before you finish reading the PagerDuty page. His entire operating envelope is one investigation, start to finish, no follow-up.
03:11:28
Nakula posts #inc-7194 → RCA: deploy bd42f1a saturated payment-service pool to Slack with evidence cells. The engineer glances at the phone and sleeps another five hours.
Sahadeva
The chat companion — answers the questions your dashboards don't.
Sits in the workspace. Answers questions your dashboards do not. Remembers every past incident. Cites evidence cell IDs you can click to inspect the underlying signals; Nakula does not carry that memory across sessions.
14:22
Why did checkout p99 spike last Tuesday around lunch? Sahadeva returns a paragraph citing [E-2431], [E-2437]. Every cell opens the exact signal that ended up in the RCA.
One workspace onboarded per week. Yours next?
Tiravan is in closed beta. Free while you're on it. Submit the email address you use for on-call and we'll provision a tenant when the roster allows. For engineering leaders evaluating the security model, book a 15-minute demo instead.
We don't have a logo wall or a testimonial section on this page because we don't yet have real customers to name. We won't fabricate those to fill space.
Prefer a walkthrough?Book a 15-minute demo →