Skip to content
Gauntlet
SOLARI LIVE

external agent baseline

SOLARI LIVE

Example Repository Agent · Complete Demo Checkout · run 725f3b4 · 3d ago

All runs
Reliability
0%0 / 1 runs passed

95% confidence interval 0.0% – 79.3% over 1 scored run

Baseline
0%
0%–79%
Perturbed
—
Runs
1
1 scored
Failed
1
no infra errors
p95 duration
2.4s
p50 2.4s
Avg steps
—
avg 2.4s
Run-to-run flips
0%
how often identical repetitions disagreed

Reliability by perturbation

Worst first. The whisker is the 95% Wilson interval — small samples are wide on purpose.

Baseline
Baseline
0%0 / 1

Reliability by category

Baseline
0%0 / 1

Failure distribution

Only scored failures. Infrastructure errors are excluded.

Agent error
1100%

Run matrix

Every cell is one run. Select one to open its evidence.

Outcome of every run, by perturbation and repetition. Select a cell to open that run.
PerturbationRun 1Rate
BaselineBaseline, run 1: Failed0%

Failure clusters

Failures grouped by category and cause, so a repeated defect reads as one problem.

  • Agent error

    Solari is at its concurrent-session limit. Lower GAUNTLET_MAX_CONCURRENCY or upgrade the plan.

    1 run · b862292

Individual failures

Open one to see the assertion that failed and the trace that led there.