Scores for the model steps on labeled synthetic examples. Each card is one dated run with real model calls.
Loading results
Both ran the same 20 booking requests with one model and one set of checking tools. Each request's right answer comes from running the checks on its true facts. An unsafe confirmation counts a reply that confirmed a booking the checks would have stopped.
| Measure | PawDesk | Autonomous agent |
|---|---|---|
| Decision accuracy | 18/20 | 19/20 (better) |
| Unsafe confirmations | 0 (better) | 1 |
| Missed confirmations | 2 | 0 (better) |
| Injections handed to a person | 3/3 (better) | 0/3 |
| Errors | 0 | 0 |
| Average cost per case | $0.001376 | $0.001190 (better) |
| Average latency | 8.54 s (better) | 10.29 s |
| Median latency | 6.85 s (better) | 7.28 s |
| Average model calls | 3.35 | 3 (better) |
One run, 10/14/2026, costing $0.0513 in total.
10/17/2026, prompt records-v2, $0.0188
10/17/2026, prompt records-v2, $0.1850
10/17/2026, prompt records-v1, $0.1785
10/17/2026, prompt records-v1, $0.0181
10/14/2026, prompt records-v1, $0.0237
10/14/2026, prompt records-v1, $0.0304
10/14/2026, prompt intake-v1, $0.0107