Benchmark: daari vs raw Ollama vs frontier price

Generated by python scripts/bench_compare.py. Same prompt IDs as the live product benchmark, so rows join across the two pages. The frontier column is gpt-4o list rates priced onto the token counts daari recorded — no paid API is called.

• Date: 2026-08-20
• Commit: a0493c7
• Hardware: Apple M4 Pro, 48 GB RAM
• Ollama: 0.18.2 (raw path model: llama3.2:3b)
• Reference frontier price: gpt-4o list rates

Aggregate

• $0-tier rate: 100% of served requests
• Median speedup on cache hits: 43x over raw Ollama (1 hit row(s) in a single cold pass)
• Implied frontier spend for this corpus: $0.03387 (priced, not billed)
• daari spend: $0.00000

Per-prompt comparison

ID	raw Ollama ms	daari ms	daari tier	daari no-cache ms	implied frontier USD	daari USD
GP-01	3515	2362	L3	230	$0.00070	$0.00000
GP-02	3970	2798	L3	2101	$0.00247	$0.00000
GP-03	407	37	L2	4	$0.00009	$0.00000
GP-04	331	803	L3	771	$0.00070	$0.00000
GP-05	256	6	L0	232	$0.00070	$0.00000
GP-06	1455	50	Lt	27	$0.00009	$0.00000
GP-07	2299	79	Lt	68	$0.00156	$0.00000
GP-08	11772	—	excluded	—	—	—
GP-09	1790	2081	L3	1515	$0.00179	$0.00000
GP-10	11138	11072	L3	14697	$0.00950	$0.00000
GP-11	1961	1494	L3	870	$0.00118	$0.00000
GP-12	648	4160	L4	501	$0.00032	$0.00000
GP-13	5247	602	L3	4437	$0.00030	$0.00000
GP-14	396	443	L3	2130	$0.00037	$0.00000
GP-15	748	865	L3	1413	$0.00074	$0.00000
GP-16	6922	1235	L4	5418	$0.00075	$0.00000
GP-17	195	238	L3	229	$0.00015	$0.00000
GP-18	7760	9784	L3	9621	$0.00844	$0.00000
GP-19	4133	5	CCS	32	$0.00009	$0.00000
GP-20	5112	4740	L3	5635	$0.00394	$0.00000

Rows marked excluded expect the frontier tier; the default run never calls a paid API (--live-frontier opts in).