The slowest first token of 8 agent requests to Qwen2 on Metal, with continuous batching.
Qwen2.5-0.5B · Candle · Apple M4 Pro
Inertia Labs is an independent research lab.
Small models that run where your data lives. The code is open at github.com/inertialabsxyz/fornace.
| No. | Result | Claim | Date |
|---|---|---|---|
| 09 | 8.3 → 0.9 s | Slowest first token of 8 agent requests, with continuous batching | 13 Nov 2026 |
| 08 | 2.5× | llama.cpp over Fornace with 8 agent requests at once, 1.1× with one request | 13 Nov 2026 |
| 07 | 1.35× | Faster decode on code with a 0.5B draft, 38 % slower on prose | 12 Nov 2026 |
| 06 | 2.7× | Sooner first token with a shared system prompt, no paged cache needed | 12 Nov 2026 |
| 05 | 1.85× | Faster decode with Q4 weights, at a 23 % perplexity cost | 9 Nov 2026 |
| 04 | 1.6× | Total decode at batch 4, not 4×. Each sequence runs at 0.40× | 5 Nov 2026 |
| 03 | 4.5× | Faster 4k-token prefill with fused attention, and less than half the peak | 4 Nov 2026 |
| 02 | 14.5 → 3.4 GiB | Peak for a 4k-token prefill, with a cap on Metal buffer reuse | 4 Nov 2026 |
| 01 | 2.4× | Faster decode at 2k context from one .contiguous() call |
29 Oct 2026 |
For a research collaboration, a project, or more information, send a message.