Can Apple's on-device model save my coding agent tokens?
No :) One day, eight experiments with afmux and the macOS 27 fm tool. None of them cost less than letting the agent do the work itself.
macOS 27 ships Apple’s on-device model with a command line tool, fm. It is free, it runs locally and it is always there. My idea was simple: let Claude Code hand small jobs to it and pay for fewer tokens.
I built afmux to test this and spent a day measuring it. The answer is no. I tried eight different ways of using the local model, and none of them cost fewer tokens than the agent doing the work itself. The best setup was about 10% more expensive, the typical one 15 to 25%.
How I measured
All agent numbers come from the harness, not from the agent describing its own work: claude -p with JSON output for cost and token counts, and the token usage the Agent tool reports for the field tests. Controls ran the same prompt in a fresh directory. Correctness was graded against known answers or hidden unit tests.
Some facts about fm worth knowing before you build on it. The context window is 8,192 tokens in total - prompt, instructions and output together. Inputs over about 15k tokens come back as a “safety guardrails” refusal, not a size error. It is nearly deterministic and has no temperature or seed flag. And inside Claude Code’s sandbox fm respond fails with ModelManager error 1008, while fm available says everything is fine.
Attempt 1: pipe long output through the local model
The first design was a skill. The agent pipes long command output (test runs, logs, build output) into afm and reads a short answer instead of the whole thing.
In round one the inputs were 16k to 62k tokens, and afm never answered - everything was over its window. In round two I cut the inputs to 1.7k-6.9k tokens. It answered 4 of 4 and was right once. The log summary said three of four workers timed out when all four did. The build summary counted 13 warnings against 10. The commit message ignored a 176-line file. The agent caught every wrong answer with grep, so it paid twice, and cost went up 15% on Opus.
A skill costs the agent extra turns, so the next try was a Claude Code PostToolUse hook. It can rewrite Bash output before the model sees it, with no extra turns. I ran the same seven tasks three ways: no hook, a hook that only truncates, and a hook that summarises with fm.
Table
| No hook | $0.490 |
|---|---|
| Truncate only | $0.498 |
| fm summary | $0.491 |
The hook fired three times per arm. Opus rarely prints more than 4k characters of output - it greps, heads and tails on its own, and Claude Code already truncates Bash output at 30k characters. There was nothing left to save.
Attempt 2: the agent writes the spec, the local model writes the code
Next I flipped it around. The agent writes a short spec and a test for each step, afm run gets the local model to write the code, runs the test, retries once with the failure, and reports back.
First a benchmark without an agent: seven single-function specs with hidden tests.
Table
| On-device (fm) | 37 of 48 | 4 of 7 tasks fully passed, $0 |
|---|---|---|
| Haiku 5.5 | 48 of 48 | 7 of 7 tasks, $0.010 |
| Sonnet 5.5 | 48 of 48 | 7 of 7 tasks, $0.037 |
That was the best showing of the local model all day. Then the field test: five Opus agents built small modules through afmux (an INI parser, a log analyser, text utilities, date arithmetic without datetime, a CSV CLI), and five more built the same modules by hand. I ran four rounds, fixing whatever the agents complained about between them, and a fifth round on boilerplate-heavy code.
Round 3 was the “best” configuration: one module per step, with the spec written as pseudocode. But all five agents said the same thing - the spec they wrote was as long as the code. One wrote a 70-line spec for a 70-line module. In round 4 I asked for plain-English specs to fix that. The model then failed five of eight modules, the agents wrote them by hand, and the cost went back up.
The boilerplate round was the last hope: a short table in the spec, a long module out. The specs were a third to an eighth the length of the code. The model produced nothing usable in eight attempts - syntax errors, spec text pasted into the code, a module importing itself, and two timeouts on 250-line output. Opus wrote the same modules in 74 to 119 lines by factoring the repetition out.
Retries didn’t help either. Every retry produced byte-identical code, because fm has no usable randomness.
Attempt 3: review and tests after the fact
I took five working modules, planted a one-token bug in three of them, and asked the local model to review them. It found 0 of 3 bugs, said “no bugs” on all eight files, and made two false reports on correct code. As a test writer it produced no runnable tests for one module, 27 tests with 7 wrong for another (and missed the bug), and timed out on the third. A review the agent has to read and refute costs more than no review.
Attempt 4: question answering over a long file
The last mode borrows from HazyResearch’s MinionS: split a long file into 600-token chunks, ask the local model for a verbatim quote or nothing from each one, drop any quote that isn’t actually in the chunk, and give the agent the surviving quotes with line numbers. The agent never reads the file.
I asked five questions about a 23.5k-token documentation page, five runs each way, on Opus.
Table
| Agent alone | mean $0.198, range $0.177-$0.256 |
|---|---|
| afm read | mean $0.205, range $0.161-$0.219 |
Table
| Agent alone | 17-50 s |
|---|---|
| afm read | 154-521 s |
A wash on cost and 7 to 17 times slower. The agent alone never paid for the whole document anyway - Claude Code truncated it and the agent grepped for what it needed.
Where the tokens actually go
On Opus, 76% of an agent’s cost is cache writes. An extra turn costs about $0.01 and loading a skill about $0.045. The spend is the agent’s own turns, not text a local model could absorb.
I also looked at 631 of my own session transcripts. Subagents produce more than twice as many assistant messages as the main conversation, and 98.5% of those messages ran on Opus or Fable. Putting the Explore subagent on Haiku would touch far more of my token use than any local-model scheme.
What would change the answer
A local model that can be sampled - the FoundationModels framework exposes temperature, the fm tool does not. A stronger local model: the MinionS paper’s 8B model recovers 98% of cloud accuracy on extraction, and nobody has tested either on code. Or a workload where the agent really reads long prose whole, not structured files it can grep.
All the details of this failed experiment are in the full report in the afmux repo.