A small local model can produce an answer quickly. That does not yet tell me whether using it saved work.

I would judge a local agent by the useful result after verification.

Give it a job with a finish line

“Improve this repository” is difficult to assess. “Locate the configuration loader and identify the callers that depend on its defaults” has a clearer output.

A bounded task makes it easier to provide relevant context and check the answer. It also makes a failed attempt easier to discard.

Compare the whole workflow

For a repository question, I would compare two routes:

• Find and verify the answer with ordinary search and inspection.
• Ask the local agent, then verify its answer against the same source.

The comparison should include preparation, generation, review, corrections, and unsuccessful attempts. If setup is reused across many tasks, report it separately rather than hiding it.

A reduction in remote-model tokens can be useful. It is not the same measurement as reduced time or improved correctness.

Keep the useful failure cases

An agent that names a plausible but nonexistent function creates work. An agent that points to the correct file but misreads a condition still needs correction.

Those cases belong in the assessment alongside successful answers.

Local Agent Toolkit provides a bounded interface for this kind of work. Its existence does not establish a fixed saving across models or tasks.

The question I care about is practical: after checking the result, am I further ahead?