Why and How to Run a Local Coding Agent on Your Own Machine
According to Ahead of AI, the author walks through building a fully local coding agent using open‑weight models and a lightweight serving stack.
Local coding agents let you keep code generation, file edits, and command execution inside your own hardware. That matters for anyone who worries about API costs, data privacy, or the risk of a cloud provider throttling a model.
The Stack in Plain Terms
A coding agent consists of two parts:
- The LLM (large language model) – the brain that writes code and reasons about changes.
- The harness – a thin program that gives the model access to your filesystem, a shell, and a way to report success or failure.
The harness is language‑agnostic; it merely forwards prompts and returns results. The LLM does the heavy lifting. In this tutorial the author pairs the Qwen‑Code harness with the Qwen3.6 35B‑A3B model, but the same harness can run other models such as Cohere’s North Mini Code.
Getting the Model Running
The author recommends Ollama as the serving engine because it works on macOS, Linux, and Windows with a single CLI. After installing Ollama you pull a model:
# Apple Silicon (optimised with Metal)
ollama pull qwen3.6:35b-mlx
# Generic Linux/Windows
ollama pull qwen3.6:35b
The ‑mlx variant compiles the model for Apple’s Metal performance shaders, cutting inference latency on M‑series chips.
The model size is 22 GB on disk and needs roughly 30‑40 GB of RAM to stay responsive during long‑context runs (up to 50 k‑word prompts). The author reports smooth operation on a Mac Mini with an M4 and on an NVIDIA DGX Spark.
Quick Speed‑and‑Memory Check
Before committing a model to a daily workflow, the author suggests a short benchmark that measures:
- Tokens per second (throughput)
- Prefill latency (time to load the prompt)
- RAM usage over long contexts
The script ollama_speed_memory_bench.py ships with the tutorial repo. A typical run on a Mac Mini looks like:
uv run speed-memory-benchmark/ollama_speed_memory_bench.py --model qwen3.6:35b-mlx
The test takes about five minutes and prints a table of speeds for prompts ranging from 1 k to 50 k words. If your machine has less than 30 GB of RAM you’ll need a smaller model, e.g., gemma4:e2b, which stays under 8 GB.
How Qwen3.6 Stacks Up
The article references two external benchmarks:
| Model | Size (GB) | RAM needed | Tokens/s (approx.) | Benchmark notes |
|---|---|---|---|---|
| Qwen3.6 35B‑A3B | 22 | 30‑40 GB | 12‑15 on M4, 20‑25 on DGX Spark | Tops Cohere’s North Mini Code in all but one metric (June 2024 North Mini Code report) |
| North Mini Code 1.0 | 22 | 30‑40 GB | 10‑13 on M4, 18‑22 on DGX Spark | Strong alternative, slightly slower on the same hardware |
The Polar: Agentic RL paper (May 2026) also shows the older Qwen3.5‑4B model leading the Qwen‑Code harness, suggesting the architecture is tuned for that harness. The author assumes the newer Qwen3.6 inherits those optimisations.
What Actually Changes – The Trade‑Off Nobody Highlights
Local agents replace recurring API fees with upfront hardware cost. If you already own a machine that meets the RAM requirement, the marginal cost per token drops to electricity and wear‑and‑tear – essentially zero. However, you now bear the burden of maintenance: OS updates, driver compatibility, and occasional model upgrades that can break your harness.
Predictability vs. flexibility. A cloud service offers instant scaling and automatic model upgrades. A local setup gives you a fixed version of the model, which is great for reproducibility but means you miss out on improvements unless you manually pull a new checkpoint. The author notes that a future GPT upgrade could solve a query you previously struggled with, but the same upgrade could also break scripts that depend on a specific output format.
Privacy vs. convenience. Running receipts or proprietary code through a local model eliminates data egress. The trade‑off is that you must trust the model’s own training data – open‑weight models can embed biases or outdated libraries, and you have no warranty on correctness.
Hardware ceiling. Even the “best” open‑weight model in the 35 B class still needs a high‑end laptop or a server. Teams without that hardware will fall back to smaller, less capable models, which the author describes as “pretty bad local coding agents.” In those cases a hybrid approach—cloud for heavy lifts, local for sensitive data—makes sense.
Actionable Steps for Today
- Install Ollama – follow the one‑click installer for your OS.
- Pull the MLX variant (if you have Apple Silicon):
ollama pull qwen3.6:35b-mlx. - Run the speed‑memory benchmark with the provided script to confirm the model fits your RAM budget.
- Create a test harness: copy the minimal
codex‑style JSON prompt that asks the model to add a comment to a Python file, then execute it via the harness’s CLI. - Record the time and compare it to a quick OpenAI Codex call (if you have an API key). You’ll now see the concrete speed and cost difference on your own machine.
By completing these five minutes you’ll know whether a local coding agent is viable for your daily workflow and you’ll have a repeatable script ready for future projects.
