Run an AI coding agent fully offline with Ollama or LM Studio
Install OHarness, start Ollama (port 11434) or LM Studio's server (port 1234), and run oharness -m ollama/<model>. Both runtimes are built-in providers that need no key, so every model request stays on your machine.
Updated
Ta strona jest napisana po angielsku. Terminy, które wyjaśnia, są tymi, których szuka się po angielsku, a przetłumaczony termin fachowy to już inny termin.
What you need
- Node 22.6 or newer, for the CLI.
- Ollama or LM Studio, with a model that supports tool calling. An agent that cannot call tools can only talk about your code.
- Enough memory for the model. A 20–30B coding model is the practical floor for agent work; smaller models answer, but lose track of multi-step edits.
Set it up with Ollama
- Install the CLI.
- Pull a model that calls tools, for example
qwen3-coder:30borgpt-oss:20b. - Start the agent on it. The part before the first slash is the provider, the rest is the model name exactly as Ollama lists it.
npm install -g oharnessollama pull qwen3-coder:30boharness -m ollama/qwen3-coder:30bThere is no key and nothing to configure: the ollama provider points at http://localhost:11434/v1 and asks the server for its model list, so anything you have pulled appears under /models.
Set it up with LM Studio
Load a model in LM Studio and start its local server (the Developer tab, or lms server start). LM Studio model ids contain a slash of their own, which is fine — only the first slash separates the provider:
oharness -m lmstudio/qwen/qwen3-coder-30bGive the model enough context
Agent sessions are long: every file read and every tool result goes back into the context. Ollama serves a context much shorter than most models support unless you raise it, and when a local server does not report its window OHarness assumes 128K — so compaction fires later than the server actually truncates. Raise the server's window before a long session:
OLLAMA_CONTEXT_LENGTH=32768 ollama serveIn LM Studio the same setting is the context length slider when the model is loaded. 32K is a workable minimum for agent work; more is better if memory allows.
What still touches the network
| Traffic | When | How to turn it off |
|---|---|---|
| Model requests | Never — they go to localhost | — |
| Update check | At most once a day, to the npm registry | OHARNESS_NO_UPDATE_CHECK=1 |
| Session sync | Only when signed in to an OHarness account | Do not sign in, or /sync off |
| MCP servers | Only the ones you configure | Use stdio servers, or none |
When a local model is the wrong choice
Local models are free per token and private, and they are behind the best hosted models at long multi-file changes. A reasonable split: local for exploration, reading and small edits; a hosted model for the large refactor — switched in the same session with /model, which converts nothing. Switching models mid-session covers how.
Common questions
Can an AI coding agent run completely offline?
Yes. With OHarness pointed at Ollama or LM Studio, the agent loop, the tools and the model all run on your machine. The only other traffic is a once-a-day npm update check, which OHARNESS_NO_UPDATE_CHECK=1 turns off, and sync, which runs only when you sign in.
Which local models work for a coding agent?
Models that support tool calling, ideally at 20–30B parameters or more. Coding-tuned models such as Qwen3-Coder and general models such as gpt-oss both work; models without tool calling can answer questions but cannot edit files or run commands.
Do I need an API key to use OHarness with Ollama?
No. The ollama and lmstudio providers need no key and are built in, pointing at localhost:11434 and localhost:1234. Start the server and run oharness -m ollama/<model>.
Why does a long session with a local model forget earlier work?
Usually the server's context window is smaller than the model's. Ollama serves a short window by default; raise it with OLLAMA_CONTEXT_LENGTH (or num_ctx), and OHarness's compaction will summarise at a safe point instead of the server truncating silently.