Codex Router community docs
English
English中文日本語
GitHub
Last updated

Local LLMs in Codex — Run Ollama & GGUF Models Locally

Models running on your own machine can appear in Codex’s model picker like any other provider. This page covers the local-models commands: which model to download, how to verify it actually works, and how to use a local model as the vision reader for text-only providers.

Local LLMsexperimentalfree · offline

Enable local models

Open the tray’s Model Settings → Local LLMs, check the models you want, then fully quit and reopen Codex. The CLI equivalents are:

./bin/control local-models list                  # installed, plus what to download
./bin/control local-models install llama3.2:3b   # download, with progress
./bin/control local-models set llama3.2:3b on    # publish it to Codex
./bin/control local-models uninstall llava --yes # delete it from disk

Checking, installing and removing are three separate actions on purpose: unchecking never deletes a download, and removing needs explicit confirmation. The local provider turns itself on with the first checked model and off when the last one clears.

Which model should I get?

local-models list answers that question instead of making you memorize tags. It groups models by what they are good for, with size and status:

For coding — experimental. Codex's prompt uses about 20K of the 32K window:

  llama3.2:3b          2.0 GB verified  ran a real tool call through Codex
  qwen2.5-coder:1.5b   1.0 GB untested  smallest coder
  devstral            14.3 GB untested  built for agents

For reading images only — cannot code:

  qwen2.5vl:3b         3.2 GB  accurate
  moondream            1.7 GB  captions-only

Verify a model actually works

The tool template is a floor, not a prediction. What settles it is running the real client:

./bin/control local-models agent-check llama3.2:3b

This runs codex exec in a scratch workspace twice and requires both runs to verify a marker file that only exists there — proof the model dispatched a tool and read real output. Both runs must pass; a mixed result is reported as flaky.

Be realistic about the context window

Every local model is advertised to Codex at 32K, and Codex’s own instructions and tool definitions take about 20K of that before your code is added — so roughly 12K is left to work in, whatever the model natively holds. Tool support and native context are read from the model’s own files (the chat template and GGUF header, about a megabyte of ranged requests), which is how phi4 turns out to hold 16K rather than the 128K its family suggests.

Inspect before you download

./bin/control local-models inspect llama3.2:3b   # tools:true  context:131072
./bin/control local-models inspect phi4          # tools:false context:16384

The same lookup carries the download size and a memory-fit estimate — unified memory on Apple Silicon, GPU memory where NVIDIA reports it, and system RAM otherwise, with about 20% on top for context and cache:

fitMeaning
fitsRuns at full speed
tightRuns, but spills onto the CPU and is slow
too-largeCannot run on this machine

install refuses a too-large model before downloading anything. A tight model warns and proceeds — that one is a judgement call, not a wall.

Codex needs tool calling

Codex drives every turn through tool calls, so a model without them fails on its first request. Only models Ollama reports as tool-capable are published to the picker; the rest stay installed and stay usable as vision readers, labelled “no tools — vision only”.

Use a local model as the vision reader

The Vision Bridge is on by default: paste a screenshot into a text-only model and a vision-capable model reads it first, as text, before the text-only model ever sees it. If every provider you have is text-only, point the bridge at a small vision model running on your own machine — it costs nothing, the image never leaves your computer, and it works offline.

From the CLI, list, pull and pin one:

./bin/control vision-bridge models
./bin/control vision-bridge pull qwen2.5vl:3b   # downloads via Ollama, then pins it
./bin/control vision-bridge local qwen2.5vl:3b  # pin the local reader

The bridge does not run the model itself — it POSTs to any OpenAI-compatible /v1/chat/completions endpoint with no credential. Ollama, llama.cpp and LM Studio all work; vision-bridge probe detects the common runtimes and reports what your hardware suggests (roughly moondream under 8 GB, qwen2.5vl:3b at 8 GB, qwen2.5vl:7b at 16 GB+).

Not all vision models can read: most small ones describe the scene convincingly and invent the codes and numbers. The picker labels accuracy from measurement, not reputation — run node src/vision-benchmark.mjs to score every installed model against a known invoice image.