Local large model API for Codex usage tutorial: Ollama, LM Studio and vLLM

If you want Codex to use local large models, don’t plug any OpenAI compatible addresses directly into the project configuration. Currently Codex has a more reliable local model path: OSS mode. It natively supports selecting Ollama or LM Studio as a local provider.

The shortest command is:

1

codex --oss --local-provider ollama

or:

1

codex --oss --local-provider lmstudio

If you deploy a vLLM, LiteLLM, or other OpenAI-compatible gateway yourself, you can explore the advanced configuration of openai_base_url. However, this path requires the service to be truly compatible with the API behavior required by Codex, and the troubleshooting cost is higher. It should not be confused with the built-in OSS mode.

Choose the right route first

Ordinary individual users are recommended to run through Ollama or LM Studio first. Codex’s --oss will use the specified local OSS provider; if --local-provider is not passed and no default value is set, the interactive CLI will prompt you to choose, but codex exec will directly report an error.

Solution 1: Use Ollama to connect to Codex

1. Confirm that Ollama and models are available

First check if Ollama is available:

1
2

ollama -v
ollama ls

When there is no model, first download a code or general model suitable for the local video memory, for example:

1

ollama pull qwen3:8b

Test separately:

1

ollama run qwen3:8b

If the model cannot run in Ollama, first solve the problem of video memory, driver, model download or Ollama service; do not directly go to Codex for troubleshooting.

2. Single-use local model

Execute in the project directory:

1

codex --oss --local-provider ollama

Then enter the task as usual, for example:

1

阅读这个仓库的 README，列出本地启动步骤，不要修改文件。

This only affects the current session. If you want to temporarily switch back to the regular Codex, just start it without --oss.

3. Set Ollama as the default local provider

If you frequently use local models, put the following content in the user-level Codex configuration file:

1

oss_provider = "ollama"

You can then run directly:

1

codex --oss

User-level configuration of Codex is usually located under CODEX_HOME, and the default is ~/.codex/config.toml; common Windows paths are:

1

C:\Users\你的用户名\.codex\config.toml

Reopen the Codex after modification. If there is a complex configuration, back up config.toml first and add only this line. Do not overwrite the original sandbox, MCP, skills and other settings.

Option 2: Use LM Studio to connect to Codex

LM Studio is suitable for people who have downloaded the GGUF model and want to use the graphical interface to adjust the context and GPU offload.

1. Start the local service in LM Studio and load the model

Enter the Developer page of LM Studio, start the server, and confirm that a chat/instruct model has been loaded. LM Studio’s local API defaults to listening at:

1

http://localhost:1234

You can verify the model service first:

1

curl http://localhost:1234/v1/models

What is returned here is the LM Studio side model status; it helps confirm that both the service and the model are ready.

2. Start in Codex OSS mode

1

codex --oss --local-provider lmstudio

Long term default configuration:

1

oss_provider = "lmstudio"

Then use:

1

codex --oss

LM Studio’s model context length, GPU offload, and inference parameters are still managed by LM Studio. If the Codex response is slow, first check whether the model size exceeds the video memory, whether the context is set too long, and whether other local inference services occupy the GPU at the same time.

Option 3: Advanced connection method for OpenAI compatible APIs such as vLLM

vLLM, LiteLLM, enterprise gateways, and some agents provide an OpenAI-compatible /v1 interface. Codex’s official configuration reference provides openai_base_url, which overrides the base address of the built-in openai provider.

Schematic configuration:

1

openai_base_url = "http://127.0.0.1:8000/v1"

If the service is on a LAN host:

1

openai_base_url = "http://192.168.1.20:8000/v1"

There are four boundaries to pay attention to on this road:

1. **Written only at user level ~/.codex/config.toml. ** Codex will ignore openai_base_url, model_provider and model_providers in project .codex/config.toml to prevent the repository from secretly changing the machine’s model provider.
2. A service is not just “/v1/chat/completions” that is enough. Codex specific workflows may require models, streaming responses, tool calls, or other compatible behaviors.
3. Authentication is determined by your gateway. If the gateway requires a Bearer token, it should be set correctly according to the current authentication configuration of the gateway and Codex; do not write the token into the warehouse file.
4. This is not an official Codex listed local provider for OSS. When encountering an exception, first use Ollama or LM Studio to verify the Codex OSS mode, and then check the gateway compatibility.

The vLLM service can be independently verified first:

1

curl http://127.0.0.1:8000/v1/models

Only after the command returns a stable list of models does the codex continue to be checked for user-level openai_base_url.

How to choose a model

Whether a local model can “use” and whether it can be “as reliable as the official Codex model” are two different things. Code Agents typically require long context, stable tool calls, strong code understanding, and sufficiently fast generation speeds.

When selecting, look at at least:

• Whether the video memory can accommodate model weights and common context;
• Whether the model is an instruct/chat or specialized code model;
• Whether it can stably comply with file modification, testing and command execution requirements;
• Whether it supports the tool calls or JSON output you need;
• Is it easy to go off track, forget constraints, or produce incomplete modifications under long tasks?

The local 7B/8B model is suitable for warehouse browsing, simple scripts, document organization and local modifications. Multi-file refactoring, complex test repairs, and long-term Agent tasks require higher model and hardware requirements; don’t assume that just because the local API can be connected, it is suitable for high-risk automatic changes.

A safe way to get started

When using a local model to drive Codex for the first time, it is recommended to restrict permissions first:

1

codex --oss --local-provider ollama --sandbox read-only

Let the model complete the read-only task first:

1

分析当前仓库的目录结构，指出启动命令和测试命令。不要修改文件。

After confirming that the model understands the warehouse and the output is stable, gradually allow workspace writing and test execution. Do not enable sandbox-less or approval-skipping mode directly to save the confirmation step.

FAQ

1. codex exec --oss reports an error directly

Usually no local provider is specified. use:

1

codex exec --oss --local-provider ollama "只分析当前仓库，不修改文件"

Or set oss_provider in user-level configuration.

2. Codex cannot connect to Ollama or LM Studio

First verify the services separately:

1
2

ollama ls
curl http://localhost:1234/v1/models

Then check whether the service is started, whether the model is loaded, and whether the local port is affected by a firewall or other processes.

3. Local models always break code

Reduce the task first: make it read-only for analysis, change only one file, and give the plan first before executing it. And use Git branches or commit points to save revertable state. When the model capability is insufficient, increasing the complexity of prompt words usually cannot solve the fundamental problem.

4. The configuration is written but does not take effect.

Check whether the project .codex/config.toml was written by mistake. The provider-related key needs to be written in user-level ~/.codex/config.toml; restart Codex after modification.

Summarize

To allow local large models to be used by Codex, the order of priority should be:

1
2
3
4
5

Ollama / LM Studio 跑通模型
-> codex --oss --local-provider ollama|lmstudio
-> 只读任务验证
-> 设置 oss_provider 作为默认
-> 再考虑 vLLM 等 OpenAI 兼容网关

For most users, --oss is the shortest and most controllable entry. openai_base_url is suitable for advanced scenarios that already have compatible gateways and operation and maintenance requirements, but should be configured at the user level and interface compatibility verified first.

refer to:

• Codex OSS mode local providers
• Codex configuration reference
• Codex developer commands
• LM Studio OpenAI compatible interface tutorial
• vLLM KV Cache insufficient memory troubleshooting

LM Studio OpenAI-Compatible Local API in Detail

LM Studio can turn locally loaded models into OpenAI compatible interfaces. For existing projects, there is usually no need to rewrite the calling logic: change base_url of the OpenAI client to the local address of LM Studio, and then change model to the model identifier in LM Studio.

The most commonly used addresses are:

1

http://localhost:1234/v1

It is suitable for plugging into existing Python, JavaScript, C# or other OpenAI client code. The following instructions are in the order of “run through first, then access the project”.

Let’s talk about the conclusion first

To use LM Studio’s OpenAI-compatible interface, you only need to complete four steps:

1. Start the local server in LM Studio’s Developer page.
2. Load a chat model.
3. Request http://localhost:1234/v1/models, confirm model ID.
4. Change client base_url to http://localhost:1234/v1.

Minimal Python writing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
)

response = client.chat.completions.create(
    model="你的 LM Studio 模型 ID",
    messages=[
        {"role": "user", "content": "用一句话解释什么是 KV cache。"}
    ],
)

print(response.choices[0].message.content)

api_key="lm-studio" is just a placeholder value for OpenAI SDK when authentication is not enabled; if you enable API token in the LM Studio service settings, it should be changed to the real token.

Step 1: Start the LM Studio local server

Open LM Studio, enter the Developer page, and turn on the Start server switch. The default service will listen for:

1

http://localhost:1234

You can also use the LM Studio command line tool to start:

1

lms server start

If lms is not available on your computer, you can install the CLI according to the official LM Studio documentation:

1

npx lmstudio install-cli

Service startup only means that the API port is listening, but does not mean that there is a model that can be reasoned about. Continue to load a model in the Chat or Developer page, or load it with lms load.

Step 2: Get the model ID first

Don’t guess the model parameter based on the file name. The most stable way is to request a list of models:

1

curl http://localhost:1234/v1/models

Windows PowerShell can be used:

1

Invoke-RestMethod http://localhost:1234/v1/models

The returned data list will have the model identifier. Then fill in model with the actual returned ID in the request.

This step can avoid two common problems: the model has been downloaded but not loaded, or the name written in the code is inconsistent with the model ID currently exposed by LM Studio.

Step 3: Test Chat Completions with curl

First test with the most intuitive OpenAI compatible endpoint:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "你的 LM Studio 模型 ID",
    "messages": [
      {"role": "system", "content": "你是一个简洁的中文助手。"},
      {"role": "user", "content": "解释什么是向量数据库。"}
    ],
    "temperature": 0.7
  }'

When successful, the answer is usually:

1

choices[0].message.content

Chat Completions automatically apply the chat model’s prompt template. As long as the model itself is of chat/instruct type, there is usually no need to manually splice special control tokens on the client side.

How to replace OpenAI with Python project

If the project originally uses the OpenAI Python SDK, there are usually only two focuses: base_url and model.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
)

completion = client.chat.completions.create(
    model="你的 LM Studio 模型 ID",
    messages=[
        {"role": "system", "content": "你是一名 Python 助手。"},
        {"role": "user", "content": "写一个读取 JSON 文件的最小示例。"},
    ],
    temperature=0.2,
    max_tokens=500,
)

print(completion.choices[0].message.content)

The advantage of this is that the application layer still uses the objects and return formats of the OpenAI SDK, and the backend can switch between the cloud OpenAI API and the local LM Studio.

But “compatibility” does not mean that every cloud model feature can be copied as is. Availability of tool calls, structured output, visual input, inference content, and the Responses API is still dependent on the LM Studio version, current model capabilities, and corresponding endpoint support.

Streaming output

Set stream=True in Chat Completions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

stream = client.chat.completions.create(
    model="你的 LM Studio 模型 ID",
    messages=[{"role": "user", "content": "写一首四行小诗。"}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Streaming output is suitable for chat interfaces, terminal tools, and long answers. It improves the user waiting experience and does not make the local model itself generated faster.

How to choose between Embeddings, Responses and native REST API

LM Studio’s OpenAI compatibility layer includes common endpoints:

LM Studio also has its own native interface. The current recommended path is /api/v1/*, such as /api/v1/chat and /api/v1/models. The native API is more suitable for projects that require model loading/unloading, stateful chat, MCP or LM Studio-specific capabilities.

Simple judgment: If you already have an OpenAI SDK project, use /v1 first; if a new project requires in-depth management of local models or uses LM Studio’s exclusive capabilities, then consider /api/v1.

Structured output and tool calls

LM Studio’s OpenAI compatibility layer supports tool calls and structured output in the corresponding endpoints, but first confirm two things:

1. The model you load must have reliable tool calling or JSON output capabilities.
2. The versions of LM Studio and client SDK must be new enough.

Don’t assume that the model can stably generate results that conform to the schema just because there are no errors in the request. Testing should be done using real parameters, abnormal branches and multiple rounds of requests before going online.

Troubleshooting common errors

1. Connection refused or Connection refused

First confirm that the server in the Developer page has been started, and then test:

1

curl http://localhost:1234/v1/models

If you can’t connect here, first check the port, whether LM Studio is still running, or whether the local security software blocks the local port.

2. 404 Not Found

The most common reason is that the path is written incorrectly. OpenAI compatible chat endpoints are:

1

/v1/chat/completions

Not /api/v1/chat/completions. The latter belongs to another set of native API paths.

3. The model does not exist or an empty list is returned

First load the model in LM Studio and then check the return of /v1/models. model in the code must use the actual returned identifier, and do not copy other people’s model names.

4. Can answer but the format is strange

Check that the base model is loaded instead of the instruct/chat model; also check that the chat template is automatically applied by LM Studio. For tool calls and JSON output, also confirm that the model actually supports the capability.

5. The LAN device cannot be accessed

LM Studio can configure services to the local network on the Developer page. After enabling network access, you also need to confirm the firewall, listening address and API token settings. Do not expose unauthenticated local model services directly to the public network.

A minimal access checklist

1
2
3
4
5
6
7

1. LM Studio Developer -> Start server
2. 加载一个 instruct/chat 模型
3. curl http://localhost:1234/v1/models
4. 客户端 base_url 改为 http://localhost:1234/v1
5. model 填实际返回的 ID
6. 用 chat/completions 跑通单轮请求
7. 再测试流式、工具调用、Embeddings 或结构化输出

Run through the minimum request first, and then connect RAG, Agent or editor plug-in, troubleshooting will be much easier.

Summarize

LM Studio’s OpenAI compatible interface is suitable for connecting local models to existing OpenAI SDK projects. The most critical thing is not to copy a piece of code, but to confirm that the service is started, the model is loaded, base_url points to /v1, and model uses the actual model ID.

If what you need is deeper local model management, stateful chat, or MCP, LM Studio’s /api/v1/* native interface will be more suitable; if the goal is quick compatibility with old projects, continuing to use /v1/chat/completions is usually the easiest thing to do.

refer to:

• LM Studio OpenAI Compatibility Endpoints
• LM Studio Chat Completions
• LM Studio Local API Server
• LM Studio REST API