EP7 Claude with other AIs: local models and other providers, through one door

In EP5 we matched Claude models to jobs. In EP6 we got a common standard for connecting AI to other systems. This episode goes one step further: Claude working with other AIs, both models running on your own machine and other providers’ models.

Why more than one provider

• Some data must not leave the machine, such as customer data. A local model processes it without sending it anywhere
• High volume, like summarising thousands of texts. A local model charges nothing per token
• A second opinion. Models from other providers are trained differently and miss different things, so they can check each other
• Not betting everything on one provider. If one goes down or raises prices, the system keeps working

Principle: route every model call through one thin layer. Switching provider means changing a setting, not rewriting the system.

The two most common routes

1. Local models with Ollama. Install it, then pull a model with one command:

ollama pull qwen2.5:14b
ollama run qwen2.5:14b "Summarise this in three lines: ..."

Ollama serves an API at http://localhost:11434 in the same format as OpenAI’s, so anything that talks to OpenAI can talk to it.

2. Many providers through a gateway such as OpenRouter. One key reaches models from many providers; switch by changing the model name. Same API format again.

The 15-line layer

Since both routes share one API format, one function talks to both:

import json, os, urllib.request

PROVIDERS = {
    "local": ("http://localhost:11434/v1", None),
    "openrouter": ("https://openrouter.ai/api/v1", os.environ.get("OPENROUTER_API_KEY")),
}

def ask(provider, model, prompt):
    base, key = PROVIDERS[provider]
    body = json.dumps({"model": model, "messages": [{"role": "user", "content": prompt}]}).encode()
    req = urllib.request.Request(base + "/chat/completions", body, {"Content-Type": "application/json"})
    if key:
        req.add_header("Authorization", "Bearer " + key)
    with urllib.request.urlopen(req) as r:
        return json.load(r)["choices"][0]["message"]["content"]

Plain Python, nothing to install, and the key comes from an environment variable, not the code.

How Claude calls other AIs

Claude Code can be the “lead” in three ways, from simplest to most involved:

1. Through a script. Save the code above as ask.py and tell Claude: “use python ask.py to have the local model summarise all 500 log files, then analyse the summaries”
2. Through MCP. Wrap ask as a tool called ask_local_model in an MCP server (EP6), and Claude sees it as one more tool it can call on its own
3. The other way round. Another system is the lead and calls Claude only for the work that needs real capability, as in the example below

A real example: a team of 17, only one on Claude

We run a team of 17 agents on a machine at home, each with a role: planning, quality checks, content writing, data analysis and so on.

• 16 run on local models (Qwen at 9B and 14B, and DeepSeek-R1 at 7B) on a graphics card with 12GB of memory
• Only one uses Claude: the developer role, calling Claude Sonnet through OpenRouter, because writing code is where a small model’s mistakes cost the most

A real limit we hit: a 12GB card runs one 14B model at a time, or two small models together, so agents take turns. A home machine is not truly free; you pay in electricity, hardware and waiting time instead of tokens.

What to watch

• Data sent through a gateway has gone out to whoever runs the model at the other end. Before sending customer data or private code, decide whether it may leave. If not, use a local model
• Small models make more mistakes. Use them where results are easy to check, or have a larger model check them
• Each provider responds differently. A prompt that works well on Claude may not on another model; try real work first
• More parts, more places to break. Set timeouts, and have a fallback when one provider does not answer

Try it in 10 minutes

1. Install Ollama and run ollama pull qwen2.5:7b (small enough for an ordinary machine)
2. Save the layer as ask.py, add print(ask("local", "qwen2.5:7b", "Explain MCP in two sentences")) at the end, and run it
3. Open Claude Code and have it use ask.py to send a summarising job to the local model, then check the result itself

Where a person is needed

A person decides which data may leave the machine and which work is worth paying for a capable model. Those decisions depend on client contracts, budget and acceptable risk. AI can do the sums; it should not make the call.

Next episode

With hundreds of tools, skills, agents and models in play, a new problem appears: which one to use. EP8, the last episode, builds something that matches a job to the right tool, and measures whether it really picks correctly.