Open and self-hosted models

This guide is for developers running open weight models, either on a vendor API or on their own hardware. Cycls treats both the same way: a vendor/model string, a base URL, and a key.

Vendor APIs for open weight models

import os

import cycls

llm = (
    cycls.LLM()
    .model("deepseek/deepseek-flash")
    .base_url("https://api.deepseek.com/v1")
    .api_key(os.environ["DEEPSEEK_API_KEY"])
    .system("You are a helpful assistant.")
    .thinking("medium")
    .context(128_000)
    .max_tokens(8_192)
)

llm = (
    cycls.LLM()
    .model("moonshotai/kimi-k3")
    .base_url("https://api.moonshot.ai/v1")
    .api_key(os.environ["MOONSHOT_API_KEY"])
    .thinking("high")
    .context(1_000_000)
)

llm = (
    cycls.LLM()
    .model("zai/glm-5.3")
    .base_url("https://open.bigmodel.cn/api/paas/v4")
    .api_key(os.environ["ZAI_API_KEY"])
    .thinking("medium")
)

The vendor prefix picks the reasoning dialect. DeepSeek receives a thinking toggle plus an effort level, Kimi receives effort tiers, GLM receives a binary toggle. See Models for the full mapping.

Model identifiers change between releases. Confirm the current name in the vendor’s documentation before pinning it.

Your own vLLM or SGLang server

vllm serve moonshotai/Kimi-K3 --port 8000 --served-model-name kimi-k3

llm = (
    cycls.LLM()
    .model("local/kimi-k3")
    .base_url("http://localhost:8000/v1")
    .api_key("unused")
    .context(256_000)
    .max_tokens(16_384)
)

local has no reasoning dialect, so .thinking() sends nothing and prints one warning. Pass the parameter your server expects directly:

llm = cycls.LLM().model("local/kimi-k3").base_url(...).extra_body({"reasoning_effort": "high"})

http://localhost:8000 resolves inside the container, not on your laptop. When cycls run builds a container, point it at a reachable host such as http://host.docker.internal:8000/v1 on macOS, or at the server’s network address.

Endpoints behind proxy authentication

Some hosts authenticate with headers rather than a bearer token.

llm = (
    cycls.LLM()
    .model("modal/moonshotai/Kimi-K3")
    .base_url("https://your-workspace--kimi-k3.modal.direct/v1")
    .api_key("unused")
    .headers({"Modal-Key": os.environ["MODAL_TOKEN_ID"],
              "Modal-Secret": os.environ["MODAL_TOKEN_SECRET"]})
)

.headers() is sent on every model request. Clients are cached per configuration, so changing headers creates a new client rather than mutating a shared one.

Text-only models

Models without vision reject base64 media and fail the whole request. Turn vision off and attachments stay in the workspace, with a note naming the file so the model can open it with a tool.

llm = cycls.LLM().model("zai/glm-5.3").base_url(...).vision(False)

Set the context window

Compaction starts when the conversation approaches the window, and the default assumes 1M tokens. Set it to the real value or long chats will overflow before compaction triggers.

llm = cycls.LLM().model("deepseek/deepseek-flash").context(128_000)

Web search on any model

Native server-side search is Anthropic only. The portable pair works everywhere and needs BRAVE_API_KEY in the container.

llm = (
    cycls.LLM()
    .model("moonshotai/kimi-k3")
    .base_url("https://api.moonshot.ai/v1")
    .allowed_tools(["WebSearch"])
    .web_search("brave")
)

MCP on any model

The harness speaks MCP itself, so remote MCP servers work regardless of provider. .server_side() is the exception: it hands the server to Anthropic’s connector and is Anthropic only.

llm = cycls.LLM().model("deepseek/deepseek-flash").mcp(
    cycls.MCP("https://mcp.example.com/mcp").name("example").token(cycls.env("MCP_TOKEN"))
)

Complete example

oss_agent.py

import os

import cycls

image = cycls.Image().copy(".providers.env", ".env")

llm = (
    cycls.LLM()
    .model("deepseek/deepseek-flash")
    .base_url("https://api.deepseek.com/v1")
    .api_key(os.environ["DEEPSEEK_API_KEY"])
    .system("You are a helpful assistant. Be concise.")
    .allowed_tools(["Bash", "Editor", "WebSearch", "Canvas"])
    .web_search("brave")
    .thinking("medium")
    .context(128_000)
    .max_tokens(8_192)
    .price(input=0.27, output=1.10)
)


@cycls.agent(
    image=image,
    web=cycls.Web().auth(cycls.Clerk()).title("Open agent"),
    volumes={"/workspace": cycls.Volume("open-agent")},
)
async def open_agent(context):
    async for ev in llm.run(context=context):
        yield ev

Set prices from the vendor’s published rates so cycls cost reports real spend.

Troubleshooting

The provider rejects the request with an unknown parameter

A reasoning parameter may not exist on that endpoint. Use .thinking(None) and add what the server supports with .extra_body().

Tool calls never happen

Confirm the endpoint implements OpenAI tool calling. Some community servers accept the field and ignore it. Test with a single simple tool first.

Long chats fail with a context error

Set .context() to the model’s real window. The loop compacts based on that number, and the default is 1M.

Images are rejected

Pass .vision(False) for text-only models.

Python reference

Every exported name in one table.

Get started

Functions

Apps

Agents

Interface and users

Ship it

Guides

Reference

Resources

For LLMs

Open and self-hosted models

Vendor APIs for open weight models

Your own vLLM or SGLang server

Endpoints behind proxy authentication

Text-only models

Set the context window

Web search on any model

MCP on any model

Complete example

Troubleshooting

Next

Python reference

​Vendor APIs for open weight models

​Your own vLLM or SGLang server

​Endpoints behind proxy authentication

​Text-only models

​Set the context window

​Web search on any model

​MCP on any model

​Complete example

​Troubleshooting

​Next

Python reference

Vendor APIs for open weight models

Your own vLLM or SGLang server

Endpoints behind proxy authentication

Text-only models

Set the context window

Web search on any model

MCP on any model

Complete example

Troubleshooting

Next