Use local models with StremAI
Keep inference on your machine with Ollama, LM Studio, llama.cpp, Open WebUI, or another OpenAI-compatible server. StremAI adds recall before the model runs and stores useful outcomes afterward.
Local brain: agentbay local setup --local keeps memories in SQLite on this machine. No StremAI account is needed.
Shared brain: agentbay local setup --shared opens browser authorization, then StremAI() uses your authorized workspace automatically. The model remains local while connected agents can share context.
Running a local model does not automatically make memory local-only. Pick the memory mode deliberately.
With --local: memory content stays in ~/.agentbay/local.db, setup does not open a StremAI browser sign-in, and the verification marker is removed after the store/recall check.
Model traffic: prompts go to the provider endpoint you selected. The built-in Ollama, LM Studio, and llama.cpp defaults use loopback; a custom OpenAI-compatible URL is your responsibility.
Strict no-network runtime: set AGENTBAY_TELEMETRY=0 and use local memory. This disables the SDK's bounded anonymous telemetry as well as cloud memory. No credential is required.
Saved configuration: provider, model, memory mode, and optional base URL only. The local-model profile does not copy API keys or OAuth credentials.
Quick start with Ollama
ollama pull qwen3:8b pip install "stremai[local-models]" agentbay local setup
The guided setup detects your running model, asks whether memory should stay on this machine or be shared through StremAI, and verifies a memory store-and-recall before it finishes. Shared memory opens the same browser authorization used by connected agents; local-only mode never opens a browser.
from stremai import StremAI
brain = StremAI() # Uses the provider and memory mode selected during setup
response = brain.chat(
[{"role": "user", "content": "What decisions have we made about authentication?"}],
)
print(response.choices[0].message.content)For noninteractive local-memory setup, run agentbay local setup --local. Add AGENTBAY_TELEMETRY=0 for a strict no-StremAI-network runtime. Use --shared in scripts to open browser authorization without an interactive choice.
Disconnect or remove it safely
Removing an MCP entry or uninstalling the SDK never deletes your local brain. Export first if you want a portable copy, then remove the client configuration.
Delete ~/.agentbay/local.db only when you explicitly want permanent local erasure. For shared mode, revoke the hosted grant or key in StremAI after removing the client entry.
See the hosted MCP connection guide for reconnect and revocation details.
Load a model in LM Studio and start its local server, then check it:
agentbay local setup --provider lmstudio
The setup saves the model ID it finds, so StremAI().chat(...) uses it automatically.
Start the OpenAI-compatible server exposed by llama.cpp:
agentbay local setup --provider llamacpp
The setup saves the loaded model name and verifies its memory connection.
Any OpenAI-compatible local server
This covers local servers such as vLLM, LocalAI, Jan, or a custom OpenAI-compatible endpoint.
agentbay local setup --provider openai-compatible \ --endpoint http://127.0.0.1:8000/v1/models
response = brain.chat(
[{"role": "user", "content": "Summarize the current project decisions."}],
provider="openai-compatible",
base_url="http://127.0.0.1:8000/v1",
model="your-model-name",
)Open WebUI
For an Open WebUI agent, add StremAI's Streamable HTTP MCP server in Open WebUI's MCP configuration:
https://stremai.com/api/mcp
Complete the browser authorization when prompted. That path gives the local-model agent tools for shared StremAI memory; grant only the project access it needs. For a custom Python agent that calls Open WebUI's local API, use the OpenAI-compatible example above.