Local AI MCP
One MCP server for every local model runtime. It knows what fits your GPU, and it never sends a prompt to the cloud unless you tell it to.
What it is
An operations-first control plane for the models on your own machine. Your agent can see what is installed and loaded across runtimes, check what will actually fit in your VRAM, get models ready, and hand work to the best local model with one delegate call. It runs over stdio only and never opens a network listener.
Local first, by construction
Each provider is local or hosted. Calls without a provider argument only use live local providers that have the model. Hosted APIs such as Moonshot are used only when named explicitly, or when you set LOCAL_AI_ALLOW_HOSTED_FALLBACK=1. Every result reports provider, host and local.
Hardware-aware
fit_check sizes the KV cache from the model's real layer architecture, sums VRAM across GPUs, models partial GPU/CPU offload and quantized KV caches, and understands Apple Silicon unified memory. Hardware probes cover NVIDIA, AMD (Linux sysfs, Windows registry), Intel, and Apple Silicon.
Supported runtimes
| Runtime | Enabled by | Notes |
|---|---|---|
| Ollama | default localhost:11434 | Full lifecycle; pull progress; contextLength sets num_ctx. |
| LM Studio | default localhost:1234 | Lifecycle via the lms CLI (TTL, context length). |
| llama.cpp | default localhost:8080 | /health, /props, /slots, embeddings. |
| Docker Model Runner | default localhost:12434 | Pull with progress, remove, OpenAI routes under /engines/v1. |
| MLX | MLX_HOST | mlx_lm.server on Apple Silicon. |
| vLLM | VLLM_HOST | KV-cache usage and queue depth from /metrics. |
| Foundry Local | FOUNDRY_LOCAL_HOST | Download, load and unload via its REST API. |
| OpenAI-compatible | OPENAI_COMPAT_HOST | Jan, LocalAI, and other /v1 servers. |
| Moonshot AI (Kimi) | MOONSHOT_API_KEY | Hosted: only used when named explicitly. |
Tools (19)
delegate
One call: pick, ready, and run the best local model for a task.
complete
Completion with JSON-schema output, images, context length, streaming.
embed
Embeddings, one vector per input.
ensure_model
Pull, fit-check, evict-to-fit, load, verify.
fit_check
gpu / partial / cpu / none, with the numbers.
suggest_model
Best models for a task on this machine.
search_models
Hugging Face GGUF search with per-quant fit.
search_available
Offline curated catalog.
list_providers
Runtimes, kind, live status, capabilities.
list_models
Installed models per runtime.
list_loaded
Resident models and VRAM use.
model_info
Metadata and layer architecture.
pull_model heavy
Download with progress.
remove_model destructive
Asks you to confirm via elicitation.
load_model
Load with keep-alive and context length.
unload_model
Free VRAM/RAM.
benchmark heavy
TTFT and tokens/sec; compare models.
health_check
Reachability, version, runtime metrics.
system_resources
RAM, CPU, GPUs and VRAM.
Install
Claude Code
claude mcp add local-ai -- npx -y @tmhs/local-ai-mcpClaude Desktop: open the .mcpb bundle from the latest release. Any other client:
{ "mcpServers": { "local-ai": { "command": "npx", "args": ["-y", "@tmhs/local-ai-mcp"] } } }Configuration is through environment variables; see the README and .env.example.