Architecture

How colabhive-mcp fits between your AI agent and the ColabHive platform.

ColabHive runs the MCP layer in two deployment shapes simultaneously:

Hosted — https://mcp.colabhive.com/mcp — multi-tenant, public.
Local stdio — uvx colabhive-mcp — single-tenant, private to one machine.

The transformation logic, the manifest, the protocol — all identical. Only the transport and tenancy model differ.

Hosted topology (`mcp.colabhive.com`)

┌────────────────────────────────────────────────────────────────────┐
│           AI Agent client (Claude Desktop / Cursor / n8n)          │
│              Authorization: Bearer hive_<their_key>             │
└──────────────────────────────┬─────────────────────────────────────┘
                               │ MCP over HTTP+SSE (POST + GET)
                               │
                ┌──────────────▼──────────────┐
                │      Cloudflare proxy       │
                └──────────────┬──────────────┘
                               │
                ┌──────────────▼──────────────┐
                │   nginx (443 TLS)           │
                │   SSE-friendly:             │
                │   • proxy_buffering off     │
                │   • read_timeout 3600s      │
                │   • X-Accel-Buffering off   │
                └──────────────┬──────────────┘
                               │ http://127.0.0.1:8765
                ┌──────────────▼──────────────┐
                │   colabhive-mcp.service     │
                │   (systemd, hardened)       │
                │   ┌────────────────────┐    │
                │   │  ServerPool (LRU)  │    │
                │   │  per SHA(api_key)  │    │
                │   │  max 64 tenants    │    │
                │   └────────┬───────────┘    │
                │   For each tenant:          │
                │   • httpx client            │
                │   • ManifestCache (TTL)     │
                │   • MCPHandler              │
                └──────────────┬──────────────┘
                               │ HTTPS, X-API-Key=tenant's
                               │
        ┌──────────────────────▼──────────────────────┐
        │  Builder Gateway (api.colabhive.com)        │
        │  /api/builder/v1/mcp/manifest               │
        │  /api/builder/v1/actions/{slug}:invoke      │
        │  /api/builder/v1/invocations/{id}           │
        └──────────────────────┬──────────────────────┘
                               │
        ┌──────────────────────▼──────────────────────┐
        │  Orchestrator → Node Runtime (unchanged)    │
        │  → InferenceServerManager / Specialists /   │
        │    DockerExecutor                           │
        └──────────────────────────────────────────────┘

Properties:

Stateless wrt user data. No prompts, no outputs persisted in mcp.colabhive.com. Caches are metadata (the manifest) only.
Multi-tenant by API key. A ServerPool keyed by SHA-256(api_key)[:16] keeps each tenant's manifest cache warm. LRU evicts to 64 resident tenants by default. No tenant ever sees another's tools.
No service-wide auth. The hosted box has no master key — every request must supply its own.
TLS terminates at nginx, Cloudflare in front. Backend talks plain HTTP to 127.0.0.1:8765.
SSE knobs explicit. Streaming-protocol-safe nginx config (buffering off, long read timeouts).

Local stdio topology

┌────────────────────────────────────────────────────────────────────┐
│                          Your AI Agent                             │
│                  (Claude Desktop / Cursor / n8n)                   │
└──────────────────────────────┬─────────────────────────────────────┘
                               │ JSON-RPC over stdin/stdout
                ┌──────────────▼──────────────┐
                │   colabhive-mcp (local)     │
                │   • single tenant           │
                │   • COLABHIVE_API_KEY env   │
                │   • ManifestCache (TTL)     │
                │   • Filter (allow/deny)     │
                │   • Sync/async glue         │
                └──────────────┬──────────────┘
                               │ HTTPS, X-API-Key
                ┌──────────────▼──────────────┐
                │   Builder Gateway (same)    │
                └─────────────────────────────┘

Properties:

One tenant per process. Started by the MCP client; lives only for that session.
Key in env. Comes from COLABHIVE_API_KEY (or --api-key). Never sent over a network except to api.colabhive.com.
Same protocol layer. The only differences are the transport (stdio vs HTTP+SSE) and the auth source (env vs request header).

Request lifecycle

Cold start (first request from a tenant in hosted mode)

Client → POST /mcp with header X-API-Key: hive_X
server_http: extracts the key
ServerPool.get(key) → cache miss → instantiate ColabHiveMCPServer
ColabHiveClient initialized (httpx connection pool)
ManifestCache GETs /mcp/manifest (~150 ms)
MCPHandler.dispatch(initialize) → 200 OK
   total: ~200-400 ms wall

`tools/list`

Client POST → handler → ManifestCache.get()
  cache fresh? return immediately.
  TTL expired? conditional GET with If-None-Match.
    304 → reuse cache.
    200 → replace cache.
Return filtered tools to client.

`tools/call`

Client POST tools/call {name, arguments}
Look up name in cached manifest (404 if absent).
Re-apply filters (allow/deny, side-effects, stability).
POST /actions/{slug}:invoke sync=true.
  succeeded → result.
  queued    → poll /invocations/{id} until terminal or poll_max_wait.
  failed    → translate to MCP error.
Result → result_to_content_blocks → MCP content[].

The sync/async glue is the main value-add of the wrapper: clients see one synchronous call; ColabHive handles warm/cold paths transparently.

Why a thin wrapper rather than "native" MCP in builder-gateway?

Reason	Detail
Transport mismatch	MCP runs over stdio or SSE. Builder-gateway is request/response HTTP, single port. Stdio in particular requires a process-launched-by-client model that doesn't fit a backend.
Local-first option	Users who want their key never to leave their machine need the stdio variant. We share the entire code path with the hosted server — config swaps the transport, nothing else.
Edge filtering	`--deny-side-effects`, `--allow-kinds`, manifest pinning happen per process / per tenant config. Putting these in builder-gateway would couple per-client policy with server policy.
Per-client tuning	Different MCP clients have different quirks. The wrapper is the place to special-case them.

The manifest layer

Manifests live in two places that must stay in sync:

DB (source of truth): columns mcp_* on inference_endpoints plus the JOIN with model_configs and training_jobs.
Cache (transient): in colabhive-mcp memory, refreshed every manifest_ttl seconds.

The MCP server never invents manifest data. Missing field in DB == missing in manifest. This honors REGLA #1: DB is the only source of truth.

kind is derived in _compute_kind() (in actions.py and inlined in mcp_manifest_lib.py for the routers) — never hardcoded strings. This honors REGLA #0: no engine names hardcoded.

Auth flow

Hosted (live)

Per request:
  header → API key extracted (Authorization: Bearer ... or X-API-Key)
  no key → 401 with structured JSON-RPC error
  key   → ServerPool[hash] → tenant server
  tenant server uses that key on outbound /api/builder/v1/* calls

No server-side login, no token store, no cookies.

Local stdio (live)

Process start:
  COLABHIVE_API_KEY env or --api-key arg → cfg.api_key
  ColabHiveClient uses that for every outbound call
  Process lives until the MCP client disconnects

Threat model snapshot

Threat	Mitigation
API key exfil from config file	Dedicated keys, filesystem permissions and immediate rotation
Replay of mutating tool call	Idempotency keys where documented and host confirmation for mutating tools
Cross-tenant data leak	Visibility rules inherited from Actions API; LRU pool isolation; SHA-keyed cache
Server crash kills client	systemd auto-restart; structured exit codes
Manifest exfil for unauthenticated discovery	`/mcp/manifest` requires auth; unauth POST returns 401

See Security for the full doc.

Performance notes

Manifest size

Order of magnitude: a few hundred endpoints × ~3 KB/manifest ≈ ~1 MB payload.

Mitigations:

ETag → 304 Not Modified on unchanged manifests (zero body)
Optional gzip — ~70% reduction
Client-side filtering keeps the manifest cached but presents a subset to the agent

Cold-path latency

When the agent calls a tool whose underlying model isn't loaded:

Client       → MCP server      (~5 ms hosted, ~0 stdio)
MCP server   → builder-gateway (~30 ms LAN)
              → orchestrator    (~10 ms)
              → node-runtime WS (~10 ms)
              → pull image / load weights (~30 s - 5 min — only on real cold)
              → inference       (200 ms - 10 s)

The MCP server polls /invocations/{id} until terminal. The agent sees one synchronous tool call.

latencyClass in each manifest tells the agent upfront: slow or batch means "expect long first call".

Observability

Each tenant in hosted mode emits structured JSON logs:

mcp.session.started — client connected, identity, protocol version
manifest.fetched — duration, byte count, tool count, ETag
mcp.tool.invoked — slug, sync, latency_ms, status, error
pool.tenant_added / pool.tenant_evicted — pool capacity events

In F4, these will also surface as Prometheus metrics: mcp_tool_calls_total, mcp_tool_latency_seconds, mcp_errors_total, mcp_tenants_resident.

Hosted topology (mcp.colabhive.com)​

Local stdio topology​

Request lifecycle​

Cold start (first request from a tenant in hosted mode)​

tools/list​

tools/call​

Why a thin wrapper rather than "native" MCP in builder-gateway?​

The manifest layer​

Auth flow​

Hosted (live)​

Local stdio (live)​

Threat model snapshot​

Performance notes​

Manifest size​

Cold-path latency​

Observability​

See also​