Architecture & Backend Compatibility

This is the single registry snapshot for recognized model architectures, candidate backends and measured combinations. Recognition is database-driven: registration reads config.json and resolves a candidate backend. It is not proof that every model in that architecture runs on every hardware target.

Check your specific model, not this list

Counts and per-architecture details are a snapshot. To know whether a specific HuggingFace repo will run — and on which backend — ask the live compatibility check, which reads the repo and returns an authoritative answer:

curl "https://api.colabhive.com/api/builder/v1/models/hf/{owner}/{repo}/info" \
  -H "X-API-Key: hive_..."

The compatibility object reports status (compatible · requires_review · incompatible), the runtime_profile_id (backend that will be used), and checks; estimated_requirements reports VRAM and a recommended config.

---

How a backend is chosen

ColabHive selects an inference backend from the model's architecture:

• vLLM (native). Architectures with first-class vLLM support get PagedAttention, continuous batching, and CUDA graphs — the fast path for production LLM serving.
• Transformers (broad fallback). A HuggingFace *ForCausalLM architecture that vLLM does not natively accelerate may resolve to the transformers backend. Custom code, weight format, model license, memory and backend/version compatibility can still reject it; fallback is a candidate path, not a 100% guarantee.
• Specialized backends. Diffusion/generative models, time-series specialists, SSM/hybrid models, and CPU GGUF (llama.cpp) each route to their own backend (see the backend table below).

You do not choose the backend manually — registration resolves it from the architecture and the model's runtime profile.

---

Supported architectures (snapshot 2026-08-15)

As of this snapshot the registry recognizes 393 active architecture/backend rows across the families below. “Recognized” means a registration rule exists; it does not mean that 393 distinct architectures were exercised successfully. This is a moving target — it more than quadrupled between the 2026-07-23 and 2026-08-15 snapshots; verify a specific repo with the compatibility check above.

Representative families (not exhaustive):

Family	Example architectures	Example models
Llama	LlamaForCausalLM, Llama4ForCausalLM, CodeLlamaForCausalLM	Llama 2/3/4, CodeLlama, Vicuna
Mistral	MistralForCausalLM, MixtralForCausalLM	Mistral 7B, Mixtral MoE
Qwen	Qwen2ForCausalLM, Qwen3ForCausalLM, Qwen2MoeForCausalLM, QWenLMHeadModel	Qwen 1/2.5/3, Qwen MoE
Gemma	GemmaForCausalLM, Gemma2ForCausalLM, Gemma3ForCausalLM	Gemma, Gemma 2, Gemma 3
Phi	PhiForCausalLM, Phi3ForCausalLM, PhiMoEForCausalLM	Phi-2, Phi-3/3.5, Phi MoE
DeepSeek	DeepseekForCausalLM, DeepseekV2ForCausalLM, DeepseekV3ForCausalLM	DeepSeek, DeepSeek Coder, v2/v3
GPT / classic	GPT2LMHeadModel, GPTNeoXForCausalLM, GPTJForCausalLM, BloomForCausalLM, OPTForCausalLM, FalconForCausalLM, MPTForCausalLM	GPT-2, Pythia, GPT-J, BLOOM, OPT, Falcon, MPT
GLM / ChatGLM	ChatGLMModel, Glm4ForCausalLM	ChatGLM, GLM-4
MoE (incl. MLA)	Glm4MoeForCausalLM, Qwen3MoeForCausalLM, GptOssForCausalLM, NemotronHForCausalLM	GLM-4.7 Flash, Qwen3-Coder 30B-A3B, gpt-oss, Nemotron-3 Super
Multimodal / vision-language	Qwen3VLForConditionalGeneration, Qwen3VLMoeForConditionalGeneration, Mistral3ForConditionalGeneration, Glm4vForConditionalGeneration, InternVLForConditionalGeneration, KimiVLForConditionalGeneration, NemotronH_Nano_VL_V2	Qwen3-VL, Ministral 3, GLM-4V, InternVL, Kimi-VL, Nemotron Nano VL
Other LLMs	BaichuanForCausalLM, InternLMForCausalLM, InternLM2ForCausalLM, YiForCausalLM, XverseForCausalLM, CohereForCausalLM, Cohere2ForCausalLM, DbrxForCausalLM, ArcticForCausalLM, NemotronForCausalLM, GraniteForCausalLM, OlmoForCausalLM, Olmo2ForCausalLM, StableLmForCausalLM, Starcoder2ForCausalLM, ExaoneForCausalLM, SolarForCausalLM, …	InternLM, Yi, Command, DBRX, Arctic, Nemotron, Granite, OLMo, StableLM, StarCoder 2, EXAONE, SOLAR
SSM / hybrid	MambaForCausalLM, Mamba2ForCausalLM, FalconMambaForCausalLM, JambaForCausalLM, ZambaForCausalLM	Mamba, Falcon-Mamba, Jamba, Zamba — fine-tuning templates only (see below)
Diffusion / generative	diffusers pipelines	SDXL, SD3, FLUX, MusicGen, AudioLDM2 (see Generative)
Time series	PatchTSTForPrediction, PatchTSTForClassification, BiTCN, TimesFM	PatchTST, BiTCN, TimesFM 2.5

SSM / hybrid (Mamba, Jamba, Zamba) are training templates, not inference endpoints

These architectures are present for fine-tuning (mamba-ssm-finetune, jamba-hybrid-finetune). They are not benchmarked and there are no public SSM inference endpoints. In the compatibility data their rows are untested (throughput unknown). Do not treat them as ready-to-call chat models.

---

Benchmarked throughput (tested)

Only a small set of architecture/backend pairs is marked tested (14 of them today), and only the ten below carry a real throughput number — all on the vllm-cu121 (NVIDIA) backend. All other rows in the compatibility table are untested with no throughput figure — treat those as "should work", not "measured".

Architecture	Backend	Throughput (tok/s)	Benchmark model
GPT-2 (GPT2LMHeadModel)	vllm-cu121	150	gpt2
Mixtral MoE (MixtralForCausalLM)	vllm-cu121	120	mistralai/Mixtral-8x7B-Instruct-v0.1
Phi-2 (PhiForCausalLM)	vllm-cu121	110	microsoft/phi-2
Phi-3 (Phi3ForCausalLM)	vllm-cu121	105	microsoft/Phi-3-mini-4k-instruct
Llama (LlamaForCausalLM)	vllm-cu121	100	meta-llama/Llama-2-7b-chat-hf
Mistral (MistralForCausalLM)	vllm-cu121	95	mistralai/Mistral-7B-Instruct-v0.3
Gemma (GemmaForCausalLM)	vllm-cu121	90	google/gemma-7b-it
Qwen 2 (Qwen2ForCausalLM)	vllm-cu121	90	Qwen/Qwen2.5-7B-Instruct
Qwen 1 (QWenLMHeadModel)	vllm-cu121	85	Qwen/Qwen-7B-Chat
GPT-NeoX (GPTNeoXForCausalLM)	vllm-cu121	80	EleutherAI/pythia-6.9b

Numbers are single-GPU order-of-magnitude figures for the benchmark model, not a guarantee for every model in the family. Transformers-fallback serving is typically several times slower than native vLLM.

The four remaining tested pairs are the GLM-4 MoE architectures (Glm4MoeForCausalLM, Glm4MoeLiteForCausalLM and their MTP variants) on vllm-xpu — validated on Intel Arc on 2026-08-03 during the GLM-4.7 Flash bring-up, recorded as tested but without a throughput figure.

---

Inference backends

ColabHive runs 19 active inference backends across NVIDIA, AMD, Intel, and CPU — not just vLLM. Thirteen of them appear in the current compatibility matrix. Images are referenced by repo; the platform resolves the running version from system_versions at dispatch (never :latest), with two legacy rows still carrying an explicit v1.0.0 pin.

backend_id	Vendor	Framework	Image (repo)
vllm-cu121	NVIDIA	vLLM	registry.colabhive.com/inference-vllm
transformers-cu121	NVIDIA	transformers	registry.colabhive.com/inference-base
tgi-cu121	NVIDIA	TGI	registry.colabhive.com/inference-tgi:v1.0.0
mamba-vllm-cu121	NVIDIA	vLLM (+mamba-ssm)	registry.colabhive.com/inference-vllm
diffusers-cu121	NVIDIA	diffusers	registry.colabhive.com/inference-generative
generative-cu121	NVIDIA	transformers	registry.colabhive.com/inference-generative
specialists-ts-cu121	NVIDIA	specialists	registry.colabhive.com/inference-specialists
specialists-clm-cu121	NVIDIA	specialists (CLM only)	registry.colabhive.com/clm-cu121
vllm-rocm	AMD	vLLM	registry.colabhive.com/inference-rocm
transformers-rocm	AMD	transformers	registry.colabhive.com/inference-rocm
specialists-rocm	AMD	specialists	registry.colabhive.com/inference-rocm
specialists-clm-rocm	AMD	specialists (CLM only)	registry.colabhive.com/clm-rocm
vllm-xpu	Intel	vLLM	registry.colabhive.com/inference-ipex
ipex-xe	Intel	transformers (IPEX)	registry.colabhive.com/inference-ipex
specialists-ipex-xpu	Intel	specialists	registry.colabhive.com/inference-ipex
specialists-clm-xpu	Intel	specialists (CLM only)	registry.colabhive.com/clm-xpu
transformers-cpu	CPU	transformers	registry.colabhive.com/inference-base
specialist-cpu-optimized	CPU	specialists	registry.colabhive.com/inference-base-cpu:v1.0.0
llm-llamacpp-cpu	CPU	llama.cpp	registry.colabhive.com/inference-llamacpp-cpu

Intel XPU (vllm-xpu / ipex-xe) and AMD ROCm backends are in production alongside NVIDIA — each vendor now has a matching specialists backend as well, so embeddings/rerank/OCR/STT/moderation run on whatever silicon the node has. The CPU GGUF backend (llm-llamacpp-cpu) powers the CPU LLM tier.

Superseded backend IDs

vllm-rocm60 and transformers-rocm60 were the earlier AMD entries pointing at a local inference-base-rocm image. They are deactivated — the live AMD backends are vllm-rocm, transformers-rocm and specialists-rocm on registry.colabhive.com/inference-rocm.

---

CLM-8B scorer (specialist:clm)

CLM-8B is served by three backends of its own, one per GPU vendor, and no other backend declares its runtime family. Each image is built on the universal image of its vendor and carries the CLM serving and training code, so a head is trained and served with the same encoder numerics. The encoder always runs in bf16.

Backend	Hardware	Inference	Head training	Measured GPU memory	Evidence date
specialists-clm-cu121	NVIDIA RTX 3090 24 GB	{{MEASURED:clm.status.inference.nvidia_rtx3090}}	{{MEASURED:clm.status.training.nvidia_rtx3090}}	{{MEASURED:clm.vram_mb.nvidia_rtx3090}} MB	{{MEASURED:clm.evidence_date.nvidia_rtx3090}}
specialists-clm-xpu	Intel Arc Pro B70 32 GB	{{MEASURED:clm.status.inference.intel_b70}}	{{MEASURED:clm.status.training.intel_b70}}	{{MEASURED:clm.vram_mb.intel_b70}} MB	{{MEASURED:clm.evidence_date.intel_b70}}
specialists-clm-xpu	Intel, two 16 GB GPUs (layers split across both)	{{MEASURED:clm.status.inference.intel_2x16gb}}	not offered	{{MEASURED:clm.vram_mb.intel_2x16gb}} MB	{{MEASURED:clm.evidence_date.intel_2x16gb}}
specialists-clm-rocm	AMD Instinct {{MEASURED:clm.amd.board}}	{{MEASURED:clm.status.inference.amd_instinct}}	{{MEASURED:clm.status.training.amd_instinct}}	{{MEASURED:clm.vram_mb.amd_instinct}} MB	{{MEASURED:clm.evidence_date.amd_instinct}}
—	CPU	{{MEASURED:clm.status.inference.cpu}}	{{MEASURED:clm.status.training.cpu}}	— (RAM {{MEASURED:clm.ram_mb.cpu}} MB)	{{MEASURED:clm.evidence_date.cpu}}

Measured latency per vendor is on the model page.

---

Requesting or adding architecture support

1. Try the compatibility check first — most models resolve to vLLM-native or the transformers fallback automatically.
2. If a model reports incompatible or you need native vLLM acceleration for a new architecture, open a GitHub issue with the repo ID, the architecture name from config.json, and your use case.

---

Related

• Model Catalog — curated base + HuggingFace import
• LLM Inference · CPU LLM (GGUF)
• Models API — registration, compatibility, lifecycle
• Choosing a Model

---

Authors: José Luis Minich, Maximiliano Lucius.