vLLM is the high-throughput inference backbone of self-hosted LLMs in 2026 — ~81,544 stars (Yuzec), 36K in earlier curated list, PagedAttention gives 2-4x throughput over HuggingFace TGI. From Junagadh I serve Qwen 14B Q4 at 44 tok/s on M3 Max and 3B at 62 tok/s on Pi 5 — versus cloud frontier $8-15 per 1M, we serve at ₹65/hr on IndiaAI Mission 38K GPUs.
I curate at /repos and serve via AI Development. Contact for throughput benchmark on your traffic.
Why vLLM in 2026
NocoBase + Yuzec: Inference & Serving lane = vLLM 81K, Ollama 172K, llama.cpp 114K. Every self-hosted stack needs one. PagedAttention (virtual memory for KV cache) is why vLLM wins throughput. Per Bitdoze, treat stars as momentum — vLLM holds momentum since 2023 because it is the boring reliable layer.
| Engine | Stars | Throughput | Setup |
|---|---|---|---|
| vLLM | 81.5K | 2-4x TGI | Python, CUDA, PagedAttention |
| Ollama | 172K | 1x (ease) | 1-command, Apple Silicon |
| HuggingFace TGI | — | 1x baseline | Good but less throughput |
I run vLLM for server (A100/H100/ L4 on IndiaAI 65/hr) and Ollama for edge (Pi/Mac). Hybrid router picks cheapest that hits latency SLA (P95 800ms, paged via Grafana Tempo alerts).
from vllm import LLM
llm = LLM(model="Qwen2.5-14B-Instruct-AWQ", gpu_memory_utilization=0.9)
outputs = llm.generate(["Explain DPDP in Gujarati"], sampling_params)
Bottom Line: vLLM + PagedAttention in 2026 is the boring reliable 2-4x layer — if you self-serve LLMs at scale on IndiaAI 65/hr GPUs, this is the engine.
Frequently Asked Questions
vLLM vs Ollama — which to run?
Ollama for dev/edge ease, vLLM for server throughput. I run both — Ollama Pi, vLLM cloud.
Does it run on IndiaAI Mission GPUs?
Yes — Gaudi 2, MI300X, H100, L4 all in IndiaAI 38K pool, 65/hr subsidized. That is how Gujarat SMEs afford 14B.
What is PagedAttention?
OS-like paging for KV cache — fragments eliminated, memory 4x better utilized → higher batch → 2-4x throughput.
Who configures vLLM from Junagadh?
Deepak Bagada — vLLM + OTel + 90-day JSONL. See Library.