vLLM if you want throughput on a GPU you already run. LocalAI if you want a MIT OpenAI-compatible box. Ollama if the job is one daemon — /compare/vllm-vs-localai and /compare/ollama-vs-vllm.

Hugging Face or Replicate?

Same published stack. You run the server. The hosted catalog stays behind.

Do I still pay a GPU?

Usually. The software is the server. Weights and machines are yours.

Stacks / AI chat

Medium

vLLM / LocalAI

Model serving you run. vLLM is Apache-2.0; LocalAI is MIT — Hugging Face Inference Endpoints and Replicate without the hosted inference meter.

The problem

Hugging Face Inference Endpoints rent a GPU endpoint. Replicate meters model runs. You wanted an OpenAI-compatible server on hardware you control. vLLM is Apache-2.0 (vllm-project/vllm). LocalAI is MIT (already in the catalog). Ollama is the MIT daemon on /best/local-llm. Hugging Face text-generation-inference is archived — not this path. This is not Hugging Face's every hardware SKU, Replicate's hosted model catalog, or free tokens. You still bring weights and usually a GPU. A chat UI stays on /replace/chatgpt.

Replaces

Hugging Face Inference Endpoints / Replicate

Replace Hugging Face Inference Endpoints Replace Replicate See all best open-source model servers

Estimated SaaS saved

$1,200

per year, small team, before lock-in

Start on Railway

Open WebUI if this is the chat window, not the serving engine

Sponsor this stack Tip jar

A GPU host is the honest path for vLLM. LocalAI and Ollama can run smaller models on CPU. No one-click inference cluster in this repo — Start on Railway. Chat UIs stay on /stacks/open-webui-litellm.

The stack

vLLM
Apache-2.0 high-throughput OpenAI-compatible serving engine
Catalog entry →
LocalAI
MIT OpenAI-compatible engine you host
Catalog entry →
Ollama
MIT local daemon and API if the job is one box
Catalog entry →

Cost table

You stop paying	Typical / year	Self-host instead
Hugging Face Endpoints / Replicate (small team)	$1,200	vLLM or LocalAI + weights you run

Agent setup prompt

Replace Hugging Face Inference Endpoints or Replicate with vLLM (throughput) or LocalAI (smaller box).

- Official docs. Persist weights. Point one client at the OpenAI-compatible URL. Do not claim the hosted model catalog.
- Hugging Face TGI is archived — do not deploy it as the path. Chat UIs stay on /stacks/open-webui-litellm.
- Write AGENT_PROMPT.md.

Also as markdown: /stacks/vllm-inference.md

FAQ

vLLM or LocalAI?: vLLM if you want throughput on a GPU you already run. LocalAI if you want a MIT OpenAI-compatible box. Ollama if the job is one daemon — /compare/vllm-vs-localai and /compare/ollama-vs-vllm.
Hugging Face or Replicate?: Same published stack. You run the server. The hosted catalog stays behind.
Do I still pay a GPU?: Usually. The software is the server. Weights and machines are yours.