Best / Model serving
Best open-source Model serving
The best self-hosted model-serving split in the c17.io catalog is vLLM for an Apache-2.0 high-throughput OpenAI-compatible engine (the published Hugging Face Endpoints/Replicate stack), LocalAI for a MIT OpenAI-compatible box, Ollama for an MIT daemon on one machine, and llama.cpp when you want MIT C/C++ inference you compile. Hugging Face text-generation-inference is archived — not this path. A chat UI stays on /best/llm-chat-ui. Laptop runners stay on /best/local-llm.
Open the stackStart on Railway
Ranked picks
1
vLLMApache-2.0 high-throughput serving engine — the c17 Hugging Face Endpoints/Replicate path.
When to pick it: Throughput on a GPU you already run.
94,158 stars in the catalog
2
LocalAIMIT OpenAI-compatible engine you host — /compare/vllm-vs-localai.
When to pick it: A smaller OpenAI-shaped box if vLLM is more product than you wanted.
49,623 stars in the catalog
3
OllamaMIT local daemon and API already on /best/local-llm — /compare/ollama-vs-vllm.
When to pick it: One box next to Open WebUI.
183,730 stars in the catalog
4
llama.cppMIT LLM inference in C/C++. A runner you compile, not a chat UI.
When to pick it: Low-level local inference you wire yourself.
131,637 stars in the catalog
How they compare
| Tool | License | Self-host | Best for |
|---|---|---|---|
| vLLM | Apache License 2.0 | Medium | Throughput on a GPU you already run. |
| LocalAI | MIT License | Medium | A smaller OpenAI-shaped box if vLLM is more product than you wanted. |
| Ollama | MIT License | Easy | One box next to Open WebUI. |
| llama.cpp | MIT License | Harder | Low-level local inference you wire yourself. |
Related pages
- vLLM / LocalAI stack
- Open WebUI + LiteLLM stack
- Replace Hugging Face Inference Endpoints
- Replace Replicate
- Replace ChatGPT Team
- Replace OpenAI API
- Replace Claude
- Replace Google Gemini
- Replace Jasper
- Replace Copy.ai
- Replace Notion AI
- Replace Writesonic
- Replace Rytr
- vLLM vs LocalAI
- All best open-source pages
- All compare pages
- All replace pages
- All stacks
- Tools catalog
FAQ
- vLLM or LocalAI?
- vLLM if you want throughput on a GPU you already run. LocalAI if you want a MIT OpenAI-compatible box — /compare/vllm-vs-localai.
- Is this Ollama?
- Ollama is the one-box daemon — /best/local-llm and /compare/ollama-vs-vllm. This page is the serving engine.
- Where is the chat UI?
- /best/llm-chat-ui and /stacks/open-webui-litellm. Point Open WebUI at the server.
- Are these free?
- vLLM is Apache License 2.0. LocalAI, Ollama, and llama.cpp are MIT. You pay hardware. Weights have their own licenses. Template users never pay c17.
- How do I deploy it?
- Open /stacks/vllm-inference. Persist weights. Point one client at the OpenAI-compatible URL. Do not deploy archived TGI.
Stack of the Week
Get one stack a week that replaces a SaaS bill.