Tools / product

vLLM

Apache-2.0 high-throughput LLM inference and serving engine. An OpenAI-compatible server for weights you host, usually on a GPU.

URL

https://vllm.ai

GitHub

https://github.com/vllm-project/vllm

Collection

AI & Agents

Language

Python

License

Apache License 2.0

Stars

94,158

Open source

Yes

llminferenceservingopen-sourceself-hosted

Compared with

• Ollama vs vLLM