Directory · Local LLM
vLLM
High-throughput LLM inference engine — PagedAttention, OpenAI-compatible server, multi-GPU serving for production self-host.
Open Source
Best for
Production local inference, high QPS on open models, GPU cluster serving
Pros and cons
Pros
- Best inference throughput for self-hosted models
- OpenAI-compatible API drop-in
- Multi-GPU and quantization support
Cons
- Requires NVIDIA GPUs and ML ops skill
- Overkill for laptop experimentation
- Not a chat UI — pair with Open WebUI
Alternatives
More local llm tools
Ollama
Local LLM
Local LLM runtime. Pull models, run them on your laptop, expose an OpenAI-compatible API.
Best for
Privacy-sensitive tasks, offline use, embedded LLM workflows
Pros
- Free
- Local-first
- Wide model support
Cons
- GPU strongly recommended
- Slower than hosted APIs
- Quality < frontier models
Alternatives
LM StudiovLLM
Open WebUI
Local LLM
Self-hosted ChatGPT-style interface for local models — Ollama, OpenAI-compatible APIs, RAG, and multi-user workspaces.
Best for
Private team chat UI, local model access, air-gapped deployments
Pros
- Polished UI on top of Ollama
- RAG and document upload built in
- Active community and fast releases
Cons
- Another service to maintain
- Feature surface can overwhelm solo users
- Quality still bounded by underlying models
Alternatives
Ollama (CLI only)LM StudioChatGPT / Claude (hosted)