Directory · Local LLM
vLLM
High-throughput LLM inference engine — PagedAttention, OpenAI-compatible server, multi-GPU serving for production self-host.
Best for
Production local inference, high QPS on open models, GPU cluster serving
Pros and cons
Pros
- Best inference throughput for self-hosted models
- OpenAI-compatible API drop-in
- Multi-GPU and quantization support
Cons
- Requires NVIDIA GPUs and ML ops skill
- Overkill for laptop experimentation
- Not a chat UI — pair with Open WebUI
Alternatives
More local llm tools
GPT4All
Local LLM · Nomic AI · United States
Nomic's open-source desktop app for running local LLMs on consumer hardware, with local file chat via LocalDocs.
Best for
Chatting with local models and documents on everyday laptops
Jan
Local LLM · Menlo Research
Open-source ChatGPT-style desktop app that runs models locally via llama.cpp, with an offline-first design.
Best for
A private, open-source local chat app with optional cloud model connections
llama.cpp
Local LLM · ggml.ai
C/C++ inference engine for running quantized LLMs in GGUF format efficiently on CPUs and GPUs across platforms.
Best for
Maximum-efficiency local inference and embedding into other software