Local LLM

Tool Guide

Ollama Guide

Run LLMs locally with Ollama — install, model selection, hardware requirements, OpenAI-compatible API, and pairing with Open WebUI for a private ChatGPT experience.

Comparison

Local Models vs API Models

Should you run LLMs on your own hardware or pay OpenAI and Anthropic? An honest comparison on privacy, cost, latency, quality, and the ops burden nobody talks about upfront.

Directory

GPT4All

Nomic's open-source desktop app for running local LLMs on consumer hardware, with local file chat via LocalDocs.

Directory

Jan

Open-source ChatGPT-style desktop app that runs models locally via llama.cpp, with an offline-first design.

Directory

llama.cpp

C/C++ inference engine for running quantized LLMs in GGUF format efficiently on CPUs and GPUs across platforms.

Directory

LM Studio

Free desktop app for discovering, downloading, and running local LLMs with a chat UI and OpenAI-compatible local server.

Directory

Ollama

Local LLM runtime. Pull models, run them on your laptop, expose an OpenAI-compatible API.

Directory

Open WebUI

Self-hosted ChatGPT-style interface for local models — Ollama, OpenAI-compatible APIs, RAG, and multi-user workspaces.

Directory

vLLM

High-throughput LLM inference engine — PagedAttention, OpenAI-compatible server, multi-GPU serving for production self-host.