LibreAI ENESPTDEFRRUJA
← All categories// local-llm

Best Open-Source Local LLM Tools

Run large language models on your own machine.

128 tools · 💎 77 hidden gems

// start here

Start here

The most-starred projects in this category, plus a rotating pick of three more from deeper in the list — refreshed so the whole catalogue gets its turn in the window.

Local LLM

Ollama

The simplest way to run open language models on your own machine.

★ 182.1k starsMIT
View guide →
Local LLM

Open Webui

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

★ 153.9k starsCustom (check repo)
View guide →
Local LLM

llama.cpp

LLM inference in C/C++

★ 130.2k starsMIT
View guide →
Local LLM

node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

★ 2.2k starsMIT
View guide →
Local LLM

Orkas

Local-first desktop where a commander LLM directs specialist coding agents.

★ 2.2k starsMIT
View guide →
Local LLM

agent-safehouse

Sandbox for local AI agents so they only touch what they need.

★ 2.1k starsApache-2.0
View guide →

More categories: VideoImageAudioAI MusicAI 3DClippingAI ChatAI AgentsRAG & KnowledgeAutomationProductivityDev ToolsDesignWritingSelf-HostedEasy Install

// about this category

About Best Open-Source Local LLM Tools

Local LLM tools let you run large language models entirely on your own hardware: runtimes that serve the models, desktop apps that wrap them in chat interfaces, and launchers that manage downloads, quantisation, and settings for you.

Running locally flips the usual trade-offs: no per-token bills, no prompts leaving your machine, and models that keep working offline — in exchange for answers bounded by your hardware. Quantised models are the key trick, shrinking big models to fit consumer GPUs and even CPUs.

How to choose

  • Start from your hardware: the guides assume you match model size to your VRAM or RAM, not the other way round.
  • If you want to build on top, pick a runtime with a local API; if you just want to chat, pick a desktop app.
  • Quantisation level is a quality dial — mid-level quants are the usual sweet spot for home machines.
// faq

Questions, answered

Which open-source Local LLM tools are listed here?

This page collects 128 open-source Local LLM projects curated by LibreAI — Run large language models on your own machine. Each one is verified on GitHub and has its own guide page with an overview, install steps, use cases, and an FAQ, so you can judge fit, requirements, and licensing before you download anything.

Are these Local LLM tools free to use?

Yes. Every tool here is open source and free to download and use under its own license — most use MIT, Apache-2.0, or GPL-3.0, and any custom source-available license is flagged on the tool's page. Check the LICENSE file before commercial use.

Does LibreAI host these tools?

No. LibreAI publishes guides only — no downloads are served from this site, and no tool here is mirrored or repackaged by us. Every install comes from the project's official GitHub repository, which each guide links to directly, so you always get the genuine, current release from the people who build and maintain the tool.

How do I install a Local LLM tool from this page?

Open the tool's guide and follow its 'Run it yourself' section — the steps come from the project's own documentation — or use the 'View on GitHub' button to go straight to the official repository. Each guide also states the tool's requirements first, so you can check your machine qualifies before downloading anything.

How were these tools chosen?

Every project is verified live through the GitHub API — stars, forks, license, and recent activity — and listed only if it has real momentum and an active community. Projects are re-checked as the catalogue is refreshed, and any project that goes quiet or changes its licensing terms has that noted on its page.

How big a model can my computer run?

As a rough shape: quantised models need slightly more memory than their size on disk. An 8B model at a mid-level quant fits comfortably in 8GB of VRAM or shared RAM; bigger models want more. Each guide follows the project's own stated requirements.

Are local models as good as the big hosted ones?

The largest open models are strong at chat, summarising, and coding help, and they run privately and offline. Hosted frontier models still lead on the hardest reasoning — the local trade is privacy, cost, and control for a capability step down.