llama.cpp
LLM inference in C/C++
FreeToken est un projet de LLM locaux open source, créé par FlashML-org (FlashML-org/FreeToken sur GitHub). Il compte 17 211 étoiles sur GitHub et 1 705 forks, il est publié sous la licence Apache-2.0 et a été mis à jour pour la dernière fois sur GitHub le 15 novembre 2026.
Run 290B+ frontier MoE models on your own gaming PC at interactive speed
Get the source and move into the project folder.
Install with uv, including the acceleration extras.
Choose a supported open-weight MoE model and follow the README serving example.
Run the engine and point your client at it.
290B+ open-weight MoE models on a gaming PC, per the project’s own claim.
CPU-GPU co-execution that adapts to the bandwidth your machine actually has.
Semantic anchor checkpoints for recurrent state and KV caches, so agentic context edits skip recomputation.
Dynamic VRAM re-allocation between expert caches and KV memory, without restarts or weight reloading.
DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other frontier open-weight MoE models.
The FTW fast weight format plus double-buffered prefill streaming for interactive speeds.
Download the source and move into the folder.
git clone https://github.com/FlashML-org/FreeToken.git && cd FreeTokenInstall in editable mode with the acceleration extras.
uv pip install -e ".[accel]"Pick a supported model and follow the README serving example.
# see README for the exact serve command and model flagsFor comparison, the same shelf also holds llama.cpp — LLM inference in C/C++ and colibri — Tiny C engine that runs huge MoE models on the. The complete list is the Local LLM shelf. Before running a model locally, check how much VRAM you need for local LLMs — model size has to match your hardware.
An edge-native MoE serving engine that runs frontier-scale open-weight models on personal and consumer hardware.
Frontier open-weight MoE models including DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2.
No — that is the point: it treats your GPU, CPU, host memory and interconnects as one elastic inference platform.
Yes, Apache-2.0 licensed. You still need the model weights, which you download yourself.
Clone the repo and install with uv, including the [accel] extras; the README has the serving examples.
LLM inference in C/C++
Tiny C engine that runs huge MoE models on the machine you already own — GLM, Kimi K3, DeepSeek, Qwen, Apache-2.0
GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required
The simplest way to run open language models on your own machine.
Run controllable pre-trained AI models locally, written in Rust.
Find the best local models for your hardware — and how to run them.
Les autres projets de la catégorie LLM locaux avec le plus d'étoiles du catalogue LibreAI — la sélection réaliste d'alternatives si FreeToken ne vous convient pas.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projets de la catégorie LLM locaux proches dans le catalogue, les plus proches en étoiles de FreeToken — continuez à explorer de part et d'autre.
High-performance In-browser LLM Inference Engine
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
GPT-OSS — OpenAI open-weight language models: gpt-oss-120b for production reasoning and gpt-oss-20b for local use, Apache-2.0
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Gratuit et ouvert : récupérez le code source, mettez une étoile au dépôt et commencez à créer.