llama.cpp
LLM inference in C/C++
FreeToken ist ein Open-Source-Projekt aus der Kategorie Lokale LLMs von FlashML-org (FlashML-org/FreeToken auf GitHub). Es hat 17.211 Sterne auf GitHub und 1.705 Forks, wird unter der Lizenz Apache-2.0 veröffentlicht und wurde zuletzt am 15. November 2026 auf GitHub aktualisiert.
Run 290B+ frontier MoE models on your own gaming PC at interactive speed
Get the source and move into the project folder.
Install with uv, including the acceleration extras.
Choose a supported open-weight MoE model and follow the README serving example.
Run the engine and point your client at it.
290B+ open-weight MoE models on a gaming PC, per the project’s own claim.
CPU-GPU co-execution that adapts to the bandwidth your machine actually has.
Semantic anchor checkpoints for recurrent state and KV caches, so agentic context edits skip recomputation.
Dynamic VRAM re-allocation between expert caches and KV memory, without restarts or weight reloading.
DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other frontier open-weight MoE models.
The FTW fast weight format plus double-buffered prefill streaming for interactive speeds.
Download the source and move into the folder.
git clone https://github.com/FlashML-org/FreeToken.git && cd FreeTokenInstall in editable mode with the acceleration extras.
uv pip install -e ".[accel]"Pick a supported model and follow the README serving example.
# see README for the exact serve command and model flagsFor comparison, the same shelf also holds llama.cpp — LLM inference in C/C++ and colibri — Tiny C engine that runs huge MoE models on the. The complete list is the Local LLM shelf. Before running a model locally, check how much VRAM you need for local LLMs — model size has to match your hardware.
An edge-native MoE serving engine that runs frontier-scale open-weight models on personal and consumer hardware.
Frontier open-weight MoE models including DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2.
No — that is the point: it treats your GPU, CPU, host memory and interconnects as one elastic inference platform.
Yes, Apache-2.0 licensed. You still need the model weights, which you download yourself.
Clone the repo and install with uv, including the [accel] extras; the README has the serving examples.
LLM inference in C/C++
Tiny C engine that runs huge MoE models on the machine you already own — GLM, Kimi K3, DeepSeek, Qwen, Apache-2.0
GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required
The simplest way to run open language models on your own machine.
Run controllable pre-trained AI models locally, written in Rust.
Find the best local models for your hardware — and how to run them.
Die anderen Projekte der Kategorie Lokale LLMs mit den meisten Sternen im LibreAI-Katalog — die realistische Auswahl, falls FreeToken nicht das Richtige ist.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Verwandte Projekte der Kategorie Lokale LLMs im Katalog, die FreeToken bei den Sternen am nächsten stehen — zum Weiterstöbern auf beiden Seiten.
High-performance In-browser LLM Inference Engine
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
GPT-OSS — OpenAI open-weight language models: gpt-oss-120b for production reasoning and gpt-oss-20b for local use, Apache-2.0
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Kostenlos und offen — hol dir den Quellcode, gib dem Repository einen Stern und leg los.