llama.cpp
LLM inference in C/C++
FreeTokenは、FlashML-orgが公開しているオープンソースのローカルLLMプロジェクトです(GitHub: FlashML-org/FreeToken)。GitHubスターは17,211、フォークは1,705で、Apache-2.0ライセンスの下で公開されています。GitHubでの最終更新は2026年11月15日です。
Run 290B+ frontier MoE models on your own gaming PC at interactive speed
Get the source and move into the project folder.
Install with uv, including the acceleration extras.
Choose a supported open-weight MoE model and follow the README serving example.
Run the engine and point your client at it.
290B+ open-weight MoE models on a gaming PC, per the project’s own claim.
CPU-GPU co-execution that adapts to the bandwidth your machine actually has.
Semantic anchor checkpoints for recurrent state and KV caches, so agentic context edits skip recomputation.
Dynamic VRAM re-allocation between expert caches and KV memory, without restarts or weight reloading.
DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other frontier open-weight MoE models.
The FTW fast weight format plus double-buffered prefill streaming for interactive speeds.
Download the source and move into the folder.
git clone https://github.com/FlashML-org/FreeToken.git && cd FreeTokenInstall in editable mode with the acceleration extras.
uv pip install -e ".[accel]"Pick a supported model and follow the README serving example.
# see README for the exact serve command and model flagsFor comparison, the same shelf also holds llama.cpp — LLM inference in C/C++ and colibri — Tiny C engine that runs huge MoE models on the. The complete list is the Local LLM shelf. Before running a model locally, check how much VRAM you need for local LLMs — model size has to match your hardware.
An edge-native MoE serving engine that runs frontier-scale open-weight models on personal and consumer hardware.
Frontier open-weight MoE models including DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2.
No — that is the point: it treats your GPU, CPU, host memory and interconnects as one elastic inference platform.
Yes, Apache-2.0 licensed. You still need the model weights, which you download yourself.
Clone the repo and install with uv, including the [accel] extras; the README has the serving examples.
LLM inference in C/C++
Tiny C engine that runs huge MoE models on the machine you already own — GLM, Kimi K3, DeepSeek, Qwen, Apache-2.0
GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required
The simplest way to run open language models on your own machine.
Run controllable pre-trained AI models locally, written in Rust.
Find the best local models for your hardware — and how to run them.
LibreAIカタログに掲載されたローカルLLMプロジェクトの中で最もスターの多いもの — FreeTokenが合わなかった場合に検討したい、現実的な候補リストです。
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
カタログの中でFreeTokenとスター数が近いローカルLLMプロジェクト — 前後の棚もそのまま見て回れます。
High-performance In-browser LLM Inference Engine
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
GPT-OSS — OpenAI open-weight language models: gpt-oss-120b for production reasoning and gpt-oss-20b for local use, Apache-2.0
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.