llama.cpp
LLM inference in C/C++
FreeToken é um projeto de LLM local de código aberto de FlashML-org (FlashML-org/FreeToken no GitHub). Tem 17.211 estrelas no GitHub e 1.705 forks, é publicado sob a licença Apache-2.0 e foi atualizado pela última vez no GitHub em 15 de novembro de 2026.
Run 290B+ frontier MoE models on your own gaming PC at interactive speed
Get the source and move into the project folder.
Install with uv, including the acceleration extras.
Choose a supported open-weight MoE model and follow the README serving example.
Run the engine and point your client at it.
290B+ open-weight MoE models on a gaming PC, per the project’s own claim.
CPU-GPU co-execution that adapts to the bandwidth your machine actually has.
Semantic anchor checkpoints for recurrent state and KV caches, so agentic context edits skip recomputation.
Dynamic VRAM re-allocation between expert caches and KV memory, without restarts or weight reloading.
DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other frontier open-weight MoE models.
The FTW fast weight format plus double-buffered prefill streaming for interactive speeds.
Download the source and move into the folder.
git clone https://github.com/FlashML-org/FreeToken.git && cd FreeTokenInstall in editable mode with the acceleration extras.
uv pip install -e ".[accel]"Pick a supported model and follow the README serving example.
# see README for the exact serve command and model flagsFor comparison, the same shelf also holds llama.cpp — LLM inference in C/C++ and colibri — Tiny C engine that runs huge MoE models on the. The complete list is the Local LLM shelf. Before running a model locally, check how much VRAM you need for local LLMs — model size has to match your hardware.
An edge-native MoE serving engine that runs frontier-scale open-weight models on personal and consumer hardware.
Frontier open-weight MoE models including DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2.
No — that is the point: it treats your GPU, CPU, host memory and interconnects as one elastic inference platform.
Yes, Apache-2.0 licensed. You still need the model weights, which you download yourself.
Clone the repo and install with uv, including the [accel] extras; the README has the serving examples.
LLM inference in C/C++
Tiny C engine that runs huge MoE models on the machine you already own — GLM, Kimi K3, DeepSeek, Qwen, Apache-2.0
GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required
The simplest way to run open language models on your own machine.
Run controllable pre-trained AI models locally, written in Rust.
Find the best local models for your hardware — and how to run them.
Os outros projetos de LLM local com mais estrelas no catálogo do LibreAI — a lista realista de alternativas se FreeToken não for o ideal para você.
The simplest way to run open language models on your own machine.
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
LLM inference in C/C++
A high-throughput and memory-efficient inference and serving engine for LLMs
🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.
Projetos de LLM local próximos no catálogo, os mais próximos em estrelas de FreeToken: continue explorando dos dois lados dele.
High-performance In-browser LLM Inference Engine
Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context
GPT-OSS — OpenAI open-weight language models: gpt-oss-120b for production reasoning and gpt-oss-20b for local use, Apache-2.0
Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.
A fast LLM inference server for Apple Silicon, managed from the macOS menu bar
A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.
Grátis e aberto: pegue o código-fonte, dê uma estrela no repositório e comece a criar.