LibreAI / Local LLM / FreeToken
// local-llm · FlashML-org/FreeToken

FreeToken

FreeToken is an open-source local llm project by FlashML-org (FlashML-org/FreeToken on GitHub). It has 17,211 GitHub stars and 1,705 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 15 November 2026.

Run 290B+ frontier MoE models on your own gaming PC at interactive speed

★ 17.2k stars⑂ 1.7k forksApache-2.0updated 2026-11
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
// project facts

Project facts

GitHub stars
17,211
Forks
1,705
License
Apache-2.0
Last updated on GitHub
15 November 2026
Category
Local LLM
Listing
Full install guide
Official site
Official site
// how it works

How it works

  1. Clone the repo

    Get the source and move into the project folder.

  2. Install it

    Install with uv, including the acceleration extras.

  3. Pick a model

    Choose a supported open-weight MoE model and follow the README serving example.

  4. Serve it

    Run the engine and point your client at it.

// features

Why people love it

Frontier MoE on consumer hardware

290B+ open-weight MoE models on a gaming PC, per the project’s own claim.

Bandwidth-adaptive co-execution

CPU-GPU co-execution that adapts to the bandwidth your machine actually has.

Semantic-aware caching

Semantic anchor checkpoints for recurrent state and KV caches, so agentic context edits skip recomputation.

Elastic memory

Dynamic VRAM re-allocation between expert caches and KV memory, without restarts or weight reloading.

Broad MoE support

DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other frontier open-weight MoE models.

Fast weight format

The FTW fast weight format plus double-buffered prefill streaming for interactive speeds.

// run it

Run it yourself

  1. Step 1 — Clone the repo

    Download the source and move into the folder.

    git clone https://github.com/FlashML-org/FreeToken.git && cd FreeToken
  2. Step 2 — Install it

    Install in editable mode with the acceleration extras.

    uv pip install -e ".[accel]"
  3. Step 3 — Start serving

    Pick a supported model and follow the README serving example.

    # see README for the exact serve command and model flags
// use cases

What you can build

  • Run a 290B+ open-weight MoE model on hardware you already own.
  • Serve frontier models locally for agents without a datacenter bill.
  • Keep long agent contexts cheap with semantic-aware KV caching.
  • Reallocate VRAM between experts and KV memory without restarting.

For comparison, the same shelf also holds llama.cpp — LLM inference in C/C++ and colibri — Tiny C engine that runs huge MoE models on the. The complete list is the Local LLM shelf. Before running a model locally, check how much VRAM you need for local LLMs — model size has to match your hardware.

// faq

Questions, answered

What is FreeToken?

An edge-native MoE serving engine that runs frontier-scale open-weight models on personal and consumer hardware.

Which models does it support?

Frontier open-weight MoE models including DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2.

Do I need a datacenter GPU?

No — that is the point: it treats your GPU, CPU, host memory and interconnects as one elastic inference platform.

Is it free?

Yes, Apache-2.0 licensed. You still need the model weights, which you download yourself.

How do I install it?

Clone the repo and install with uv, including the [accel] extras; the README has the serving examples.

// keep exploring

Similar tools

Local LLM

llama.cpp

LLM inference in C/C++

★ 130.2k starsMIT
View guide →
Local LLM

colibri

Tiny C engine that runs huge MoE models on the machine you already own — GLM, Kimi K3, DeepSeek, Qwen, Apache-2.0

★ 49.3k starsApache-2.0
View guide →
Local LLM

GPT4All

GPT4All — run local LLMs on any device: a free desktop chat client and Python SDK that run open models entirely on your CPU, no GPU or cloud required

★ 77.4k starsMIT
View guide →
Local LLM

Ollama

The simplest way to run open language models on your own machine.

★ 182.1k starsMIT
View guide →
Local LLM

kalosm

Run controllable pre-trained AI models locally, written in Rust.

★ 2.2k starsApache-2.0
View guide →
Local LLM

whatcanirun

Find the best local models for your hardware — and how to run them.

★ 247 starsMIT
View guide →
// alternatives

Alternatives in Local LLM

The most-starred other Local LLM projects in the LibreAI catalogue — the realistic shortlist if FreeToken is not the right fit.

  • Ollama★ 182.1k

    The simplest way to run open language models on your own machine.

  • Open Webui★ 153.9k

    User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

  • llama.cpp★ 130.2k

    LLM inference in C/C++

  • vLLM★ 93.1k

    A high-throughput and memory-efficient inference and serving engine for LLMs

  • lobehub★ 83.0k

    🤯 LobeHub is your Chief Agent Operator, organizing your agents into 7×24 operations by hiring, scheduling, and reporting on your entire AI team.

// more in local-llm

More in Local LLM

Nearby Local LLM projects in the catalogue, closest in stars to FreeToken — keep exploring the shelf either side of it.

  • Web LLM★ 19.2k

    High-performance In-browser LLM Inference Engine

  • Qwen3-Coder★ 16.8k

    Qwen3-Coder - Alibaba's open code model series for agentic coding, up to 1M tokens of context

  • GPT-OSS★ 20.5k

    GPT-OSS — OpenAI open-weight language models: gpt-oss-120b for production reasoning and gpt-oss-20b for local use, Apache-2.0

  • MNN★ 16.2k

    Alibaba's lightweight, blazing-fast inference engine for on-device LLMs and edge AI.

  • omlx★ 22.6k

    A fast LLM inference server for Apple Silicon, managed from the macOS menu bar

  • nano-vllm★ 15.7k

    A lightweight vLLM implementation built from scratch in about 1,200 readable lines of Python.

Get FreeToken

Free and open — grab the source, star the repo, and start creating.