LibreAI / AI Agents / agent-memory-benchmark
// agents · vectorize-io/agent-memory-benchmark

agent-memory-benchmark

agent-memory-benchmark is an open-source ai agents project by vectorize-io (vectorize-io/agent-memory-benchmark on GitHub). It has 144 GitHub stars and 47 forks, is released under the NOASSERTION license, and was last updated on GitHub on 7 October 2026.

A benchmark for comparing AI agent memory systems head-to-head.

★ 144 stars⑂ 47 forksNOASSERTIONupdated 2026-10
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
// project facts

Project facts

GitHub stars
144
Forks
47
License
NOASSERTION
Last updated on GitHub
7 October 2026
Category
AI Agents
Listing
Full install guide
// how it works

How it works

  1. Define the job

    Know what you want the agent to accomplish before touching any code.

  2. Get the code

    Clone the repo and read the README's requirements section.

  3. Wire up dependencies

    Install what's needed and add API keys or configuration.

  4. Take it for a spin

    Run the quickstart example, then iterate from there.

// features

Why people love it

Standardized tasks

Same tests for every system.

Head-to-head comparisons

Apples-to-apples memory evals.

Memory-focused

Recall quality, not generic chat.

Open methodology

Inspect how scores are computed.

Vectorize-built

From the Hindsight team.

Active development

Updated as the field moves.

Advertisement
// run it

Run it yourself

  1. Step 1 — Clone the repo

    Download the source and move into the project folder.

    git clone https://github.com/vectorize-io/agent-memory-benchmark.git
    cd agent-memory-benchmark
  2. Step 2 — Create a venv and install

    Isolate dependencies, then install requirements.

    python -m venv .venv && source .venv/bin/activate
    pip install -r requirements.txt
  3. Step 3 — Run it

    Launch using the entry point shown in the README.

    python -m agent_memory_benchmark  # check README for the exact command
// use cases

What you can build

  • Compare memory systems fairly.
  • Validate memory product claims.
  • Research memory architectures.
  • Choose a memory layer with data.
Advertisement
// faq

Questions, answered

Who maintains agent-memory-benchmark?

It is an open-source project at vectorize-io/agent-memory-benchmark with 47 forks. Check the contributors tab and recent commits to judge how active maintenance is.

Is agent-memory-benchmark free to use?

GitHub lists no license for this repo, so legally it's all rights reserved — check the README or ask the maintainer before building on it.

What does it measure?

How well agent memory systems recall and use information — standardized tasks for head-to-head comparison.

Who's it for?

Researchers and vendors who want memory performance as numbers, not claims.

How do I get started?

Clone the repo and follow the README quickstart — the README is the best starting point.

// keep exploring

Similar tools

Agents

trpc-agent-go

Go framework for production agent systems: graphs, tools, memory, A2A, MCP.

★ 1.8k starsApache-2.0
View guide →
AI Agents

Agent_Memory_Techniques

30 runnable notebooks teaching everything about agent memory.

★ 1.1k starsApache-2.0
View guide →
AI Agents

webarena-verified

WebArena-Verified: the audited, deterministically scored release of the WebArena web-agent benchmark.

★ 62 starsApache-2.0
View guide →
AI Agents

OpenViking

A self-evolving context database unifying agent memory, RAG, and skills.

★ 39.4k starsAGPL-3.0
View guide →
AI Agents

osaurus

Osaurus — own your AI: a native macOS agent harness with any model, memory and fully offline execution.

★ 8.0k starsMIT
View guide →
AI Agents

memanto

A memory agent for your agent fleet: it curates, reconciles, and expires shared memories across tools.

★ 2.3k starsMIT
View guide →
// alternatives

Alternatives in AI Agents

The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if agent-memory-benchmark is not the right fit.

  • openclaw★ 391.6k

    A personal AI that really does things — on any OS, with your data staying yours.

  • superpowers★ 294.9k

    An agentic skills framework & software development methodology that works.

  • ECC★ 275.2k

    Agent harness optimization system with skills, memory and security for Claude Code, Codex and more

  • hermes-agent★ 252.0k

    The agent that grows with you — a Python framework for adaptive AI agents.

  • deepseek-harness★ 245.5k

    DeepSeek's plugin-based agent harness where everything is a plugin.

// more in agents

More in AI Agents

Nearby AI Agents projects in the catalogue, closest in stars to agent-memory-benchmark — keep exploring the shelf either side of it.

  • octomind★ 151

    An MCP-native AI coding agent and runtime in a single binary.

  • OpenYabby★ 130

    A voice-driven multi-agent assistant that orchestrates in realtime.

  • evals★ 230

    A complete evaluation toolkit for AI agents.

  • babyagi_archive★ 110

    A frozen snapshot of the original BabyAGI autonomous-agent experiment.

  • WorkArena★ 275

    WorkArena: ServiceNow's benchmark for how well web agents handle everyday knowledge-work tasks.

  • ActPlane★ 104

    eBPF-powered information-flow enforcement for safer AI agents.

Get agent-memory-benchmark

Free and open — grab the source, star the repo, and start creating.