AI Agents
bisheng
Enterprise LLM app platform: GenAI workflows, RAG, agents, model management, evaluation, fine-tuning.
★ 12.0k starsApache-2.0
View guide →// agents · strands-agents/evals
evals
evals is an open-source ai agents project by strands-agents (strands-agents/evals on GitHub). It has 230 GitHub stars and 71 forks, is released under the Apache-2.0 license, and was last updated on GitHub on 7 October 2026.
A complete evaluation toolkit for AI agents.
★ 230 stars⑂ 71 forksApache-2.0updated 2026-10
▦ visual — this project's README ships no embeddable screenshot,
so here's a placeholder. The guide below is complete regardless.
so here's a placeholder. The guide below is complete regardless.
// project facts
Project facts
- Repository
- strands-agents/evals
- GitHub stars
- 230
- Forks
- 71
- License
- Apache-2.0
- Last updated on GitHub
- 7 October 2026
- Category
- AI Agents
- Listing
- Full install guide
- Official site
- Official site
// how it works
How it works
- Define what you want
Write down the job you will use the project for before touching any code.
- Clone the repository
Download the source and read the README's setup section first.
- Install dependencies
Run the install steps and add any required API keys or config.
- Run the quickstart
Follow the example in the docs to confirm everything works.
// features
Why people love it
Agent evaluation
Measure agent quality systematically.
Comprehensive
Covers multiple evaluation dimensions.
LLM-app ready
Works for agents and LLM apps alike.
Strands ecosystem
From the Strands Agents team.
Python-native
Fits Python agent workflows.
Apache-2.0
Permissive licensing.
Advertisement
// run it
Run it yourself
- Step 1 — Clone the repo
Download the source and move into the project folder.
git clone https://github.com/strands-agents/evals.git cd evals - Step 2 — Install
Set up a virtual environment and install dependencies.
python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt - Step 3 — Run an evaluation
Execute a first eval per the README.
# see README for the eval runners
// use cases
What you can build
- Evaluate a Strands agent before launch.
- Track agent quality across iterations.
- Compare agent configurations objectively.
- Build eval-driven agent development.
Advertisement
// faq
Questions, answered
Who maintains evals?
It is maintained by the Strands Agents team at strands-agents/evals, with 71 forks.
Is evals free to use?
Yes. It is released under the Apache-2.0 license, which permits commercial use, modification, and redistribution. Read the LICENSE file for the exact terms.
Do I need a paid API to use it?
The framework is free; evaluations consume model calls through your own keys.
How does it compare to DeepEval?
Compare on metrics, workflow, and how tightly each fits your agent stack.
How do I get started?
Follow the README to run a first evaluation against your agent.
// keep exploring
Similar tools
AI Agents
vocode-core
Build voice-based LLM agents with a modular, open-source toolkit.
★ 3.8k starsMIT
View guide →AI Agents
promptfoo
Test prompts, agents and RAGs — LLM evals, red teaming and model comparisons from the CLI
★ 25.8k starsMIT
View guide →AI Agents
phoenix
Observability and evaluation for AI apps — see what your agents do.
★ 11.7k starsNOASSERTION
View guide →AI Agents
agent-governance-toolkit
Policy, identity, and sandboxing guardrails for AI agents.
★ 6.4k starsMIT
View guide →AI Agents
mantis
Google's modular toolkit that lets AI coding agents hunt, reproduce, and patch vulnerabilities autonomously.
★ 2.4k starsApache-2.0
View guide →// alternatives
Alternatives in AI Agents
The most-starred other AI Agents projects in the LibreAI catalogue — the realistic shortlist if evals is not the right fit.
- openclaw★ 391.6k
A personal AI that really does things — on any OS, with your data staying yours.
- superpowers★ 294.9k
An agentic skills framework & software development methodology that works.
- ECC★ 275.2k
Agent harness optimization system with skills, memory and security for Claude Code, Codex and more
- hermes-agent★ 252.0k
The agent that grows with you — a Python framework for adaptive AI agents.
- deepseek-harness★ 245.5k
DeepSeek's plugin-based agent harness where everything is a plugin.
// more in agents
More in AI Agents
Nearby AI Agents projects in the catalogue, closest in stars to evals — keep exploring the shelf either side of it.
- WorkArena★ 275
WorkArena: ServiceNow's benchmark for how well web agents handle everyday knowledge-work tasks.
- octomind★ 151
An MCP-native AI coding agent and runtime in a single binary.
- OSWorld-V2★ 353
Benchmark computer-use agents on long, real-world desktop tasks.
- agent-memory-benchmark★ 144
A benchmark for comparing AI agent memory systems head-to-head.
- A-mem-sys★ 401
Agentic memory for LLM agents, from a published research project.
- OpenYabby★ 130
A voice-driven multi-agent assistant that orchestrates in realtime.
Get evals
Free and open — grab the source, star the repo, and start creating.