AWS open-sources UAEF for cross-framework evaluation of AI agents
The story so far
AWS has released the open-source framework UAEF, used for cross-framework closed-loop evaluation of AI agents. It unifies the execution traces of LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into AgentTrace through adapters, then scores them using 7 dimensions and 36 built-in metrics. UAEF is open-sourced under the Apache 2.0 license in the awslabs/uaef repository. Standard answers (ground truth) are optional, and only metrics that need to be compared with reference answers use them; when standard answers are not available, UAEF automatically skips these metrics, so partially labeled datasets and unlabeled production traffic can still be evaluated. Running this library does not produce any persistent content; if you choose to persist a single run, experiment metadata will be written to Amazon DynamoDB, and complete results will be written to Amazon S3.
AI-written from the reports · Updated 3 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- AWS Machine Learning BlogFeaturedAWS open-sources UAEF: cross-framework closed-loop evaluation AI agents
AWS releases open-source framework UAEF, using adapters to unify the execution traces of LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into AgentTrace, then scoring with 7 dimensions and 36 built-in metrics.
Heat over time
- Now
- 3
- Peak
- 1012 Nov, 01:00
- 24h change
- -49%
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.