AWS open-sources UAEF agent evaluation framework

AWS has open-sourced UAEF (Universal Agent Evaluation Framework) under the Apache 2.0 license in the awslabs/uaef GitHub repository. The framework-agnostic system uses adapters to normalize agent traces from LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore and Langfuse into a single canonical AgentTrace, then scores them with 36 built-in metrics across 7 dimensions. Running the library alone creates nothing; persisting a run writes experiment metadata to Amazon DynamoDB and full results to Amazon S3, letting teams compare agent versions against a baseline and flag regressions at a default 5 percent threshold, with statistical reports and LLM-written insights. AWS says 12 organizations in telecom, healthcare and financial services used UAEF before release. The DynamoDB tables, S3 buckets and Cognito user pool are created with a retain policy, so deleting the stack leaves them and their contents in place.

สร้างจากรายงาน · อัปเดต 20 ชม. ที่แล้ว

ยังมีการเฝ้าดูต่อเนื่องไม่พอจะวาดแนวโน้ม