AgentCore Scores Any OpenTelemetry Agent—Framework Lock-In Shrinks

The August 26, 2026 AWS announcement details how Amazon Bedrock AgentCore Evaluations can score agents built with LangGraph, LlamaIndex, OpenAI Agents SDK, Google ADK, Claude Agent SDK and Strands Agents. The same generic route can cover other frameworks when their telemetry follows recognized OpenTelemetry GenAI or OpenInference conventions.

The practical boundary is telemetry compatibility, not the orchestration SDK. AgentCore must be able to identify a complete user turn, correlate its model and tool activity, and reconstruct a session before an evaluator can judge it. An independent technical analysis published the following day also identifies correctly structured OpenTelemetry data as the common contract across the six named frameworks.

OpenTelemetry reduces framework coupling, with conditions

AgentCore does not require access to a framework’s internal graph, state machine or private execution model. It reads the observable work recorded in spans and correlated event records, classifies that activity and supplies the reconstructed interaction to the evaluation layer.

Three span roles carry the minimum evidence described by AWS. An invoke-agent span anchors the top-level request and final response; inference spans contain the messages supplied to a model and its replies; execute-tool spans identify tool calls, arguments and results. Retrieval, reranking, memory and guardrail activity can add context, but those records do not replace the three core roles.

Scope names determine which generic parser handles the data. Scopes under opentelemetry.instrumentation.* use the OpenTelemetry GenAI path, while openinference.instrumentation.* selects the OpenInference path. A custom scope outside those namespaces is not accepted merely because some of its attributes look similar.

Complete content matters as much as span structure. Related records need a consistent session.id; each trace represents a user turn, and message content must remain available either on the spans or in correlated event records. A trace containing operation names but no prompt or response cannot support a response-quality judgment.

Compatibility preflight for the six named frameworks

The named integrations use different instrumentors and attribute layouts, but their minimum path converges: produce a top-level turn, inference and tool records; attach the session identifier; retain conversational content; and deliver all correlated telemetry to the CloudWatch source used by the evaluation workflow.

• LangGraph: use a supported OpenTelemetry or OpenInference instrumentor and preserve the root agent turn, model exchanges, tool calls and session correlation.
• LlamaIndex: instrument a workflow agent such as FunctionAgent or ReActAgent. The root workflow span anchors the turn; child inference and tool spans provide the evidence needed for scoring.
• OpenAI Agents SDK: install a compatible instrumentor and keep the SDK tracing pipeline enabled. Disabling that pipeline also prevents its instrumentation hooks from producing evaluation spans.
• Google ADK: emit records through its supported instrumentation route with recoverable user messages, model replies and tool activity associated with the same session.
• Claude Agent SDK: use the supported OpenInference route and retain indexed input and output messages, tool names, arguments and results.
• Strands Agents: enable its supported telemetry integration so the agent turn, inference activity and tool evidence can be collected and correlated.

An unlisted framework faces the same substantive test. Its instrumentation scope must select one of the generic convention paths, and its payload must expose enough structured content to reconstruct the interaction. “Any OpenTelemetry agent” therefore means any agent satisfying this contract—not arbitrary application spans carrying an OpenTelemetry label.

External agents still need the AWS telemetry path

Hosting is not the compatibility boundary. The AgentCore Evaluations documentation states that the service can evaluate agents running both inside and outside AgentCore Runtime. External agents still require supported instrumentation and telemetry available through an accessible CloudWatch data source.

For AgentCore Runtime deployments, AWS Distro for OpenTelemetry routes spans and event records to CloudWatch. In external environments, developers must configure the equivalent delivery and session propagation themselves. Moving the agent outside AgentCore Runtime does not create a direct framework-to-evaluator shortcut.

Telemetry placement can determine whether a session is usable. Under unified observability, spans and message content can reside in the agent’s log group; older split configurations may place structural spans and correlated content in separate groups. Selecting only the structural records can leave the evaluator without the prompt or answer it is expected to score.

Buffered delivery introduces another completeness risk. Tracing and logging providers can hold spans and event records in separate client-side queues, so both need to finish exporting before a short-lived runtime suspends the process. Otherwise, CloudWatch may receive only part of the interaction.

The same reconstructed session feeds shared metrics

Once AgentCore has reconstructed the session, framework-specific parsing is finished. GoalSuccessRate evaluates behavior across a session, while Correctness and Helpfulness operate at the trace level. Teams can also create custom LLM-as-a-judge or code-based evaluators without maintaining a separate metric definition for every compatible SDK.

AgentCore offers on-demand, online and batch evaluation modes. On-demand and batch workflows can incorporate reference information such as expected responses, assertions or expected tool trajectories. Online evaluation samples live sessions, so an evaluator that depends on ground-truth fields unavailable in production traffic cannot derive those fields from telemetry alone.

The lock-in reduction is therefore specific to framework choice: teams can retain different agent SDKs while applying one AWS evaluation suite. CloudWatch ingestion, recognized semantic conventions, session correlation and complete conversational telemetry remain dependencies. As of August 28, 2026, AWS has documented the common scoring path and six named framework examples, but interoperability still ends where compatible telemetry becomes incomplete or leaves the AWS evaluation data path.

Also read:

• Hermes Agent v0.12: Why Developers Are Ditching OpenClaw for Nous Research’s Self-Improving Agent
• Threatening AI Doesn’t Improve Accuracy—and Agent Tests Show a Different Risk
• AI Agents for Business, Not Demos: How Selleo Approaches Agent Development