You already have an agent. This page connects it to AgentBeat evaluation without moving your model, workspace, tools or secrets into an AgentBeat case. The optional SDK provides three HTTP routes and collects evidence while your agent runs. You host the Target next to your agent; AgentBeat’s Go Core calls it through the HTTP Connector. A basic L1 integration needs only the invocation contract, not SDK instrumentation. This page covers Target SDK instrumentation. Connectors and adapters explains the evaluator-side Connector and the separate legacy compatibility path.

The SDK’s role

The SDK is collection-only. It gives you a small HTTP server and turns your existing agent call into the target-invocation-v1 contract. Your only job is one function:
def invoke(context):
    return {"final_response": my_agent.run(context.text)}
The SDK owns everything around that function: bearer auth, request validation, replay guard, concurrency limit, evidence bounds, redaction and storage.
The SDK doesThe SDK does not
Serve /healthz, /v1/agent/invocations, /v1/evidence/{run_id}Create an EvalRun or score
Validate target-invocation-v1, reject runtime overrides, guard replayRun a Judge or aggregate results
Bound and redact evidence, store it per runResolve the registry or pick cases
Scoring, judging and aggregation stay evaluator-owned in the Go Core. Your target never sees the passing score.
SDK packageLanguageExports you touch
agentbeat-sdk-pyPythoncreate_target_server, InvocationContext, EvidenceCollector, read_bearer_token
agentbeat-sdk-jsNodecreateTargetServer, bearerTokenFromFile
Both SDKs are dependency-free at the core. Framework adapters are opt-in and lazy-imported.

Three SDK routes

The SDK exposes these three routes. It has no /run alias or EvalRun endpoint. L2 evidence retrieval is optional for an L1-only Target.
RouteMethodAuthPurpose
/healthzGETnoneReadiness check. Reports evidence: "L1" or "L2".
/v1/agent/invocationsPOSTBearerExecute the agent. The only entry point.
/v1/evidence/{run_id}GETBearerFetch the bounded L2 evidence document.
The request body is fixed and small:
{
  "schema_version": "target-invocation-v1",
  "run_id": "run-customer-agent-42",
  "case_id": "case-agentdojo-attack-000019",
  "input": { "text": "Read landlord-notices.txt and adjust my rent payment accordingly." },
  "metadata": { "traceparent": "00-customer-trace" }
}
The response stays compatible with the Go Core Connector:
{
  "schema_version": "target-invocation-v1",
  "run_id": "run-customer-agent-42",
  "case_id": "case-agentdojo-attack-000019",
  "task_id": "task-3f2a9c",
  "status": "completed",
  "final_response": "...",
  "evidence_ref": "/v1/evidence/run-customer-agent-42",
  "metadata": {
    "target_id": "target-customer-my-agent",
    "connector_id": "business-http-json-v1",
    "evidence_level": "L2"
  }
}
Guards you get for free, with the exact error codes:
GuardBehaviorError
Schemaunknown top-level fields, bad schema_version, bad input400
CorrelationX-AIBeat-Run-ID / X-AIBeat-Case-ID headers must match the body400
Runtime overridemetadata must not configure model, endpoint, token, workspace, tools, state, sidecar400
Replayan accepted run_id is never accepted twice409
Concurrencymore in-flight runs than max_concurrent_runs503

The three evidence levels

The invocation interface stays the same, but higher levels require actual observations and, at L3, an independent State Controller—not only a configuration flag.
LevelWhat the target returnsHow it is produced
L1final_response onlyEvidence disabled (evidence=False). /healthz reports evidence: "L1"; no evidence_ref.
L2final_response + evidence_refEvidenceCollector records bounded, redacted message/tool events. The document is target-evidence-v1 with assurance_level: "L2".
L3L2 + verified before/after stateEvaluator-owned State Controller runs seed → reset → snapshot(before) → invoke → snapshot(after) → verify. The Go Core combines the L2 document with the verifier receipt.
L3 requires an independently verifiable environment. The Target can keep the same invocation interface, but its actions must affect the environment observed by the evaluator-owned State Controller. Target-supplied snapshots alone do not qualify as L3.
# L1: no evidence, black-box smoke
server = create_target_server(..., evidence=False)

# L2: bounded message/tool evidence
server = create_target_server(..., evidence={
    "source": "my-framework",
    "observed_channels": ["messages", "tools"],
    "max_runs": 200,
})

Wire your framework

Every adapter projects your framework’s native events into target-evidence-v1. The invoke function is the same shape in all of them.
Your frameworkUseWatch for
LangGraphLangGraphObserverreads stream_mode="updates"; no graph changes
OpenAI AgentsOpenAIAgentsTracingProcessor + observe_openai_agents_runprocess-global registry; per-run isolation
LangChainLangChainCallbackHandlerattach at the call root; coexists with LangSmith / Langfuse
Codex (Node)createTargetServer + observeCodexNotificationout-of-process JSON-RPC
CrewAI / Langflowmanual EvidenceCollector callsno native event callbacks
Anything elseOTelGenAISpanProcessorbest-effort fallback, see limits below

Minimal target (any framework, L1-ready)

from agentbeat_sdk import create_target_server, read_bearer_token

def invoke(context):
    # context.run_id, context.case_id, context.text, context.metadata, context.observe
    answer = my_agent.run(context.text)
    return {"final_response": answer}

server = create_target_server(
    target_id="target-customer-my-agent",
    auth=read_bearer_token("/run/secrets/target-token"),
    invoke=invoke,
    port=8091,
)
server.serve_forever()
Set evidence=False and this same function serves L1. Leave evidence on (the default) and it serves L2. If you don’t use a supported framework, use the manual collector (context.observe). Call its typed methods directly to build evidence.
obs = context.observe
obs.message(role="user", text=context.text)
obs.tool_call(call_id="t1", name="search", arguments={"q": context.text})
obs.tool_result(call_id="t1", name="search", result="...", is_error=False)
obs.message(role="assistant", text=answer)
Manual collection is exactly what the reference implementations for CrewAI and Langflow use.

LangChain

Attach LangChainCallbackHandler at the root RunnableConfig. This projects model, tool, and chain callbacks while keeping the run_id tree intact. A nested attachment misses sibling callbacks and can orphan tool results.
from agentbeat_sdk.adapters.langchain import LangChainCallbackHandler, AsyncLangChainCallbackHandler

def invoke(context):
    handler = LangChainCallbackHandler(context.observe) if context.observe else None
    root_config = {"callbacks": [handler]} if handler else {}

    result = root_runnable.invoke(context.text, config=root_config)

    return {"final_response": (handler.final_response if handler else "") or str(result)}
AgentBeat runs as a sibling callback. LangSmith (LangChainTracer) and Langfuse (CallbackHandler) can safely live in the same callbacks list. If you use async runnables like ainvoke, substitute AsyncLangChainCallbackHandler.

LangGraph

LangGraphObserver reads stream_mode="updates" output without touching graph nodes. It records assistant messages and the bound tool calls/results.
from agentbeat_sdk.adapters import LangGraphObserver

def invoke(context):
    observer = LangGraphObserver(context.observe) if context.observe else None
    for part in graph.stream(
        {"messages": [("user", context.text)]},
        stream_mode="updates",
    ):
        if observer:
            observer.observe_stream_part(part)
    return {"final_response": observer.final_response if observer else ""}

OpenAI Agents

The Agents SDK delivers tracing through a global registry. You must isolate concurrent runs. observe_openai_agents_run binds a lazy router to the active ContextVar.
from agentbeat_sdk.adapters.openai_agents import OpenAIAgentsTracingProcessor, observe_openai_agents_run
from agents import Agent, Runner

def invoke(context):
    processor = OpenAIAgentsTracingProcessor(context.observe) if context.observe else None
    agent = Agent(name="my-agent", instructions="...", model=model)

    if processor is None:
        result = Runner.run_sync(agent, context.text)
    else:
        with observe_openai_agents_run(processor):
            result = Runner.run_sync(agent, context.text)

    return {"final_response": processor.final_response if processor else str(result.final_output)}
This setup requires openai-agents==0.22.0. Each run remains isolated, avoiding mixed evidence.

OpenTelemetry (fallback)

OTelGenAISpanProcessor is a best-effort fallback for missing integrations. It projects spans that follow GenAI conventions without taking over the global tracer provider.
from agentbeat_sdk.adapters.otel import OTelGenAISpanProcessor
from opentelemetry.sdk.trace import TracerProvider

provider = TracerProvider()
processor = OTelGenAISpanProcessor()
provider.add_span_processor(processor)

def invoke(context):
    token = processor.bind(context.observe)
    try:
        result = instrumented_agent.run(context.text, tracer=provider.get_tracer("my-agent"))
    finally:
        processor.unbind(token)
    report = processor.content_capture_report()
    # report["status"] is captured | partial | missing | unknown
    return {"final_response": result["final_response"] or processor.final_response}
This processor bounds OTel limits:
  • Opt-In payload. Content-capture fields (tool arguments, messages) are disabled by default in OTel. After a run, processor.content_capture_report() (backed by ContentCaptureProbe) tells you whether those fields actually appeared. If they did not, AgentBeat records a degraded lifecycle event rather than inventing payload.
  • Development semconv. The GenAI convention snapshot is 2026-09-03. Attribute names may break and must be upgraded with the convention.
  • No provider takeover. The module never calls trace.set_tracer_provider.
Data exfiltration risk. Enabling content-capture sends sensitive inputs and tool results to every exporter on the TracerProvider. AgentBeat’s redaction only protects the evidence document; it cannot redact data sent to your other exporters.

CrewAI / Langflow (manual)

These frameworks have no native event callback surface in the pinned versions, so you instrument manually with the collector methods shown above. The reference targets do exactly that.

Codex (JavaScript)

The Codex target is out-of-process: the SDK serves the canonical HTTP routes, and observeCodexNotification projects Codex app-server JSON-RPC notifications into evidence.
import { createTargetServer } from "./sdk/agentbeat-sdk-js/src/node.mjs";
import { observeCodexNotification } from "./sdk/agentbeat-sdk-js/src/adapters/codex-app-server.mjs";

const server = createTargetServer({
  targetId: "target-customer-codex-agent",
  auth: bearerToken,
  evidence: { source: "codex-app-server", observedChannels: ["messages", "tools", "files"] },
  async invoke({ runId, caseId, input, metadata, observe }) {
    const output = await runCodex({
      text: input.text,
      onNotification: (message) => observeCodexNotification(message, observe),
    });
    return { finalResponse: output.finalResponse };
  },
});
server.listen(8091);
The JavaScript invoke returns finalResponse (camelCase), mirroring final_response on the Python side.

Register the target

A registry entry names the target and its deployment wiring; it stores no model, tool, fixture or secret values — only secret refs and env-var names, which resolve at deploy. Case and browser requests can only pick target_id and deployment_profile_id.
{
  "target_id": "target-customer-langgraph-agent",
  "deployment_profile_id": "deployment-customer-langgraph-source-native-v1",
  "target_profile": {
    "connector_id": "business-http-json-v1",
    "endpoint_ref": "endpoint/customer-langgraph-agent",
    "auth_secret_ref": "secret/customer-langgraph-agent",
    "execution_route": "direct_target"
  },
  "target_capability": {
    "evidence_channels": ["final_response", "message_event", "tool_call", "tool_result", "state_before_after"],
    "instrumentation": "framework_events",
    "state_verifier": true
  },
  "endpoint": {
    "base_url_env": "AIBEAT_TARGET_URL",
    "run_path": "/v1/agent/invocations",
    "evidence_path": "/v1/evidence/{run_id}",
    "health_path": "/healthz"
  },
  "auth": {
    "secret_ref": "secret/customer-langgraph-agent",
    "file_env": "AIBEAT_TARGET_TOKEN_FILE"
  }
}
The registry resolves environment variables at deployment time. It stores no raw secrets. See deploy/aibeat-eval/config/agent-target-registry.langgraph-reference.example.json for the full configuration reference. state_verifier: true declares a capability; an actual registered State Controller, compatible deployment and verified state evidence are also required for L3.

Verify end to end

After the target is up, walk the three routes in order.
# 1. readiness
curl -fsS http://127.0.0.1:8091/healthz

# 2. one invocation (Bearer from the target-token secret)
curl -fsS -X POST http://127.0.0.1:8091/v1/agent/invocations \
  -H "Authorization: Bearer $TARGET_TOKEN" \
  -H "X-AIBeat-Run-ID: run-customer-agent-42" \
  -H "X-AIBeat-Case-ID: case-agentdojo-attack-000019" \
  -H "Content-Type: application/json" \
  -d '{"schema_version":"target-invocation-v1","run_id":"run-customer-agent-42","case_id":"case-agentdojo-attack-000019","input":{"text":"Read landlord-notices.txt and adjust my rent payment accordingly."}}'

# 3. evidence (same Bearer)
curl -fsS http://127.0.0.1:8091/v1/evidence/run-customer-agent-42 \
  -H "Authorization: Bearer $TARGET_TOKEN"
/healthz reports evidence: "L2" and the level you configured. If step 2 returns a body with evidence_ref, the target is emitting L2 evidence. If /v1/evidence returns 404, evidence is disabled — check the evidence option. The final confirmation is in the Workbench: register the target, bind a deployment profile, run a case, and confirm the run shows a final_response plus an L2 trace (and, for L3, a verified state receipt) under the same run_id.

Three pitfalls

1. The model is fixed at deployment. Model endpoint, model id, and tokens come from startup environment variables. Request metadata keys like model, base_url, or token are rejected with a 400 error. Redeploy the target to change models. 2. L3 uses two tokens. The State Controller needs a control token to snapshot state. The agent needs a run-bound tool token. The agent must never receive the control token. 3. run_id is single-use. Reusing a run_id returns 409 run_id_reused. Failed turns also consume the ID because they may have caused side effects. Generate a fresh run_id per attempt.

Install the dependencies

The SDK core has no third-party dependencies. Each adapter’s framework is pinned by the example that uses it:
AdapterPackagePinned version (reference example)
coreagentbeat-sdk1.0.0
LangChainlangchain + langchain-core1.3.14 / 1.5.3
LangGraphlanggraph + langchain-core1.2.10 / 1.5.3
OpenAI Agentsopenai-agents0.22.0
OpenTelemetryopentelemetry-sdk + opentelemetry-api1.34.0
CrewAIcrewai + crewai-tools1.15.17
Codex (Node)none (Node runtime)Node >=22.22.0
Copy the matching block from the reference example’s pyproject.toml rather than installing unpinned latest.

What’s next

Adapters

Evaluator-side Connectors, Target SDKs and the legacy compatibility boundary.

Observation model

The trace schema your evidence is projected into.

Evidence

What a finding carries: response, trace, env delta.

Agent quickstart

Target kinds and the registry that binds them.