PromptBeat and AgentBeat share an emphasis on explicit evaluation inputs and inspectable results. Their current schemas and runtimes are different, so this page keeps the vocabulary separated.

​
PromptBeat vocabulary

TermMeaning
TargetThe model or Prompt-driven endpoint being evaluated.
ScenarioA risk or behavior to test, with the expected response boundary.
SeedSource material used to create concrete test cases.
Project configThe PromptBeat YAML that selects Target, Scenario, Seed, providers, and external evaluation-backend options.
Generated caseA deterministic local case artifact produced by promptbeat generate.
Evaluation resultThe normalized result parsed from the external execution backend.
Dataset versionA selected, versioned set of cases for repeatable regression.
The PromptBeat flow is documented in PromptBeat overview.

​
AgentBeat vocabulary

TermMeaning
AgentCaseA complete 21-field evaluation case describing the task, risk, expected behavior, and observation tier. It does not own endpoints or secrets.
Target ProfileThe evaluator-owned identity and connector plan for the HTTP Agent deployment.
Target CapabilityThe input, environment, and Evidence channels a registered deployment declares.
Deployment ProfileThe Registry entry that binds profile, capability, endpoint, credentials, and optional State Controller.
Observation tierL1 final response, L2 Target Evidence, or L3 independent environment state verification.
EvidenceTier-qualified observations used by deterministic checks and the LLM Judge. Unobserved data remains unavailable.
Scoring profilePositive deterministic and Judge component weights that sum to 1.
MetricsUtility, Security, ASR, and Overall, each preserving applicability and coverage.
The AgentBeat protocol is documented in Observation model and Evidence and scoring.

​
Names that look similar but are not interchangeable

Do not collapseWhy
PromptBeat Scenario and AgentCaseThey use separate schemas and are not interchangeable.
PromptBeat provider and AgentBeat Target deploymentOne configures the external-backend model flow; the other is an evaluator-owned HTTP Agent Registry entry.
PromptBeat result and AgentBeat EvidenceA provider evaluation result is not proof of tool calls or state changes.
LLM Judge and native L3 verificationA Judge interprets Evidence; native verification observes authoritative environment state and can act as a hard gate.

​
A practical rule

Ask what the evaluator can actually observe. If you only have a response, make a response-scoped claim. If a registered Target also returns declared Evidence, qualify the claim as L2. If an independent controller verifies state, qualify it as L3.