Start with a local example. You do not need a model account or API key to validate the bootstrap project and preview its test cases.

​
1. Download the full package

Choose your operating system. These v0.2 full packages include the CLI, examples, Skills, Node.js and the evaluation runtime—no separate runtime install for this walkthrough.

macOS · Apple Silicon

Download .tar.gz

macOS · Intel

Download .tar.gz

Linux · x64

Download .tar.gz

Windows · x64

Download .zip
Release notes and asset checksums · GitHub repository
The newer v0.3-agentbeat-preview.1 release contains standalone executables, not these full packages. Use the version linked above to follow the bundled-example instructions below.

​
2. Extract and open the package folder

Open a terminal in the folder containing your download, then select the matching instructions. 
tar -xzf promptbeat-0.2-darwin-arm64.tar.gz
cd promptbeat-0.2-darwin-arm64
./bin/promptbeat --version
Stay in this package folder for the remaining commands. It contains bin/, examples/, runtime/ and promptbeat-skills/. On macOS, if the operating system blocks the downloaded executable, review its security prompt and your organization’s policy; do not disable system-wide protections.

​
3. Preview cases without calling a model

mkdir -p artifacts
./bin/promptbeat validate --config examples/bootstrap/promptbeat.yaml
./bin/promptbeat generate --config examples/bootstrap/promptbeat.yaml --count 5 --output artifacts/cases.json
Open artifacts/cases.json in your editor. These are test inputs, not model responses or evaluation scores. --count 5 means up to five cases; a smaller Seed set can produce fewer. Local generate does not call a Target, generator or Judge. You can stop here. To have a coding assistant guide these steps, load a PromptBeat Skill.

​
4. Connect your model when you are ready

For a real evaluation, use examples/llm-basic/promptbeat.yaml. Read its README, scenarios and provider settings first. This project uses three roles: an attack generator, a Judge and your target model.
The full run sends test content to the configured providers and may incur model costs. Confirm the endpoints and scope before running it. Keep keys in local environment variables, not in prompts, YAML or committed files.
Replace every placeholder below with your own local settings. Unlike bootstrap, llm-basic validation requires these variables; validation alone does not check that a remote service or credential works. 
export ATTACKER_MODEL_NAME='<provider:model>'
export ATTACKER_BASE_URL='https://your-provider.example/v1'
export ATTACKER_API_KEY='<set-locally>'
export JUDGE_MODEL_NAME='<provider:model>'
export JUDGE_BASE_URL='https://your-provider.example/v1'
export JUDGE_API_KEY='<set-locally>'
export TARGET_MODEL_NAME='<provider:model>'
export TARGET_BASE_URL='https://your-provider.example/v1'
export TARGET_API_KEY='<set-locally>'
./bin/promptbeat validate --config examples/llm-basic/promptbeat.yaml
./bin/promptbeat run --config examples/llm-basic/promptbeat.yaml --output-dir artifacts/llm-basic/run
run generates and evaluates cases through the full pipeline. It does not take the preview JSON from step 3 as input.

​
5. Read the results

After a successful full run, open artifacts/llm-basic/run/report.html in your browser. Inspect evaluation_result.json in the same folder for the normalized result. The folder also retains backend configuration and raw evaluation artifacts; filenames depend on the selected project paths. To render the HTML report again from an existing result: 
./bin/promptbeat report --input artifacts/llm-basic/run/evaluation_result.json --output artifacts/llm-basic/run/report.html
No report is produced by the local preview alone. If a command fails, check the package folder, the config type and required environment variables before rerunning an evaluation.

Choose scenarios

Read your report

Use Skills