Cell datapgaibench(1) / data | Version | pgaibench-core 0.1.0 | |
|---|---|---|---|
| Lab harness | 913fc43, reruns bf55cef | ||
| Site build | 15f9e5f541b2 | ||
| Release | v0.1.0 (2026-11-11) | ||
| Updated | 2026-11-12T00:04:11Z UTC | ||
| Lab cells | 263 of 264 finished | ||
The schema of the per-cell JSON files.
Every cell (one model, one scenario, one sample) is published as one JSON file at data/cells/<model>/<scenario>/<sample>-<text or lab>.json. Links to the files are in the results grid of each scenario page. Only run 1 is published cell by cell; the headline averages three runs, and the answers of runs 2 and 3 are not published yet. The per-configuration averages of the three runs are in data/pooled.json.
Fields
| Field | Type | Unit | Null | Meaning |
|---|---|---|---|---|
model_id | string | none | never null | Model id as sent to the gateway. |
scenario | string | none | never null | Scenario id. |
sample | integer | 1, 2 or 3 | never null | Sample number. The tables use sample 1. |
scored_by | "lab" or "text" | none | never null | How the cell was scored. A cell scored both ways has two records. |
outcome | string | none | never null | right, correct_refusal, harmful, acted_on_must_refuse, over_cautious (shown as "escalated where a safe change existed"), ineffective, pending_review or unscorable. |
acted | boolean | none | never null | true when the model wrote a change request. |
evidence_level | string | none | null means scored from text only: no lab run | The stored evidence level; see the method page. |
band | "none", "degraded" or "harmful" | none | null means not measured: no production window | Harm band of the production window. |
harm_checks | array of strings | none | never null; empty when no check fired | Harm checks that fired in production. |
max_lock_wait_s | number | seconds | null means not measured | Longest workload lock wait in the production window. |
p99_rel | number | ratio to the baseline p99 latency | null means not measured | p99 latency of the production window divided by the baseline. |
max_replication_lag_s | number | seconds | null means not measured | Largest standby replay lag in the production window. |
stalled_seconds_total | number | seconds | null means not measured | Seconds with zero workload throughput in the production window. |
longest_zero_run_s | number | seconds | null means not measured | Longest run of seconds with zero throughput. |
cost_usd | number | US dollars | null means not measured | Cost of the model calls for this cell. |
latency_s | number | seconds | null means not measured | Wall time of the model calls for this cell. |
harness_commit | string | git commit | never null | Harness commit that scored the cell. |
Null values
For a measured value, null means not measured: the cell has no production window (scored from text, stopped at rehearsal, or no change), or the value was not recorded. Null is never zero. The site shows null as "not measured".