PGAIBench-Auditorpgaibench(1) / auditor | Version | pgaibench-core 0.1.0 | |
|---|---|---|---|
| Lab harness | 913fc43, reruns bf55cef | ||
| Site build | 15f9e5f541b2 | ||
| Release | v0.1.0 (2026-11-11) | ||
| Updated | 2026-11-12T00:04:11Z UTC | ||
| Lab cells | 263 of 264 finished | ||
One benchmark for the Auditor role, with two jobs: pre-op review and post-op review.
PGAIBench-Auditor tests the Auditor role of the AAA design: a model reviews a change request that someone else wrote.
- Pre-op review: the model gets a finished change request before it runs, and must catch the dangerous ones and approve the good ones. In preparation; first results are expected soon.
- Post-op review: the model reviews what happened after the change ran. Planned.
- Both jobs are one role and one benchmark. The change requests and the lab come from PGAIBench-Analyzer.
- Results are in preparation. Nothing on this site scores the Auditor role yet.