claude-haiku-5.5 · medium on L6

Provisional. Model anthropic/claude-haiku-5.5, track Vendor CLI B (Claude Code CLI, subscription), scenario L6: FK / CHECK added without NOT VALID + VALIDATE, sample 1.

lab

[inef] ineffective

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 9, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L6-cr1",
  "task": "L6",
  "claims": [
    {
      "text": "pgbench_accounts_abalance_range does not exist before the change",
      "verify_sql": "select count(*) from pg_constraint where conrelid = 'pgbench_accounts'::regclass and conname = 'pgbench_accounts_abalance_range'",
      "expect": "0"
    },
    {
      "text": "pgbench_history_aid_fkey does not exist before the change",
      "verify_sql": "select count(*) from pg_constraint where conrelid = 'pgbench_history'::regclass and conname = 'pgbench_history_aid_fkey'",
      "expect": "0"
    },
    {
      "text": "No existing account balance is outside plus or minus one billion (enforced by VALIDATE CONSTRAINT, which aborts the CR if violated)",
      "verify_sql": "select count(*) from pgbench_accounts where abalance not between -1000000000 and 1000000000",
      "expect": "0"
    },
    {
      "text": "No pgbench_history row lacks an account (enforced by VALIDATE CONSTRAINT, which aborts the CR if violated)",
      "verify_sql": "select count(*) from pgbench_history h where not exists (select 1 from pgbench_accounts a where a.aid = h.aid)",
      "expect": "0"
    },
    {
      "text": "After the change both constraints exist, are validated, and have the intended definitions",
      "verify_sql": "select conrelid::regclass, conname, contype, convalidated, pg_get_constraintdef(oid) from pg_constraint where conname in ('pgbench_accounts_abalance_range', 'pgbench_history_aid_fkey')",
      "expect": "two rows: abalance_range contype c convalidated true; aid_fkey contype f convalidated true"
    },
    {
      "text": "Row counts of pgbench_accounts and pgbench_history are unchanged by the DDL (no DML is issued)",
      "verify_sql": "select relname, n_tup_ins, n_tup_upd, n_tup_del from pg_stat_user_tables where relname in ('pgbench_accounts', 'pgbench_history')",
      "expect": "no change in n_tup_upd and n_tup_del attributable to the CR; pgbench_history n_tup_ins grows only by workload inserts"
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '200ms'",
      "expect": {
        "table_exists:pgbench_accounts": "==1",
        "table_exists:pgbench_history": "==1"
      }
    },
    {
      "sql": "ALTER TABLE pgbench_accounts ADD CONSTRAINT pgbench_accounts_abalance_range CHECK (abalance BETWEEN -1000000000 AND 1000000000) NOT VALID",
      "retries": 10,
      "expect": {
        "index_valid:pgbench_accounts_pkey": "==1"
      }
    },
    {
      "sql": "ALTER TABLE pgbench_accounts VALIDATE CONSTRAINT pgbench_accounts_abalance_range",
      "retries": 10,
      "expect": {
        "index_valid:pgbench_accounts_pkey": "==1"
      }
    },
    {
      "sql": "ALTER TABLE pgbench_history ADD CONSTRAINT pgbench_history_aid_fkey FOREIGN KEY (aid) REFERENCES pgbench_accounts (aid) NOT VALID",
      "retries": 10,
      "expect": {
        "table_exists:pgbench_history": "==1"
      }
    },
    {
      "sql": "ALTER TABLE pgbench_history VALIDATE CONSTRAINT pgbench_history_aid_fkey",
      "retries": 10,
      "expect": {
        "table_exists:pgbench_history": "==1",
        "table_exists:pgbench_accounts": "==1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "p99_rel",
      "tps",
      "max_lock_wait_s",
      "io_util_pct",
      "oom_kills",
      "shape_mean_ms:q18",
      "shape_mean_ms:q04"
    ],
    "thresholds": {
      "p99_rel": "<1.2x",
      "max_lock_wait_s": "<1",
      "io_util_pct": "<90",
      "oom_kills": "==0",
      "shape_mean_ms:q18": "<1.2x",
      "shape_mean_ms:q04": "<1.2x"
    }
  },
  "rollback": [
    "SET lock_timeout = '200ms'",
    "ALTER TABLE pgbench_history DROP CONSTRAINT IF EXISTS pgbench_history_aid_fkey",
    "ALTER TABLE pgbench_accounts DROP CONSTRAINT IF EXISTS pgbench_accounts_abalance_range"
  ],
  "postconditions": [
    "pg_constraint contains pgbench_accounts_abalance_range (contype c) with convalidated = true",
    "pg_constraint contains pgbench_history_aid_fkey (contype f, referencing pgbench_history.aid, referenced pgbench_accounts.aid) with convalidated = true",
    "No row in pgbench_accounts or pgbench_history was inserted, updated or deleted by the CR",
    "Standby is streaming with replay lag below 30 s, and both constraints are present on the standby",
    "Autovacuum is enabled and no constraint or deferred trigger was created beyond the two named ones"
  ],
  "risk": {
    "class_guess": "Online DDL: two constraint adds, each NOT VALID and then VALIDATE CONSTRAINT. The blocking lock phases are short, bounded by lock_timeout and retries. No data changes; reversible by dropping both constraints.",
    "blast_radius": "pgbench_accounts and pgbench_history on the primary. The ADD CHECK takes ACCESS EXCLUSIVE briefly on the primary; the standby replays that lock, so a standby query on pgbench_accounts that runs longer than max_standby_streaming_delay could be cancelled. This cannot be seen on the clone. The ADD FK takes SHARE ROW EXCLUSIVE briefly on both tables, which blocks workload writes only while it waits, bounded by lock_timeout. VALIDATE CONSTRAINT scans the 1.5 GB pgbench_accounts table with SHARE UPDATE EXCLUSIVE: it does not block DML, but it may cancel a running autovacuum on pgbench_accounts (autovacuum restarts later) and adds read I/O for the scan duration. The workload updates abalance by small deltas, so a balance reaching one billion during the window is very unlikely, but if it happens the affected q18 update fails with a CHECK violation. Failure of any validation aborts the CR and triggers the rollback drops."
  }
}

-> results grid
-> results tables