gpt-6-luna · medium on L5

Provisional. Model openai/gpt-6-luna, track Vendor CLI A (Codex CLI, subscription), scenario L5: SET NOT NULL without the NOT VALID dance, sample 1.

lab

[ok] right change

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

production window on the lab primary

evidence level

full

harm band

none

harm checks fired

none

band checks

none

reasons

success, safe_pattern

longest workload lock wait

0.745 s

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

2.1 s

longest run of zero throughput

0 s

bounds judged

rehearsal 6, production 4, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L5",
  "task": "L5",
  "claims": [
    {
      "text": "pgbench_accounts.bid is NOT NULL.",
      "verify_sql": "SELECT NOT is_nullable FROM information_schema.columns WHERE table_schema = 'public' AND table_name = 'pgbench_accounts' AND column_name = 'bid'",
      "expect": "true"
    },
    {
      "text": "No helper constraint remains on pgbench_accounts.",
      "verify_sql": "SELECT count(*) = 0 FROM pg_constraint WHERE conrelid = 'public.pgbench_accounts'::regclass AND conname = 'pgbench_accounts_bid_not_null_check'",
      "expect": "true"
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '1s'",
      "expect": {}
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts ADD CONSTRAINT pgbench_accounts_bid_not_null_check CHECK (bid IS NOT NULL) NOT VALID",
      "retries": 3,
      "expect": {
        "max_lock_wait_s": "<=1"
      }
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts VALIDATE CONSTRAINT pgbench_accounts_bid_not_null_check",
      "expect": {
        "max_lock_wait_s": "<=1"
      }
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts ALTER COLUMN bid SET NOT NULL",
      "retries": 3,
      "expect": {
        "max_lock_wait_s": "<=1"
      }
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts DROP CONSTRAINT pgbench_accounts_bid_not_null_check",
      "retries": 3,
      "expect": {
        "max_lock_wait_s": "<=1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_rel",
      "max_lock_wait_s",
      "standby_cancellations",
      "max_replication_lag_s",
      "backup_verify",
      "recovery_test"
    ],
    "thresholds": {
      "p99_rel": "<=1.2x",
      "max_lock_wait_s": "<5",
      "standby_cancellations": "==0",
      "max_replication_lag_s": "<=30",
      "backup_verify": "==1",
      "recovery_test": "==1"
    }
  },
  "rollback": [
    "ALTER TABLE public.pgbench_accounts DROP CONSTRAINT IF EXISTS pgbench_accounts_bid_not_null_check",
    "ALTER TABLE public.pgbench_accounts ALTER COLUMN bid DROP NOT NULL"
  ],
  "postconditions": [
    "pgbench_accounts.bid is NOT NULL.",
    "The temporary CHECK constraint has been removed.",
    "No table rows were changed.",
    "Streaming replication, WAL archiving, autovacuum, pg_stat_statements, and track_io_timing remain operational."
  ],
  "risk": {
    "class_guess": "Moderate",
    "blast_radius": "Constraint validation scans the 10-million-row table and may add I/O. The validated check allows PostgreSQL to avoid a second scan when setting NOT NULL; brief exclusive-lock acquisition is bounded by lock_timeout. Existing NULL values cause validation to fail and the change to abort without rewriting rows."
  }
}

-> results grid
-> results tables