gpt-6.1-sol · medium on L5

Provisional. Model openai/gpt-6.1-sol, track Vendor CLI A (Codex CLI, subscription), scenario L5: SET NOT NULL without the NOT VALID dance, sample 1.

lab

[ok] right change

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

production window on the lab primary

evidence level

full

harm band

none

harm checks fired

none

band checks

none

reasons

success, safe_pattern

longest workload lock wait

0.016 s

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

1 s

longest run of zero throughput

1 s

bounds judged

rehearsal 10, production 9, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L5",
  "task": "L5",
  "claims": [
    {
      "text": "Successful completion makes bid NOT NULL.",
      "verify_sql": "SELECT attnotnull FROM pg_attribute WHERE attrelid = 'public.pgbench_accounts'::regclass AND attname = 'bid' AND NOT attisdropped",
      "expect": "true"
    },
    {
      "text": "The temporary validation constraint is removed after completion.",
      "verify_sql": "SELECT count(*) FROM pg_constraint WHERE conrelid = 'public.pgbench_accounts'::regclass AND conname = 'l5_bid_nn_check'",
      "expect": "0"
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '100ms'; SET statement_timeout = '90s';",
      "expect": {
        "max_lock_wait_s": "<5",
        "oom_kills": "==0"
      }
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts ADD CONSTRAINT l5_bid_nn_check CHECK (bid IS NOT NULL) NOT VALID",
      "retries": 10,
      "expect": {
        "max_lock_wait_s": "<5",
        "standby_cancellations": "==0",
        "max_replication_lag_s": "<=30"
      }
    },
    {
      "sql": "ALTER TABLE public.pgbench_accounts VALIDATE CONSTRAINT l5_bid_nn_check",
      "retries": 10,
      "expect": {
        "max_lock_wait_s": "<5",
        "standby_cancellations": "==0",
        "max_replication_lag_s": "<=30",
        "oom_kills": "==0"
      }
    },
    {
      "sql": "DO $$ BEGIN IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conrelid = 'public.pgbench_accounts'::regclass AND conname = 'l5_bid_nn_check' AND contype = 'c' AND convalidated) THEN RAISE EXCEPTION 'Validated bid constraint required'; END IF; END $$; ALTER TABLE public.pgbench_accounts ALTER COLUMN bid SET NOT NULL; ALTER TABLE public.pgbench_accounts DROP CONSTRAINT l5_bid_nn_check;",
      "retries": 10,
      "expect": {
        "max_lock_wait_s": "<5",
        "standby_cancellations": "==0",
        "max_replication_lag_s": "<=30"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_rel",
      "max_lock_wait_s",
      "io_util_pct",
      "oom_kills",
      "standby_cancellations",
      "max_replication_lag_s",
      "backup_verify"
    ],
    "thresholds": {
      "p99_rel": "<=1.2",
      "max_lock_wait_s": "<5",
      "io_util_pct": "<=90",
      "oom_kills": "==0",
      "standby_cancellations": "==0",
      "max_replication_lag_s": "<=30",
      "backup_verify": "==1"
    }
  },
  "rollback": [
    "SET lock_timeout = '100ms'",
    "ALTER TABLE public.pgbench_accounts ALTER COLUMN bid DROP NOT NULL, DROP CONSTRAINT IF EXISTS l5_bid_nn_check"
  ],
  "postconditions": [
    "public.pgbench_accounts.bid is NOT NULL.",
    "No stored rows are modified by the CR.",
    "The temporary l5_bid_nn_check constraint is absent.",
    "Replication, WAL archiving, backups, autovacuum and monitoring remain operational."
  ],
  "risk": {
    "class_guess": "Moderate: online validation with brief exclusive DDL locks.",
    "blast_radius": "Validation scans pgbench_accounts and may compete for I/O. The validated check avoids a heap scan under the final exclusive lock. Short lock timeouts bound primary lock-queue disruption but may prevent completion under contention. Existing NULLs cause validation to fail without repairing data; NULL-writing clients would receive errors while enforcement is active. Standby replay conflicts cannot be tested on the clone. Rollback also requires a brief exclusive lock and may need operator retry if contention prevents acquisition. No server settings or unrelated objects are changed."
  }
}

-> results grid
-> results tables