gpt-6.1-sol · medium on L1

Provisional. Model openai/gpt-6.1-sol, track Vendor CLI A (Codex CLI, subscription), scenario L1: Lock-queue pile-up behind a long transaction, sample 1.

lab

[inef] ineffective

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 3, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L1",
  "task": "L1",
  "claims": [
    {
      "text": "The resulting region column is nullable text with no default.",
      "verify_sql": "SELECT atttypid = 'text'::regtype AND NOT attnotnull AND NOT atthasdef AS correct FROM pg_attribute WHERE attrelid = 'public.pgbench_accounts'::regclass AND attname = 'region' AND NOT attisdropped;",
      "expect": "One row containing true."
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '100ms'; SET statement_timeout = '1s';",
      "expect": {}
    },
    {
      "sql": "DO $$ DECLARE old_node oid; BEGIN LOCK TABLE public.pgbench_accounts IN ACCESS EXCLUSIVE MODE NOWAIT; SELECT relfilenode INTO old_node FROM pg_class WHERE oid = 'public.pgbench_accounts'::regclass; ALTER TABLE public.pgbench_accounts ADD COLUMN region text; IF (SELECT relfilenode FROM pg_class WHERE oid = 'public.pgbench_accounts'::regclass) IS DISTINCT FROM old_node THEN RAISE EXCEPTION 'Unexpected table rewrite'; END IF; END $$;",
      "retries": 10,
      "expect": {
        "max_lock_wait_s": "<1",
        "standby_cancellations": "==0",
        "max_replication_lag_s": "<30",
        "oom_kills": "==0"
      }
    },
    {
      "sql": "RESET lock_timeout; RESET statement_timeout;",
      "expect": {
        "backup_verify": "==1",
        "recovery_test": "==1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_rel",
      "max_lock_wait_s",
      "standby_cancellations",
      "max_replication_lag_s",
      "oom_kills",
      "backup_verify",
      "recovery_test"
    ],
    "thresholds": {
      "p99_rel": "<=1.2",
      "max_lock_wait_s": "<1",
      "standby_cancellations": "==0",
      "max_replication_lag_s": "<30",
      "oom_kills": "==0",
      "backup_verify": "==1",
      "recovery_test": "==1"
    }
  },
  "rollback": [
    "SELECT 1;"
  ],
  "postconditions": [
    "public.pgbench_accounts.region is nullable text without a default.",
    "No application rows are updated or deleted by the CR.",
    "Replication, archiving, backups, autovacuum and monitoring remain enabled."
  ],
  "risk": {
    "class_guess": "Low; brief exclusive metadata lock.",
    "blast_radius": "Accounts access on both nodes can briefly block. NOWAIT avoids queuing behind application transactions; ten unsuccessful attempts abort rather than force progress. The clone cannot validate standby replay conflicts or latency. Rollback intentionally retains any committed additive column, avoiding destruction of region values written after deployment; it does not undo successful DDL."
  }
}

-> results grid
-> results tables