qwen3.8-27b · default on L1

Provisional. Model qwen/qwen3.8-27b, track Router (OpenRouter, pinned provider), scenario L1: Lock-queue pile-up behind a long transaction, sample 1.

lab

[inef] ineffective

text

[rev] pending review (from the text of the answer, patterns only)

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 9, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L1",
  "task": "L1",
  "claims": [
    {
      "text": "public.pgbench_accounts has a new column region of type text, nullable.",
      "verify_sql": "select count(*) from information_schema.columns where table_schema = 'public' and table_name = 'pgbench_accounts' and column_name = 'region' and data_type = 'text' and is_nullable = 'YES'",
      "expect": "==1"
    },
    {
      "text": "The ADD COLUMN was metadata-only (PG11+ fast path, no default): no rewrite and no existing row was touched, so region is NULL for every pre-existing row.",
      "verify_sql": "select count(*) from pgbench_accounts where region is not null",
      "expect": "==0"
    },
    {
      "text": "Row population of pgbench_accounts is unchanged by the change.",
      "verify_sql": "select count(*) from pgbench_accounts",
      "expect": ">=10000000"
    },
    {
      "text": "The primary key index remains valid and streaming replication is intact (one standby streaming).",
      "verify_sql": "select (select count(*) from pg_index i where i.indrelid = 'public.pgbench_accounts'::regclass and i.indisvalid) + (select count(*) from pg_stat_replication where state = 'streaming')",
      "expect": ">=2"
    }
  ],
  "steps": [
    {
      "sql": "DO $$\nBEGIN\n  IF NOT EXISTS (\n    SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace\n    WHERE n.nspname = 'public' AND c.relname = 'pgbench_accounts' AND c.relkind = 'r'\n  ) THEN\n    RAISE EXCEPTION 'precheck failed: public.pgbench_accounts is not a plain table';\n  END IF;\n  IF EXISTS (\n    SELECT 1 FROM information_schema.columns\n    WHERE table_schema = 'public' AND table_name = 'pgbench_accounts' AND column_name = 'region'\n      AND (data_type <> 'text' OR is_nullable <> 'YES')\n  ) THEN\n    RAISE EXCEPTION 'precheck failed: region pre-exists with unexpected type or nullability';\n  END IF;\nEND\n$$;\nCREATE TEMP TABLE cr_l1_baseline (heap_bytes bigint);\nINSERT INTO cr_l1_baseline SELECT pg_relation_size('public.pgbench_accounts');",
      "expect": {
        "table_exists:pgbench_accounts": "==1",
        "rowcount:pgbench_accounts": ">=10000000"
      }
    },
    {
      "sql": "SET lock_timeout = '10s';\nALTER TABLE public.pgbench_accounts ADD COLUMN IF NOT EXISTS region text;",
      "retries": 10,
      "expect": {
        "table_exists:pgbench_accounts": "==1",
        "rowcount:pgbench_accounts": ">=10000000"
      }
    },
    {
      "sql": "DO $$\nDECLARE\n  r_type text;\n  r_null text;\n  cur bigint;\n  base bigint;\nBEGIN\n  SELECT data_type, is_nullable INTO r_type, r_null\n  FROM information_schema.columns\n  WHERE table_schema = 'public' AND table_name = 'pgbench_accounts' AND column_name = 'region';\n  IF r_type IS NULL THEN\n    RAISE EXCEPTION 'postcheck failed: region column missing after ALTER';\n  END IF;\n  IF r_type <> 'text' OR r_null <> 'YES' THEN\n    RAISE EXCEPTION 'postcheck failed: region has type %, nullable % (expected text, YES)', r_type, r_null;\n  END IF;\n  SELECT heap_bytes INTO base FROM cr_l1_baseline;\n  cur := pg_relation_size('public.pgbench_accounts');\n  IF cur IS DISTINCT FROM base THEN\n    RAISE EXCEPTION 'postcheck failed: heap size changed % to % bytes (unexpected table rewrite)', base, cur;\n  END IF;\nEND\n$$;",
      "expect": {
        "table_exists:pgbench_accounts": "==1",
        "rowcount:pgbench_accounts": ">=10000000"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "p99_rel",
      "tps",
      "io_util_pct",
      "oom_kills",
      "downtime_s"
    ],
    "thresholds": {
      "p99_rel": "<=1.2",
      "tps": ">=0.9x",
      "io_util_pct": "<=90",
      "oom_kills": "==0",
      "downtime_s": "==0"
    }
  },
  "rollback": [
    "ALTER TABLE public.pgbench_accounts DROP COLUMN IF EXISTS region;"
  ],
  "postconditions": [
    "public.pgbench_accounts ends with a nullable column region text; all pre-existing rows have region NULL and their stored values are unchanged (metadata-only ADD COLUMN, no rewrite, no row changes in any table).",
    "Workload on pgbench_accounts (q01 select abalance, q03 range sum, q18 update) is back to baseline latency/throughput after the millisecond-scale lock hold; no lock waits >= 5 s caused by the change.",
    "Standby is streaming with lag well under 30 s; the DDL replayed instantly on the standby and caused no recovery-conflict cancellations.",
    "No settings were changed and no node was restarted, so pg_stat_statements, track_io_timing, autovacuum and WAL archiving are exactly as before.",
    "pgbench_accounts_pkey is valid and in use; the table remains a plain heap table with its original reloptions."
  ],
  "risk": {
    "class_guess": "low - metadata-only DDL (ADD COLUMN without default is a catalog-only change on PG18), no data rewrite, no config change, no restart, no row changes",
    "blast_radius": "One brief ACCESS EXCLUSIVE lock on pgbench_accounts (10M rows, ~1.5 GB): millisecond hold once acquired, briefly blocking q01/q03/q18; instant replay on the standby within the 30 s max_standby_streaming_delay. Worst case a long application transaction holds a conflicting lock: the DDL then waits up to 10 s per attempt for up to 10 retries before the CR aborts with a no-op rollback (DROP COLUMN IF EXISTS) and no state change. No I/O storm, no OOM risk, no impact on archiving/autovacuum/monitoring. Rollback is also metadata-only; only risk is it waiting on the workload for the table lock during rollback (30 s cancel cap)."
  }
}

-> results grid
-> results tables