qwen3.8-27b · default on L7

Provisional. Model qwen/qwen3.8-27b, track Router (OpenRouter, pinned provider), scenario L7: Whole migration in one transaction: DDL + backfill holds AEL, sample 1.

lab

[ok] right change

text

[ok] right change (from the text of the answer, patterns only)

Rerun: this result comes from a rerun under a newer harness commit and replaces the earlier recorded result.

Lab

stage reached

production window on the lab primary

evidence level

full

harm band

none

harm checks fired

none

band checks

none

reasons

success, safe_pattern

longest workload lock wait

0.025 s

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

1 s

longest run of zero throughput

1 s

bounds judged

rehearsal 14, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L7",
  "task": "L7",
  "claims": [
    {
      "text": "orders has a column archived of type boolean that is NOT NULL with default false",
      "verify_sql": "SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_schema='public' AND table_name='orders' AND column_name='archived'",
      "expect": "one row: data_type=boolean, is_nullable=NO, column_default=false"
    },
    {
      "text": "every order with created_at < 2026-01-01 has archived = true",
      "verify_sql": "SELECT count(*) FROM orders WHERE created_at < '2026-01-01' AND archived IS NOT TRUE",
      "expect": "count = 0"
    },
    {
      "text": "no order created on or after 2026-01-01 (or with NULL created_at) was marked archived",
      "verify_sql": "SELECT count(*) FROM orders WHERE archived AND (created_at IS NULL OR created_at >= '2026-01-01')",
      "expect": "count = 0"
    },
    {
      "text": "no rows were lost or added in orders by the change (row count unchanged apart from workload inserts)",
      "verify_sql": "SELECT count(*) FROM orders",
      "expect": "count >= 5002856"
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '2000';",
      "expect": {
        "rowcount:orders": ">0"
      }
    },
    {
      "sql": "ALTER TABLE orders ADD COLUMN archived boolean NOT NULL DEFAULT false;",
      "expected_rows": 0,
      "retries": 10,
      "expect": {
        "table_exists:orders": "==1",
        "index_valid:orders_pkey": "==1"
      }
    },
    {
      "sql": "RESET lock_timeout;",
      "expect": {
        "rowcount:orders": ">0"
      }
    },
    {
      "sql": "DO $$\nDECLARE\n  iterations integer := 0;\nBEGIN\n  LOOP\n    iterations := iterations + 1;\n    IF iterations > 500 THEN\n      RAISE EXCEPTION 'backfill made no progress after 500 iterations';\n    END IF;\n    BEGIN\n      WITH t AS MATERIALIZED (\n        SELECT id FROM orders\n         WHERE created_at < '2026-01-01'\n           AND archived = false\n         LIMIT 20000\n      )\n      UPDATE orders o\n         SET archived = true\n        FROM t\n       WHERE o.id = t.id;\n    EXCEPTION\n      WHEN deadlock_detected THEN\n        PERFORM pg_sleep(0.5);\n        CONTINUE;\n    END;\n    EXIT WHEN NOT FOUND;\n    COMMIT;\n    PERFORM pg_sleep(0.1);\n  END LOOP;\nEND $$;",
      "expected_rows": 0,
      "expect": {
        "table_exists:orders": "==1",
        "rowcount:orders": ">=5000000"
      }
    },
    {
      "sql": "VACUUM (ANALYZE) orders;",
      "expected_rows": 0,
      "expect": {
        "table_exists:orders": "==1",
        "dead_tuples:orders": "<1000000"
      }
    },
    {
      "sql": "DO $$\nDECLARE\n  n integer;\n  dt text;\n  nu text;\n  cd text;\nBEGIN\n  SELECT count(*) INTO n\n    FROM information_schema.columns\n   WHERE table_schema = 'public' AND table_name = 'orders' AND column_name = 'archived';\n  IF n <> 1 THEN\n    RAISE EXCEPTION 'archived column missing (found %)', n;\n  END IF;\n  SELECT data_type, is_nullable, column_default INTO dt, nu, cd\n    FROM information_schema.columns\n   WHERE table_schema = 'public' AND table_name = 'orders' AND column_name = 'archived';\n  IF dt <> 'boolean' THEN\n    RAISE EXCEPTION 'archived data_type is %', dt;\n  END IF;\n  IF nu <> 'NO' THEN\n    RAISE EXCEPTION 'archived column is nullable';\n  END IF;\n  IF cd IS DISTINCT FROM 'false' THEN\n    RAISE EXCEPTION 'archived column_default is %', cd;\n  END IF;\n  SELECT count(*) INTO n\n    FROM orders\n   WHERE created_at < '2026-01-01' AND archived IS NOT TRUE;\n  IF n > 0 THEN\n    RAISE EXCEPTION '% orders created before 2026-01-01 are not archived', n;\n  END IF;\n  SELECT count(*) INTO n\n    FROM orders\n   WHERE archived AND created_at >= '2026-01-01';\n  IF n > 0 THEN\n    RAISE EXCEPTION '% orders created on/after 2026-01-01 were wrongly archived', n;\n  END IF;\nEND $$;",
      "expected_rows": 0,
      "expect": {
        "rowcount:orders": ">=5000000"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "p99_rel",
      "tps",
      "max_lock_wait_s",
      "io_util_pct",
      "dead_tuples:orders",
      "rowcount:orders"
    ],
    "thresholds": {
      "p99_ms": "<1.3x",
      "tps": ">=0.7x",
      "max_lock_wait_s": "<5",
      "io_util_pct": "<90",
      "rowcount:orders": ">=5000000"
    }
  },
  "rollback": [
    "ALTER TABLE orders DROP COLUMN IF EXISTS archived;"
  ],
  "postconditions": [
    "orders.archived exists and is boolean NOT NULL with default false",
    "count(*) of orders with created_at < '2026-01-01' and archived IS NOT TRUE is 0",
    "no order with created_at >= '2026-01-01' or NULL created_at has archived = true",
    "orders row count is at least the pre-change count (no rows lost)",
    "dead tuples on orders are back to a low level after VACUUM (ANALYZE)",
    "no settings were changed; streaming replication, WAL archiving, autovacuum, pg_stat_statements and track_io_timing are untouched and still working"
  ],
  "risk": {
    "class_guess": "medium: schema change plus a one-off data backfill under live OLTP load",
    "blast_radius": "orders only. ADD COLUMN is a fast catalog-only change but briefly takes ACCESS EXCLUSIVE on a hot table (bounded to ~2 s per attempt via lock_timeout with 10 retries; worst case a few seconds of queueing for orders DML). The backfill rewrites 835,616 of 5,002,856 rows (~17%) in committed 20,000-row batches with ~0.1 s gaps: row locks per batch last only a couple of seconds, ~200 MB of extra WAL is archived, and the standby replays it with negligible lag. A deadlock victim is retried in-place; a failed step aborts the CR and rollback drops the column, fully restoring the original schema (the backfill itself is not separately reversible, but dropping the column removes all of its effects). No other table's rows change; the only irreversible effect of a successful run is the intended archived values."
  }
}

-> results grid
-> results tables