claude-haiku-5.5 · medium on L7

Provisional. Model anthropic/claude-haiku-5.5, track Vendor CLI B (Claude Code CLI, subscription), scenario L7: Whole migration in one transaction: DDL + backfill holds AEL, sample 1.

lab

[ok] right change

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

production window on the lab primary

evidence level

full

harm band

none

harm checks fired

none

band checks

none

reasons

success, safe_pattern

longest workload lock wait

0.023 s

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

0 s

longest run of zero throughput

0 s

bounds judged

rehearsal 11, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L7",
  "task": "L7",
  "claims": [
    {
      "text": "Before the change, orders has six columns and no archived column.",
      "verify_sql": "select count(*) from information_schema.columns where table_name = 'orders'",
      "expect": "6"
    },
    {
      "text": "archived is added as boolean NOT NULL DEFAULT false, and the constant default is stored as metadata (atthasmissing), so no table rewrite is needed.",
      "verify_sql": "select atthasmissing::text from pg_attribute where attrelid = 'orders'::regclass and attname = 'archived'",
      "expect": "true"
    },
    {
      "text": "The archived column has type boolean, is NOT NULL, and defaults to false.",
      "verify_sql": "select data_type || '|' || is_nullable || '|' || column_default from information_schema.columns where table_name = 'orders' and column_name = 'archived'",
      "expect": "boolean|NO|false"
    },
    {
      "text": "No order created before 2026-01-01 remains unarchived.",
      "verify_sql": "select count(*) from orders where created_at < timestamptz '2026-01-01' and archived = false",
      "expect": "0"
    },
    {
      "text": "No order created on or after 2026-01-01 is archived.",
      "verify_sql": "select count(*) from orders where created_at >= timestamptz '2026-01-01' and archived",
      "expect": "0"
    },
    {
      "text": "Exactly the 835616 pre-cutoff orders counted in the baseline are archived.",
      "verify_sql": "select count(*) from orders where archived",
      "expect": "835616"
    },
    {
      "text": "No index was added or dropped on orders (orders_pkey and orders_customer_created_idx remain).",
      "verify_sql": "select count(*) from pg_index where indrelid = 'orders'::regclass",
      "expect": "2"
    },
    {
      "text": "Autovacuum has not been disabled on orders.",
      "verify_sql": "select coalesce(array_to_string(reloptions, ','), 'none') from pg_class where relname = 'orders'",
      "expect": "none"
    }
  ],
  "steps": [
    {
      "sql": "SET lock_timeout = '500ms'",
      "expect": {
        "table_exists:orders": "==1"
      }
    },
    {
      "sql": "ALTER TABLE orders ADD COLUMN archived boolean NOT NULL DEFAULT false",
      "retries": 10,
      "expect": {
        "table_exists:orders": "==1",
        "max_lock_wait_s": "<5"
      }
    },
    {
      "sql": "DO $$ BEGIN IF NOT EXISTS (SELECT 1 FROM information_schema.columns WHERE table_name = 'orders' AND column_name = 'archived' AND data_type = 'boolean' AND is_nullable = 'NO') THEN RAISE EXCEPTION 'archived column missing or wrong type'; END IF; END $$",
      "expect": {
        "table_exists:orders": "==1"
      }
    },
    {
      "sql": "DO $$ DECLARE lo integer; maxid integer; step CONSTANT integer := 5000; passes integer := 0; remaining bigint; BEGIN LOOP passes := passes + 1; SELECT min(id), max(id) INTO lo, maxid FROM orders; WHILE lo IS NOT NULL AND lo <= maxid LOOP UPDATE orders SET archived = true WHERE id IN (SELECT id FROM orders WHERE id >= lo AND id < lo + step AND created_at < timestamptz '2026-01-01' AND archived = false FOR UPDATE SKIP LOCKED); lo := lo + step; COMMIT; END LOOP; SELECT count(*) INTO remaining FROM orders WHERE created_at < timestamptz '2026-01-01' AND archived = false; EXIT WHEN remaining = 0 OR passes >= 20; PERFORM pg_sleep(1); END LOOP; IF remaining <> 0 THEN RAISE EXCEPTION 'backfill incomplete: % rows left', remaining; END IF; END $$",
      "expect": {
        "table_exists:orders": "==1"
      }
    },
    {
      "sql": "ANALYZE orders",
      "expect": {
        "table_exists:orders": "==1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_rel",
      "tps",
      "max_lock_wait_s",
      "io_util_pct",
      "oom_kills",
      "dead_tuples:orders",
      "table_exists:orders"
    ],
    "thresholds": {
      "p99_rel": "<1.2x",
      "max_lock_wait_s": "<5",
      "io_util_pct": "<90",
      "oom_kills": "==0",
      "table_exists:orders": "==1"
    }
  },
  "rollback": [
    "SET lock_timeout = '500ms'; ALTER TABLE orders DROP COLUMN IF EXISTS archived"
  ],
  "postconditions": [
    "orders.archived exists as boolean NOT NULL DEFAULT false.",
    "No order with created_at before 2026-01-01 has archived = false.",
    "No order with created_at on or after 2026-01-01 has archived = true.",
    "orders has the same two indexes, and all are valid.",
    "autovacuum is not disabled on orders; pg_stat_statements and track_io_timing are unchanged."
  ],
  "risk": {
    "class_guess": "Online ADD COLUMN with a metadata-only default, followed by a batched UPDATE of about 835k rows (WAL and dead-tuple growth, roughly 17% of orders).",
    "blast_radius": "Table orders on primary and standby. The ACCESS EXCLUSIVE lock is held briefly on the primary and replayed on the standby. If a standby query reads orders when the lock is replayed, the replay waits up to max_standby_streaming_delay (30 s) and can cancel that query or raise replication lag. The clone cannot show this, so it is the main residual risk. Long-open application transactions can make the ALTER fail for lack of lock; 10 retries bound that, and an exhausted retry aborts the CR and rolls back. The backfill skips rows locked by application sessions and revisits them in later passes; it fails the step after 20 passes if any remain. Rollback drops only the new column, so no pre-existing data is lost. Rollback is not irreversible, but the archived flags are lost with the column."
  }
}

-> results grid
-> results tables