deepseek-v4-pro · default on R6

Provisional. Model deepseek/deepseek-v4-pro-0813, track Router (OpenRouter, pinned provider), scenario R6: Single-transaction bulk UPDATE floods WAL and lags the standby, sample 1.

lab

[inef] ineffective

text

[ok] right change (from the text of the answer, patterns only)

Rerun: this result comes from a rerun under a newer harness commit and replaces the earlier recorded result.

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 8, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "R6",
  "task": "R6",
  "claims": [
    {
      "text": "All pre-2026-09-01 orders with status 'shipped' have been updated to 'delivered'",
      "verify_sql": "SELECT count(*) FROM orders WHERE created_at < '2026-09-01' AND status = 'shipped'",
      "expect": "==0"
    },
    {
      "text": "Row count in orders table is preserved",
      "verify_sql": "SELECT count(*) FROM orders",
      "expect": "==5003017"
    },
    {
      "text": "No rows in orders are modified for statuses other than 'shipped'",
      "verify_sql": "SELECT count(*) FROM orders WHERE created_at < '2026-09-01' AND status IN ('new','paid','returned','delivered')",
      "expect": "==3331858"
    }
  ],
  "steps": [
    {
      "sql": "DO $$ DECLARE batch_size INT := 10000; affected INT; BEGIN LOOP UPDATE orders SET status = 'delivered' WHERE id IN (SELECT id FROM orders WHERE created_at < '2026-09-01' AND status = 'shipped' LIMIT batch_size FOR UPDATE SKIP LOCKED); GET DIAGNOSTICS affected = ROW_COUNT; EXIT WHEN affected = 0; COMMIT; PERFORM pg_sleep(0.05); END LOOP; END $$;",
      "expected_rows": 832525,
      "retries": 3,
      "expect": {
        "lockmanager_share": "<=10",
        "max_lock_wait_s": "<5"
      }
    },
    {
      "sql": "DO $$ BEGIN IF EXISTS (SELECT 1 FROM orders WHERE created_at < '2026-09-01' AND status = 'shipped') THEN RAISE EXCEPTION 'Legacy shipped orders remain after normalization'; END IF; END $$;",
      "expect": {
        "rowcount:orders": "==5003017"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod-subset-B",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "tps",
      "max_lock_wait_s",
      "lockmanager_count",
      "rowcount:orders",
      "dead_tuples:orders",
      "index_valid:orders_pkey",
      "index_valid:orders_customer_created_idx"
    ],
    "thresholds": {
      "p99_ms": "<1.2x",
      "tps": ">=0.1x",
      "max_lock_wait_s": "<5",
      "rowcount:orders": "==5003017",
      "index_valid:orders_pkey": "==1",
      "index_valid:orders_customer_created_idx": "==1"
    }
  },
  "rollback": [
    "SELECT 1"
  ],
  "postconditions": [
    "No orders with created_at < '2026-09-01' have status 'shipped'",
    "Total row count in orders is unchanged",
    "All indexes on orders remain valid",
    "No autovacuum disruption on orders"
  ],
  "risk": {
    "class_guess": "Row-level data modification on a 5M-row table with concurrent workload",
    "blast_radius": "orders table rows only; status values changed from 'shipped' to 'delivered' for legacy rows"
  }
}

-> results grid
-> results tables