deepseek-v4-pro · default on L2

Provisional. Model deepseek/deepseek-v4-pro-0813, track Router (OpenRouter, pinned provider), scenario L2: CREATE INDEX vs CONCURRENTLY and invalid-index leftovers, sample 1.

lab

[inef] ineffective

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 10, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "L2",
  "task": "L2",
  "claims": [
    {
      "text": "The index is created on pgbench_accounts with columns (bid, abalance).",
      "verify_sql": "SELECT indexname, indexdef FROM pg_indexes WHERE tablename = 'pgbench_accounts' AND indexname = 'pgbench_accounts_bid_abalance_idx'",
      "expect": "indexname == pgbench_accounts_bid_abalance_idx, indexdef contains (bid, abalance)"
    },
    {
      "text": "The index is valid.",
      "verify_sql": "SELECT indisvalid FROM pg_index WHERE indexrelid = 'pgbench_accounts_bid_abalance_idx'::regclass",
      "expect": "indisvalid == true"
    }
  ],
  "steps": [
    {
      "sql": "SELECT count(*) FROM pg_indexes WHERE tablename = 'pgbench_accounts' AND indexname = 'pgbench_accounts_bid_abalance_idx'",
      "expected_rows": 1,
      "expect": {
        "rowcount:pgbench_accounts": ">=10000000"
      }
    },
    {
      "sql": "DO $$BEGIN IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'pgbench_accounts' AND indexname = 'pgbench_accounts_bid_abalance_idx') THEN RAISE EXCEPTION 'index pgbench_accounts_bid_abalance_idx already exists'; END IF; END$$",
      "expected_rows": 0,
      "expect": {
        "lockmanager_count": ">=0"
      }
    },
    {
      "sql": "CREATE INDEX CONCURRENTLY pgbench_accounts_bid_abalance_idx ON pgbench_accounts (bid, abalance)",
      "expected_rows": 10000035,
      "retries": 3,
      "expect": {
        "lockmanager_share": ">=0",
        "max_lock_wait_s": "<5",
        "index_valid:pgbench_accounts_bid_abalance_idx": "==1"
      }
    },
    {
      "sql": "SELECT indisvalid FROM pg_index WHERE indexrelid = 'pgbench_accounts_bid_abalance_idx'::regclass",
      "expected_rows": 1,
      "expect": {
        "index_valid:pgbench_accounts_bid_abalance_idx": "==1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod-subset-A",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "p99_rel",
      "tps",
      "max_lock_wait_s",
      "io_util_pct",
      "checkpoints_req_share"
    ],
    "thresholds": {
      "p99_rel": "<1.2x",
      "max_lock_wait_s": "<5",
      "io_util_pct": "<=90",
      "checkpoints_req_share": "<0.5"
    }
  },
  "rollback": [
    "DROP INDEX CONCURRENTLY IF EXISTS pgbench_accounts_bid_abalance_idx"
  ],
  "postconditions": [
    "SELECT indexname FROM pg_indexes WHERE tablename = 'pgbench_accounts' AND indexname = 'pgbench_accounts_bid_abalance_idx'"
  ],
  "risk": {
    "class_guess": "Online DDL with brief lock contention",
    "blast_radius": "Single table index creation; no data modification; potential contention on pgbench_accounts during index build"
  }
}

-> results grid
-> results tables