claude-opus-5.5 · high on L2

Provisional. Model anthropic/claude-opus-5.5, track Vendor CLI B (Claude Code CLI, subscription), scenario L2: CREATE INDEX vs CONCURRENTLY and invalid-index leftovers, sample 1.

lab

[ok] right change

text

[ok] right change (from the text of the answer, patterns only)

Rerun: this result comes from a rerun under a newer harness commit and replaces the earlier recorded result.

Lab

stage reached

production window on the lab primary

evidence level

full

harm band

none

harm checks fired

none

band checks

none

reasons

success, safe_pattern

longest workload lock wait

0.048 s

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

0 s

longest run of zero throughput

0 s

bounds judged

rehearsal 14, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "CR-L2-bid-abalance-idx",
  "task": "L2",
  "claims": [
    {
      "text": "pgbench_accounts currently has only its primary key index; no index named pgbench_accounts_bid_abalance_idx exists (valid or invalid).",
      "verify_sql": "select count(*) from pg_class where relname = 'pgbench_accounts_bid_abalance_idx' and relnamespace = 'public'::regnamespace",
      "expect": "0"
    },
    {
      "text": "pgbench_accounts heap is about 1.3 GB with about 10M rows, so a concurrent build (two heap scans plus a sort) should finish well within the 600 s step limit.",
      "verify_sql": "select pg_size_pretty(pg_relation_size('public.pgbench_accounts')), reltuples::bigint from pg_class where oid = 'public.pgbench_accounts'::regclass",
      "expect": "about 1282 MB, about 10000000 rows"
    },
    {
      "text": "lock_timeout and statement_timeout are 0 by default, so CIC's waits for older transactions will not cancel the build.",
      "verify_sql": "select name, setting from pg_settings where name in ('lock_timeout','statement_timeout')",
      "expect": "both 0"
    },
    {
      "text": "Workload DML on pgbench_accounts (q01, q02, q03) takes AccessShare or RowExclusive locks, which do not conflict with the ShareUpdateExclusive lock held by CREATE INDEX CONCURRENTLY.",
      "verify_sql": "select mode, granted, count(*) from pg_locks where relation = 'public.pgbench_accounts'::regclass group by 1,2",
      "expect": "only AccessShareLock / RowExclusiveLock from the workload"
    }
  ],
  "steps": [
    {
      "sql": "DO $$ BEGIN IF to_regclass('public.pgbench_accounts_bid_abalance_idx') IS NOT NULL THEN RAISE EXCEPTION 'relation public.pgbench_accounts_bid_abalance_idx already exists; aborting'; END IF; IF to_regclass('public.pgbench_accounts') IS NULL THEN RAISE EXCEPTION 'public.pgbench_accounts not found'; END IF; END $$",
      "expect": {
        "max_lock_wait_s": "<1"
      }
    },
    {
      "sql": "SET maintenance_work_mem = '512MB'",
      "expect": {
        "oom_kills": "==0"
      }
    },
    {
      "sql": "SET lock_timeout = 0",
      "expect": {
        "max_lock_wait_s": "<1"
      }
    },
    {
      "sql": "CREATE INDEX CONCURRENTLY pgbench_accounts_bid_abalance_idx ON public.pgbench_accounts USING btree (bid, abalance)",
      "expect": {
        "index_valid:pgbench_accounts_bid_abalance_idx": "==1",
        "max_lock_wait_s": "<1",
        "oom_kills": "==0"
      }
    },
    {
      "sql": "RESET maintenance_work_mem",
      "expect": {
        "index_valid:pgbench_accounts_bid_abalance_idx": "==1"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "p99_ms",
      "p99_rel",
      "tps",
      "max_lock_wait_s",
      "io_util_pct",
      "oom_kills",
      "index_valid:pgbench_accounts_bid_abalance_idx",
      "rowcount:pgbench_accounts",
      "shape_mean_ms:q01",
      "shape_mean_ms:q02",
      "shape_mean_ms:q03",
      "checkpoints_window_count"
    ],
    "thresholds": {
      "index_valid:pgbench_accounts_bid_abalance_idx": "==1",
      "p99_ms": "<1.2x",
      "tps": ">0.9x",
      "max_lock_wait_s": "<1",
      "oom_kills": "==0",
      "shape_mean_ms:q02": "<1.5x",
      "rowcount:pgbench_accounts": ">=10000000"
    }
  },
  "rollback": [
    "DROP INDEX CONCURRENTLY IF EXISTS public.pgbench_accounts_bid_abalance_idx"
  ],
  "postconditions": [
    "public.pgbench_accounts_bid_abalance_idx exists, defined as btree (bid, abalance), with indisvalid = true and indisready = true",
    "No invalid indexes on pgbench_accounts",
    "No table data changed; pgbench_accounts row count unchanged apart from normal workload activity",
    "Standby still streaming with replay lag under 30 s; autovacuum, pg_stat_statements, track_io_timing and WAL archiving unchanged",
    "maintenance_work_mem change was session-only; no server settings changed"
  ],
  "risk": {
    "class_guess": "low: online index build (CREATE INDEX CONCURRENTLY), additive, reversible with DROP INDEX CONCURRENTLY",
    "blast_radius": "Only pgbench_accounts. No workload lock conflicts are expected. Main costs: (1) I/O from two scans of a 1.3 GB heap plus a sort, which can briefly raise latency or io_util on the primary; (2) a few hundred MB of WAL, which adds replay I/O on the standby and may trigger requested checkpoints (max_wal_size 1 GB); recovery conflicts are not expected. (3) The build waits for open transactions (W_prod holds some for several seconds), which only makes it take longer. If the build fails, it leaves an invalid index; the rollback drops it concurrently. The new index adds a small write overhead to UPDATEs of abalance (q02): those updates are no longer HOT-eligible because abalance is now indexed. That slows q02 a little and adds index bloat over time; this is inherent to the requested index."
  }
}

-> results grid
-> results tables