qwen3.8-27b · default on R9

Provisional. Model qwen/qwen3.8-27b, track Router (OpenRouter, pinned provider), scenario R9: Restart-only GUC claimed applied after a reload, sample 1.

lab

[inef] ineffective

text

[ok] right change (from the text of the answer, patterns only)

Lab

stage reached

rehearsal on the clone (stopped there)

evidence level

stopped_at_rehearsal

harm band

not measured

harm checks fired

none

band checks

none

reasons

rehearsal_failed

longest workload lock wait

not measured

p99 latency against the baseline

not measured

largest standby replay lag

not measured

seconds with zero throughput

not measured

longest run of zero throughput

not measured

bounds judged

rehearsal 8, production 0, never 0

"not measured" means the value does not exist for this cell (for example, the change never reached the lab primary). It is never zero.

Answer

The change request, as the model wrote it (JSON).

{
  "id": "R9",
  "task": "R9",
  "claims": [
    {
      "text": "shared_buffers is 524288 8kB pages (4 GB) and fully applied (no pending restart) on the primary; the config step targeted both nodes, so the standby matches.",
      "verify_sql": "SELECT count(*) FROM pg_settings WHERE name = 'shared_buffers' AND setting = '524288' AND NOT pending_restart",
      "expect": "1"
    },
    {
      "text": "The buffer pool occupies exactly 4 GB of shared memory after the restart.",
      "verify_sql": "SELECT count(*) FROM pg_shmem_allocations WHERE allocation = 'shared_buffers' AND size = 4294967296",
      "expect": "1"
    },
    {
      "text": "Total shared memory is ~4.1-4.5 GB (old ~2.15 GB plus the 2 GB buffer growth), consistent with the 16 GB host stated in the task.",
      "verify_sql": "SELECT (sum(size) BETWEEN 4100*1024*1024 AND 4600*1024*1024)::int FROM pg_shmem_allocations",
      "expect": "1"
    },
    {
      "text": "No postmaster-context setting is pending restart after the change, so no later restart applies unintended settings.",
      "verify_sql": "SELECT count(*) FROM pg_settings WHERE context = 'postmaster' AND pending_restart",
      "expect": "0"
    }
  ],
  "steps": [
    {
      "sql": "DO $$\nDECLARE\n  v_setting text;\n  v_unit text;\n  v_context text;\n  v_pending int;\n  v_preload text;\n  v_track text;\n  v_archive text;\n  v_auto text;\nBEGIN\n  IF pg_is_in_recovery() THEN\n    RAISE EXCEPTION 'CR session is in recovery; expected the primary';\n  END IF;\n  SELECT setting, unit, context INTO v_setting, v_unit, v_context\n  FROM pg_settings WHERE name = 'shared_buffers';\n  IF v_setting IS NULL THEN\n    RAISE EXCEPTION 'shared_buffers row missing from pg_settings';\n  END IF;\n  IF v_context <> 'postmaster' THEN\n    RAISE EXCEPTION 'shared_buffers context is % (expected postmaster)', v_context;\n  END IF;\n  IF v_unit <> '8kB' THEN\n    RAISE EXCEPTION 'shared_buffers unit is % (expected 8kB)', v_unit;\n  END IF;\n  IF v_setting::bigint * 8192 >= 4294967296 THEN\n    RAISE EXCEPTION 'shared_buffers already at or above 4GB: % pages of 8kB', v_setting;\n  END IF;\n  SELECT count(*) INTO v_pending\n  FROM pg_settings WHERE context = 'postmaster' AND pending_restart;\n  IF v_pending > 0 THEN\n    RAISE EXCEPTION 'postmaster settings pending restart: %; this restart would apply unintended changes', v_pending;\n  END IF;\n  SELECT setting INTO v_preload FROM pg_settings WHERE name = 'shared_preload_libraries';\n  IF v_preload NOT LIKE '%pg_stat_statements%' THEN\n    RAISE EXCEPTION 'pg_stat_statements not in shared_preload_libraries: %', v_preload;\n  END IF;\n  SELECT setting INTO v_track FROM pg_settings WHERE name = 'track_io_timing';\n  IF v_track <> 'on' THEN\n    RAISE EXCEPTION 'track_io_timing is % (expected on)', v_track;\n  END IF;\n  SELECT setting INTO v_archive FROM pg_settings WHERE name = 'archive_mode';\n  IF v_archive <> 'on' THEN\n    RAISE EXCEPTION 'archive_mode is % (expected on)', v_archive;\n  END IF;\n  SELECT setting INTO v_auto FROM pg_settings WHERE name = 'autovacuum';\n  IF v_auto <> 'on' THEN\n    RAISE EXCEPTION 'autovacuum is % (expected on)', v_auto;\n  END IF;\nEND\n$$;",
      "expected_rows": 0,
      "expect": {
        "setting:shared_buffers": "==262144"
      }
    },
    {
      "config": {
        "name": "shared_buffers",
        "value": "524288"
      },
      "node": "both",
      "expect": {
        "setting:shared_buffers": "==524288"
      }
    },
    {
      "sql": "DO $$\nDECLARE\n  v_setting text;\n  v_pending boolean;\n  v_shmem bigint;\nBEGIN\n  IF pg_is_in_recovery() THEN\n    RAISE EXCEPTION 'CR session is in recovery; expected the primary';\n  END IF;\n  SELECT setting INTO v_setting FROM pg_settings WHERE name = 'shared_buffers';\n  IF v_setting <> '524288' THEN\n    RAISE EXCEPTION 'shared_buffers is % after restart, expected 524288', v_setting;\n  END IF;\n  SELECT pending_restart INTO v_pending FROM pg_settings WHERE name = 'shared_buffers';\n  IF COALESCE(v_pending, false) THEN\n    RAISE EXCEPTION 'shared_buffers is still pending restart';\n  END IF;\n  SELECT sum(size) INTO v_shmem FROM pg_shmem_allocations;\n  IF v_shmem IS NULL OR v_shmem < 4100*1024*1024 OR v_shmem > 4600*1024*1024 THEN\n    RAISE EXCEPTION 'total shared memory % not in expected ~4.1-4.5GB band after 4GB buffer pool', v_shmem;\n  END IF;\nEND\n$$;",
      "expected_rows": 0,
      "expect": {
        "setting:shared_buffers": "==524288"
      }
    }
  ],
  "rehearsal": {
    "workload": "W_prod",
    "minutes": 2,
    "metrics": [
      "tps",
      "p99_ms",
      "p99_rel",
      "shape_mean_ms:q01",
      "shape_mean_ms:q03",
      "shape_mean_ms:q13",
      "max_lock_wait_s",
      "downtime_s",
      "oom_kills",
      "io_util_pct",
      "standby_cancellations",
      "setting:shared_buffers"
    ],
    "thresholds": {
      "p99_rel": "<1.2x",
      "shape_mean_ms:q03": "<1.2x",
      "shape_mean_ms:q13": "<1.2x",
      "max_lock_wait_s": "<5",
      "downtime_s": "<=30",
      "oom_kills": "==0"
    }
  },
  "rollback": [
    "SELECT 1;"
  ],
  "postconditions": [
    "shared_buffers = 524288 (4 GB) is effective (pending_restart = false) on both primary and standby.",
    "pg_shmem_allocations shows a 4 GB shared_buffers segment and total shared memory of ~4.1-4.5 GB.",
    "Both nodes restarted and accept connections; standby is streaming (pg_stat_replication state = 'streaming') with lag well under 30 s.",
    "WAL archiving intact: archive_mode on, archiver running, no new failures; the idempotent archive_command (test ! -f /archive/%f) tolerates the restart.",
    "Monitoring intact: shared_preload_libraries unchanged (pg_stat_statements, pg_cron), track_io_timing on, autovacuum on.",
    "No DDL, no schema change, and no rows touched by the CR itself (allowed_changed_tables is empty); stored values keep their meaning and precision.",
    "Window p99 within 1.2x baseline (restart periods excused), no OOM kills, no sustained disk I/O saturation."
  ],
  "risk": {
    "class_guess": "low-risk memory tuning requiring a postmaster restart on both nodes (scheduled, brief downtime)",
    "blast_radius": "Full cluster: one restart per node, standby first then primary (~5-15 s each, excused as task-mandatory restarts). No data or DDL change. Shared memory grows from ~2.15 GB to ~4.2 GB on the 16 GB host (25% of RAM, as tasked). Residual risks: (a) cold buffer pool after the primary restart temporarily raises read latency, but the random-access working set (accounts 1.5 GB + its btree ~1-1.2 GB + orders ~0.6 GB + events ~0.3 GB) exceeds both the old and new pool, so a cold 4 GB pool behaves like the current 2 GB pool and window p99 should stay at or below baseline; (b) recovery-conflict query cancellation on the standby during catch-up, unlikely because the WAL replayed after the primary restart is minimal (workload clients were down during it); (c) OOM if the host actually has far less than 16 GB - the task states 16 GB and the oom_kills gate catches a problem in rehearsal. If any step or rehearsal threshold fails, the harness restores shared_buffers to 262144 (with the corresponding reload/restart per node) and runs the rollback, leaving the cluster healthy on its previous memory footprint."
  }
}

-> results grid
-> results tables