SQL Server Disk Subsystem Corruption: msdb.suspect_pages, DBCC CHECKDB Diagnostics & Page Restore Runbook
Diagnosing hardware/SAN subsystem write failures and checksum mismatches, inspecting msdb.suspect_pages, DBCC CHECKDB recovery and PAGE_VERIFY CHECKSUM policy.
Operational Verification & Reliability Metadata
The error string is a symptom, not the root cause. Do not restart the server blindly. Run the diagnostic queries below to pinpoint the actual failure mode.
▸Symptom
SQL Server reports critical Severity 24 consistency errors:
Msg 823, Level 24, State 2, Line 1
I/O error 23(Data error (cyclic redundancy check).) detected during read at offset in file 'D:\Data\Prod.mdf'.
Msg 824, Level 24, State 2, Line 1
SQL Server detected a logical consistency-based I/O error: torn page (expected checksum: 0x8a1b2c3d; found 0x00000000).▸What You See in Production
Specific 8KB data pages become unreadable. Affected tables fail queries immediately. Entries populate msdb.dbo.suspect_pages.
▸Root Cause Matrix
| Probability | Root Cause | Description |
|---|---|---|
| %70 | SAN Subsystem / Controller Failure | Write caching failure or torn pages where only partial sector flushes occurred. |
| %20 | Firmware / Driver Faults | Disk controller reports write completion without committing sectors. |
| %10 | Missing PAGE_VERIFY CHECKSUM | Legacy database missing proactive hardware validation. |
▸EMERGENCY RUNBOOK (0-5 min)
-- 1. Query suspect pages catalog
SELECT database_id, file_id, page_id, event_type FROM msdb.dbo.suspect_pages;
-- 2. Execute consistency check
DBCC CHECKDB ('Prod') WITH NO_INFOMSGS, ALL_ERRORMSGS;
-- 3. Perform surgical Page Restore from verified backup
-- RESTORE DATABASE Prod PAGE = '1:2450' FROM DISK = 'D:\Backups\Prod_Full.bak' WITH NORECOVERY;
-- RESTORE LOG Prod FROM DISK = 'D:\Backups\Prod_Log.trn' WITH RECOVERY;▸PERMANENT BLUEPRINT
- 1Enforce Checksums:
ALTER DATABASE Prod SET PAGE_VERIFY CHECKSUM; - 2Weekly Integrity Audits: Schedule regular
DBCC CHECKDBmaintenance jobs. - 3Backup Checksums: Take backups
WITH CHECKSUM.
▸Verification
SELECT name, page_verify_option_desc FROM sys.databases WHERE name = 'Prod';Other Database Error Triage Playbooks
Other battle-tested diagnostic guides for high-severity database errors.
SQL Server Error 18456: Root Cause by State Code Matrix
In Error 18456, the State code reveals the actual root cause. Specific fixes for State 5, 8, 11, 18, 38, and 40, with ERRORLOG extraction scripts.
SQL Server Error 1205 Deadlock: Graph Extraction and Prevention
Extracting the deadlock graph from the system_health Extended Events ring buffer, understanding victim election, and architectural resolution.
SQL Server Pool Timeout: Connection Pool Exhaustion Fix
Difference between connection pool timeout and query command timeout, detecting connection leaks, and .NET DbContext architectural fixes.