dataiotech Labs
dbaguidedbaguideDBAdataxtoolsdataopsphere
dbaguide.com
Field Experience & Post-Mortems

Field Experience & Post-Mortems

Real-world production crises, incident root causes, community events, and practitioner field notes.

Cases / Post-Mortem

Real production incident post-mortems, root cause analyses, and preventative playbooks.

3 Case Studies

Field Notes

Short practitioner field tips and insights gained on the front lines.

4 Field Notes

Featured Post-Mortem Incident Reports

All Cases →
postgresql45 dakika kesinti

PostgreSQL Emergency Recovery During TXID Wraparound

The Problem Encountered:

A high-throughput 12TB e-commerce database suddenly rejected all client connections with "database is not accepting commands to avoid wraparound data loss".

Key Takeaways:

Continuously monitor `datfrozenxid` relative age via Prometheus; raise critical pages at 70% threshold before PostgreSQL forces emergency shutdown.

mysql20 dakika kısmi sepet çökmesi

MySQL InnoDB Gap Lock Deadlock Storm under Flash Sale

The Problem Encountered:

During flash sale traffic, cart checkout services flooded application logs with InnoDB deadlock exceptions.

Key Takeaways:

For high-concurrency transactional OLTP systems, READ COMMITTED with binlog_format=ROW eliminates the majority of phantom gap deadlocks.

sqlserver30 dakika %100 CPU darboğazı

SQL Server TempDB Allocation Bottleneck (PAGELATCH_UP)

The Problem Encountered:

A 64-core enterprise ERP SQL Server instance spiked to 100% CPU with hundreds of threads blocked on PAGELATCH_UP waits in tempdb.

Key Takeaways:

Modern multi-core SQL Server installations must allocate 8+ identical TempDB data files with fixed MB autogrowth.

Field Notes

All Notes →
postgresqldataiotech Labs • 2026-08-28

Never Manually Delete Files from pg_wal!

Running `rm -rf pg_wal/*` during a disk full panic corrupts the transaction log permanently.

💡 Pro Tip: Use `pg_archivecleanup` or drop unused temporary tables to allow a graceful checkpoint.
mysqldataiotech Labs • 2026-08-15

Use Percona pt-online-schema-change for 100M+ Row Tables

Large ALTER TABLE statements can cause Metadata Lock (MDL) queues that stall web traffic.

💡 Pro Tip: Use shadow-table based online schema migration tools to migrate large datasets with negligible lock overhead.
sqlserverdataiotech Labs • 2026-09-20

Never Leave SQL Server 'Max Server Memory' at Default Unlimited!

The default 2.1TB memory ceiling causes SQL Server to starve the Windows OS, triggering severe thread paging.

💡 Pro Tip: Reserve 4GB-8GB (10-15% total RAM) for the operating system and configure explicit max server memory.
sqlserverdataiotech Labs • 2026-09-25

Raise Cost Threshold for Parallelism from Default (5) to 40-50

Default cost threshold of 5 triggers premature query parallelism and excessive CXPACKET synchronization waits on multi-core servers.

💡 Pro Tip: Set cost threshold for parallelism to 40-50 and bound MAXDOP to the physical core count of a single NUMA node.

Share Your Production Insights

Contribute your real-world database war stories, post-mortem analyses, and production pro tips to the community knowledge base.

Explore Post-Mortems