Category: PostgreSQL → Performance Tuning → Memory Management Technical Level: Intermediate / Advanced Reading Time: 18–22 minutes

In PostgreSQL performance tuning, shared_buffers is among the most frequently debated parameters.

When setting up a new PostgreSQL server, administrators typically encounter this baseline guideline:

While this recommendation serves as a sensible starting point for many systems, adjusting this single setting without understanding PostgreSQL's broader memory architecture often yields disappointing results.

PostgreSQL does not rely on a single caching mechanism for database pages.

In a standard Linux buffered I/O execution path, two distinct caching tiers operate:

1. 1PostgreSQL Shared Buffers: The shared memory pool managed directly by the database engine.
2. 2Linux Page Cache: The kernel-level filesystem cache holding file contents in physical memory.

Although managed independently, these two tiers jointly determine database throughput and I/O latency.

When PostgreSQL fails to find a required block in Shared Buffers, it does not necessarily trigger an expensive physical disk read.

The requested block may already reside in the Linux Page Cache.

This distinction is central to accurate PostgreSQL performance diagnosis.

In this deep dive, we examine:

• •How PostgreSQL Shared Buffers and the Linux Page Cache operate together.
• •The exact layers traversed when executing an SQL query.
• •Whether the same data block can reside in both memory tiers simultaneously (*double caching*).
• •Why increasing Shared Buffers does not always improve performance.
• •Why the PostgreSQL buffer cache hit ratio is misleading on its own.
• •Practical techniques for observing Linux Page Cache behavior.
• •How to determine the ideal memory allocation for production workloads.

PostgreSQL memory consumption extends far beyond shared_buffers.

Total server memory is divided across several distinct components:

Shared Buffers is directly controlled by PostgreSQL processes.

The Linux Page Cache, by contrast, is an operating system mechanism shared across all active processes on the host.

A healthy configuration requires balancing the memory dedicated exclusively to PostgreSQL against the memory left available for kernel caching.

Shared Buffers is the fixed-size shared memory segment where PostgreSQL keeps active 8 KB database pages.

For example:

With standard 8 KB pages, this reserves room for approximately 1,048,576 buffer slots.

Any backend process can access pages residing in Shared Buffers, avoiding redundant filesystem calls.

PostgreSQL manages buffer eviction using a clock-sweep replacement algorithm that tracks page usage counts.

In addition, bulk sequential scans and maintenance routines employ dedicated buffer access strategies (ring buffers) to prevent large scans from evicting hot working sets.

The Linux Page Cache stores page-sized (typically 4 KB) chunks of regular files in unused RAM.

Under standard buffered I/O, when an application requests file data, the kernel first inspects the Page Cache.

If present (cache hit), the read completes in RAM with zero disk head movement or NVMe controller interaction.

If absent (cache miss), the kernel fetches the missing pages from physical storage and places them into the Page Cache.

Under memory pressure, Linux automatically reclaims clean page cache pages, writing dirty pages out as necessary.

Therefore, viewing memory occupied by Linux cache as "wasted" or unavailable is a fundamental operational misconception.

PostgreSQL Shared Buffers and Linux Page Cache Interaction Diagram

Consider a simple point lookup:

Assuming PostgreSQL must access the relevant table or index page:

PostgreSQL discovers the target page already cached in Shared Buffers.

No filesystem read is requested.

This is recorded in database statistics as a buffer hit.

The requested block is absent from Shared Buffers.

PostgreSQL issues a read system call to the operating system.

However, the file pages are already resident in the Linux Page Cache.

The kernel fulfills the request directly from memory without issuing physical NVMe/SSD storage I/O.

From PostgreSQL's perspective, a buffer read occurred; yet physically, no disk I/O took place.

The block is absent from both Shared Buffers and the Linux Page Cache.

The operating system issues I/O commands to physical storage.

Data is loaded into the Page Cache and copied into PostgreSQL's buffer pool slot.

Each scenario has drastically different latency characteristics.

Scenario B explains why high read counts in PostgreSQL do not automatically correlate with saturated storage controllers.

Yes.

Because PostgreSQL uses standard buffered I/O, a single 8 KB table page may exist simultaneously in Shared Buffers and in the Linux Page Cache.

This phenomenon is commonly known as *double caching*.

No.

While duplicating data might seem wasteful, each cache layer fulfills a different role:

• •Shared Buffers gives PostgreSQL pin/lock primitives, transaction visibility awareness, and clock-sweep eviction tuned for relational access.
• •Linux Page Cache handles kernel readahead, device-level write coalescing, and host-wide memory reclamation.

The goal is not to eliminate double caching entirely, but to strike an equilibrium where both layers operate efficiently.

Allocating excessive RAM to Shared Buffers starves the Linux Page Cache and limits per-query work_mem.

Potential risks include:

• •Severely diminished OS cache capacity for unbuffered file access.
• •Heightened risk of host memory exhaustion, OOM kills, and swapping.
• •Larger checkpoint spikes and heavier writeback pressure during dirty buffer flushes.
• •Unpredictable query latency degradation under mixed workloads.

Official PostgreSQL documentation recommends 25% of system RAM as an initial baseline because PostgreSQL intentionally relies on the underlying OS cache.

Allocating more than 40% of RAM rarely yields linear throughput improvements and often degrades mixed analytical/reporting workloads.

The remaining memory accommodates backend connection heaps, sorting/hash memory (work_mem), autovacuum workers, and the kernel Page Cache.

Administrators often monitor database-level cache hit ratio:

A high ratio (e.g. 99%) in OLTP databases indicates healthy buffer reuse.

However, an 80% hit ratio in an analytics database does not mean 20% of reads hit physical disks; the vast majority are likely served by the Linux Page Cache in microseconds.

Output snippet:

Here, read=1500 signifies 1,500 blocks fetched via OS read calls, not 1,500 physical NVMe transactions.

Note: effective_cache_size does not allocate memory; it informs planner cost estimates for index scans versus sequential scans.

Track MemAvailable, Cached, Dirty, and swap in/out (si/so).

A production 64 GB NVMe server hosting 500 GB mixed OLTP/reporting was resized from 16 GB to 32 GB Shared Buffers.

Reporting queries slowed noticeably because large scans that previously benefited from kernel readahead and generous Page Cache suffered from squeezed memory and increased dirty buffer eviction pressure.

1. 1Classify Workloads: OLTP benefits heavily from Shared Buffers; analytical queries rely more on kernel readahead and ring buffers.
2. 2Calculate Total Budget: Account for max_connections * work_mem, autovacuum, OS kernel, and Page Cache.
3. 3Establish Baseline: Start with 25% of RAM.
4. 4Benchmark Incrementally: Test 15%, 25%, and 35% with realistic query mixes under concurrency.
5. 5Measure Real User Latency: Focus on p95/p99 query durations rather than raw cache hit ratios alone.

• •Pitfall 1: Maximizing Shared Buffers blindly.
• •Pitfall 2: Equating blks_read with physical disk I/O.
• •Pitfall 3: Treating Linux cached RAM as unusable memory.
• •Pitfall 4: Believing effective_cache_size allocates memory.
• •Pitfall 5: Dropping kernel caches in production with drop_caches.
• •Pitfall 6: Overlooking checkpoint spread and writeback spikes.

• Is Shared Buffers proportional to RAM and current query profile?
• How does PostgreSQL cache hit ratio trend over time?
• Does pg_stat_io show abnormal eviction rates?
• Is Linux MemAvailable healthy without swap activity?
• What are the physical storage IOPS and read/write latencies?
• Do checkpoint spikes induce query latency jitter?
• Is effective_cache_size realistically tuned?

PostgreSQL performance depends as much on the Linux kernel Page Cache as it does on the internal database buffer pool.

Rather than maximizing Shared Buffers in isolation, DBAs must cultivate a balanced dual-tier architecture tailored to the host's actual workload.

1. 1PostgreSQL Documentation — Resource Consumption
2. 2PostgreSQL Documentation — Monitoring Statistics
3. 3PostgreSQL Documentation — EXPLAIN
4. 4Linux Kernel Documentation — Memory Management
5. 5Linux Kernel Documentation — /proc Filesystem