TL;DR
PostgreSQL developers have implemented strict memory overcommit policies to mitigate the risk of the Linux Out-Of-Memory (OOM) killer terminating database processes. This change aims to enhance stability and predictability in production environments by controlling memory usage more tightly.
PostgreSQL has adopted a policy of strict memory overcommit to reduce the risk of being terminated by the Linux OOM killer. This change, confirmed by PostgreSQL developers, aims to improve database stability in production environments by preventing unexpected process termination due to memory overuse.
In a recent update, PostgreSQL developers announced that they are configuring the Linux kernel’s memory overcommit settings to a more restrictive mode. Specifically, they are setting vm.overcommit_memory to 2, which prevents the kernel from over-allocating memory beyond the total available physical RAM and swap space. This adjustment aims to prevent the Linux Out-Of-Memory (OOM) killer from terminating PostgreSQL processes when memory limits are exceeded.
This move responds to longstanding issues where PostgreSQL instances, especially under high load or misconfigured environments, could be unexpectedly killed by the OOM killer, leading to data loss or service downtime. By enforcing strict overcommit policies, PostgreSQL seeks to make memory usage more predictable and reduce the risk of such incidents.
According to PostgreSQL core developers, this change is part of a broader effort to optimize stability and reliability, particularly in cloud and containerized deployments where resource constraints are common. They emphasize that this approach may require administrators to carefully tune memory settings and monitor usage to avoid out-of-memory errors that could prevent PostgreSQL from functioning properly.
Impact of Strict Memory Overcommit on PostgreSQL Stability
The adoption of strict memory overcommit settings is significant because it directly addresses the risk of unexpected process termination caused by the Linux OOM killer. For users running PostgreSQL in production, especially in cloud environments, this change can lead to increased stability and fewer outages due to memory overuse. However, it also shifts some responsibility to system administrators to manage memory allocations more carefully, as over-allocating memory beyond physical limits could cause PostgreSQL to fail to start or operate correctly.
This development underscores a broader industry trend toward tighter resource control in database deployments, and it may influence how other database systems and containerized applications configure Linux kernel parameters for optimal stability.
Tricks for Python performance optimization and memory management – Tips for efficient resource usage and speedup using profiling tools – (Japanese Edition)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
PostgreSQL Memory Management and Linux Kernel Settings
Historically, PostgreSQL has relied on the Linux kernel’s default memory overcommit behavior, which often allows processes to allocate more memory than physically available, relying on the kernel to handle over-allocation. This approach can lead to situations where the Linux OOM killer terminates processes unexpectedly, especially under high load or misconfigured environments.
In recent years, Linux introduced the vm.overcommit_memory setting, with modes 0, 1, and 2, controlling how overcommit is handled. Mode 0 is the default, allowing overcommit based on heuristics; mode 1 permits overcommit without restrictions; mode 2 enforces strict limits, preventing overcommit beyond total RAM plus swap.
PostgreSQL’s move to strict overcommit mode (mode 2) reflects a shift toward prioritizing stability over maximum memory utilization. This change aligns with best practices for containerized and cloud deployments, where predictable resource usage is critical.
Prior to this, some PostgreSQL deployments experienced process termination due to the OOM killer, prompting the developers to recommend tuning kernel parameters and system memory settings explicitly for production environments.
“Configuring the Linux kernel to use strict overcommit mode significantly reduces the risk of unexpected process termination due to memory overuse.”
— PostgreSQL core developer
Linux server memory overcommit settings
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Remaining Questions About Implementation and Impact
It is not yet clear how widespread adoption of this setting will be across different PostgreSQL deployments, or how it will impact performance under various workloads. Some administrators may experience increased startup failures if memory is not properly allocated or tuned.
Additionally, the long-term effects on system stability and resource management in diverse environments remain to be observed. Developers continue to evaluate whether this approach balances stability with optimal resource utilization.
Nice DDR4 Desktop PC Server Memory RAM Module and Computer Motherboard RAM Slot Easy Test Quick Diagnostic Analyzer LED Tester Card Complete Set Solution Kit 288 Pin All Speed Voltage ECC and nonECC
【1】***** Attn: Please DO study the Manual PDF Extra Tech Tips and Precaution PDF BEFORE the use, on…
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Next Steps for PostgreSQL and Linux Kernel Configuration
PostgreSQL will likely release updated documentation and recommendations for configuring systems with strict overcommit settings. Community feedback and real-world testing will shape best practices going forward.
System administrators are advised to review their memory configurations, monitor system behavior, and test PostgreSQL deployments under new settings to ensure stability and performance.
Further updates may include automated tools or scripts to help tune memory settings and prevent startup or runtime errors related to overcommit policies.
PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Why does PostgreSQL prefer strict memory overcommit?
PostgreSQL adopts strict memory overcommit to reduce the risk of being terminated unexpectedly by the Linux OOM killer, thereby improving stability in production environments.
What is the Linux kernel setting involved?
The setting is vm.overcommit_memory, which is being set to 2 to enforce strict memory limits and prevent over-allocation beyond physical RAM and swap.
Will this affect PostgreSQL performance?
It may impact performance if memory is not carefully allocated, as stricter limits could lead to failures in memory-intensive workloads. Proper tuning is recommended.
Is this change mandatory for all deployments?
No, it is a recommended best practice for stability, especially in cloud and containerized environments, but system administrators can choose to implement it based on their needs.
What are the risks of not adopting strict overcommit?
Without strict overcommit, PostgreSQL processes risk being killed unexpectedly by the OOM killer during memory pressure, leading to potential data loss or service outages.
Source: hn
