tag:blogger.com,1999:blog-5961897154189219803.post1382292207424341236..comments2023-06-13T05:20:31.899-07:00Comments on Ops Monkey: Linux memory overcommitAndy Millerhttp://www.blogger.com/profile/16912923643909155650noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-5961897154189219803.post-89260066304749662672013-12-27T08:58:55.798-08:002013-12-27T08:58:55.798-08:00Craig: It is not nearly as stupid as you think, no...Craig: It is not nearly as stupid as you think, nor is it behaviour unique to Linux. Processes frequently ask for more virtual memory than they need *right now*, so without overcommit you get errors when none are warranted.<br /><br />Let's take those giant JVM's. Imagine now that you fork off a child process to do something as basic as "chown". For a brief moment in time, the child will take up just as much virtual memory as the parent. In these tight memory scenarios, that would mean the fork would fail. Yet in practice the child only needs a few megabytes at most to run chown, and that is about to become very apparent once "exec()" runs. Overcommit allows the "chown" to succeed as one might hope it would.Christopher Smithhttps://www.blogger.com/profile/13820642342200638732noreply@blogger.comtag:blogger.com,1999:blog-5961897154189219803.post-67862422305234980602010-07-21T14:38:11.559-07:002010-07-21T14:38:11.559-07:00From the kernel docs:The CommitLimit is calculated...From the kernel docs:The CommitLimit is calculated with the following formula: CommitLimit = ('vm.overcommit_ratio' * Physical RAM) + Swap. For example, on a system with 1G of physical RAM and 7G of swap with a `vm.overcommit_ratio` of 30 it would yield a CommitLimit of 7.3G.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5961897154189219803.post-5885602379530240272010-06-03T11:33:06.945-07:002010-06-03T11:33:06.945-07:00Fascinating. I had never really understood before ...Fascinating. I had never really understood before what the OOM killer was all about. What an incredibly stupid idea. "Let's allow memory allocations we can't satisfy and then semi-randomly kill processes when the demand for actual pages exceeds what we can supply!" In other words, no matter how good my code is, no matter how carefully tested it is, it can still randomly die on Linux because Linux itself can't be trusted to honor its contracts with applications. That other post about passengers being ejected from airplanes has it right.Craighttps://www.blogger.com/profile/07241209659499192073noreply@blogger.comtag:blogger.com,1999:blog-5961897154189219803.post-62051639148870857542009-08-28T08:28:44.800-07:002009-08-28T08:28:44.800-07:00played about with this, it appears that overcommit...played about with this, it appears that overcommit_ratio is 'percentage of FREE physical memory', rather than total. Just though I'd highlight this as it makes a difference to what you set the ratio to...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5961897154189219803.post-75307278144585006052007-09-23T13:25:00.000-07:002007-09-23T13:25:00.000-07:00See also this RedHat document on the topic.To quot...See also this <A HREF="http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-proc-directories.html" REL="nofollow">RedHat document</A> on the topic.<BR/><BR/>To quote their description of over_commit_memory setting 2:<BR/> "The kernel fails requests for memory that add up to all of swap plus the percent of physical RAM specified in /proc/sys/vm/overcommit_ratio. This setting is best for those who desire less risk of memory overcommitment." <BR/><BR/>And from their coverage of overcommit_ratio:<BR/> "Specifies the percentage of physical RAM considered when /proc/sys/vm/overcommit_memory is set to 2. The default value is 50."<BR/><BR/>They also suggest that overcommit_memory should only be set to 2 on systems with large swap space. I suppose that's something to keep in mind if you're running an embedded system with no writeable hard drive, for instance. Even in that case, though, I would prefer my malloc()s return failure immediately rather than lead me to believe the memory is available.Unknownhttps://www.blogger.com/profile/01860744546467721721noreply@blogger.com