Where the remaining memory of vm.overcommit_ratio goes?
Clash Royale CLAN TAG#URR8PPP
If I disable memory overcommit by setting vm.overcommit_memory to 2, by default the system will allow to allocate the memory up to the dimension of swap + 50% of physical memory, as explained here.
I can change the ratio by modifying vm.overcommit_ratio parameter.
Let's say I set it to 80%, so 80% of physical memory may be used.
My question are:
- what the system will do with the remaining 20%?
- why is this parameter required in first place?
- why I should not always set it to 100%?
linux kernel memory out-of-memory
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
add a comment |
If I disable memory overcommit by setting vm.overcommit_memory to 2, by default the system will allow to allocate the memory up to the dimension of swap + 50% of physical memory, as explained here.
I can change the ratio by modifying vm.overcommit_ratio parameter.
Let's say I set it to 80%, so 80% of physical memory may be used.
My question are:
- what the system will do with the remaining 20%?
- why is this parameter required in first place?
- why I should not always set it to 100%?
linux kernel memory out-of-memory
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
add a comment |
If I disable memory overcommit by setting vm.overcommit_memory to 2, by default the system will allow to allocate the memory up to the dimension of swap + 50% of physical memory, as explained here.
I can change the ratio by modifying vm.overcommit_ratio parameter.
Let's say I set it to 80%, so 80% of physical memory may be used.
My question are:
- what the system will do with the remaining 20%?
- why is this parameter required in first place?
- why I should not always set it to 100%?
linux kernel memory out-of-memory
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
If I disable memory overcommit by setting vm.overcommit_memory to 2, by default the system will allow to allocate the memory up to the dimension of swap + 50% of physical memory, as explained here.
I can change the ratio by modifying vm.overcommit_ratio parameter.
Let's say I set it to 80%, so 80% of physical memory may be used.
My question are:
- what the system will do with the remaining 20%?
- why is this parameter required in first place?
- why I should not always set it to 100%?
linux kernel memory out-of-memory
linux kernel memory out-of-memory
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
edited Jul 9 '16 at 14:43
Kusalananda
129k16246404
129k16246404
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
asked Jul 6 '16 at 9:15
Dan TumaykinDan Tumaykin
987
987
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
What the system will do with the remaining 20%?
The kernel will use the remaining physical memory for its own purposes (internal structures, tables, buffers, caches, whatever). The memory overcommitment setting handle userland application virtual memory reservations, the kernel doesn't use virtual memory but physical one.
Why is this parameter required in first place?
The overcommit_ratio parameter is an implementation choice designed to prevent applications to reserve more virtual memory than what will reasonably be available for them in the future, i.e. when they actually access the memory (or at least try to).
Setting overcommit_ratio to 50% has been considered a reasonable default value by the Linux kernel developers. It assumes the kernel won't ever need to use more than 50% of the physical RAM. Your mileage may vary, the reason why it is a tunable.
Why I should not always set it to 100%?
Setting it to 100% (or any "too high" value) doesn't reliably disable overcommitment because you cannot assume the kernel will use 0% (or too little) of RAM.
It won't prevent applications to crash as the kernel might preempt anyway all the physical memory it demands.
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
add a comment |
Setting the ratio to 100% will not reserve any space for file-backed pages, or in-kernel allocations such as kernel code, network buffers, etc.
In-kernel structures will be allocated regardless, causing overcommit. They're generally limited individually (e.g. there's a setting for network buffers). I don't think there's an overall limit of 50%, although an overall limit is something that's been worked on for the purpose of hosting containers.
File-backed pages are where you usually run userspace code from, so you need space for that too.
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f294180%2fwhere-the-remaining-memory-of-vm-overcommit-ratio-goes%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
What the system will do with the remaining 20%?
The kernel will use the remaining physical memory for its own purposes (internal structures, tables, buffers, caches, whatever). The memory overcommitment setting handle userland application virtual memory reservations, the kernel doesn't use virtual memory but physical one.
Why is this parameter required in first place?
The overcommit_ratio parameter is an implementation choice designed to prevent applications to reserve more virtual memory than what will reasonably be available for them in the future, i.e. when they actually access the memory (or at least try to).
Setting overcommit_ratio to 50% has been considered a reasonable default value by the Linux kernel developers. It assumes the kernel won't ever need to use more than 50% of the physical RAM. Your mileage may vary, the reason why it is a tunable.
Why I should not always set it to 100%?
Setting it to 100% (or any "too high" value) doesn't reliably disable overcommitment because you cannot assume the kernel will use 0% (or too little) of RAM.
It won't prevent applications to crash as the kernel might preempt anyway all the physical memory it demands.
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
add a comment |
What the system will do with the remaining 20%?
The kernel will use the remaining physical memory for its own purposes (internal structures, tables, buffers, caches, whatever). The memory overcommitment setting handle userland application virtual memory reservations, the kernel doesn't use virtual memory but physical one.
Why is this parameter required in first place?
The overcommit_ratio parameter is an implementation choice designed to prevent applications to reserve more virtual memory than what will reasonably be available for them in the future, i.e. when they actually access the memory (or at least try to).
Setting overcommit_ratio to 50% has been considered a reasonable default value by the Linux kernel developers. It assumes the kernel won't ever need to use more than 50% of the physical RAM. Your mileage may vary, the reason why it is a tunable.
Why I should not always set it to 100%?
Setting it to 100% (or any "too high" value) doesn't reliably disable overcommitment because you cannot assume the kernel will use 0% (or too little) of RAM.
It won't prevent applications to crash as the kernel might preempt anyway all the physical memory it demands.
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
add a comment |
What the system will do with the remaining 20%?
The kernel will use the remaining physical memory for its own purposes (internal structures, tables, buffers, caches, whatever). The memory overcommitment setting handle userland application virtual memory reservations, the kernel doesn't use virtual memory but physical one.
Why is this parameter required in first place?
The overcommit_ratio parameter is an implementation choice designed to prevent applications to reserve more virtual memory than what will reasonably be available for them in the future, i.e. when they actually access the memory (or at least try to).
Setting overcommit_ratio to 50% has been considered a reasonable default value by the Linux kernel developers. It assumes the kernel won't ever need to use more than 50% of the physical RAM. Your mileage may vary, the reason why it is a tunable.
Why I should not always set it to 100%?
Setting it to 100% (or any "too high" value) doesn't reliably disable overcommitment because you cannot assume the kernel will use 0% (or too little) of RAM.
It won't prevent applications to crash as the kernel might preempt anyway all the physical memory it demands.
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
What the system will do with the remaining 20%?
The kernel will use the remaining physical memory for its own purposes (internal structures, tables, buffers, caches, whatever). The memory overcommitment setting handle userland application virtual memory reservations, the kernel doesn't use virtual memory but physical one.
Why is this parameter required in first place?
The overcommit_ratio parameter is an implementation choice designed to prevent applications to reserve more virtual memory than what will reasonably be available for them in the future, i.e. when they actually access the memory (or at least try to).
Setting overcommit_ratio to 50% has been considered a reasonable default value by the Linux kernel developers. It assumes the kernel won't ever need to use more than 50% of the physical RAM. Your mileage may vary, the reason why it is a tunable.
Why I should not always set it to 100%?
Setting it to 100% (or any "too high" value) doesn't reliably disable overcommitment because you cannot assume the kernel will use 0% (or too little) of RAM.
It won't prevent applications to crash as the kernel might preempt anyway all the physical memory it demands.
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
edited Jul 9 '16 at 20:20
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
answered Jul 8 '16 at 12:36
jlliagrejlliagre
47.2k784134
47.2k784134
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
add a comment |
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
If the kernel can take all the memory it needs anyway, what's the point of exposing this parameter (or even creating it)?
– Dan Tumaykin
Jul 8 '16 at 14:14
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
Is there any official document where those parameters are explained in details? Besides kernel.org/doc/Documentation/vm/overcommit-accounting - any reference to kernel memory is missing there.
– Dan Tumaykin
Jul 11 '16 at 8:40
1
1
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
I haven't found yet documentation explaining with enough details what is precisely taken into account when referring to "total address space commit". The fact the document you provide a link to states "in most situations this means a process will not be killed..." is however sufficient to confirm some memory can be used elsewhere, and the obvious consumer for this memory is the kernel.
– jlliagre
Jul 11 '16 at 11:30
add a comment |
Setting the ratio to 100% will not reserve any space for file-backed pages, or in-kernel allocations such as kernel code, network buffers, etc.
In-kernel structures will be allocated regardless, causing overcommit. They're generally limited individually (e.g. there's a setting for network buffers). I don't think there's an overall limit of 50%, although an overall limit is something that's been worked on for the purpose of hosting containers.
File-backed pages are where you usually run userspace code from, so you need space for that too.
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
add a comment |
Setting the ratio to 100% will not reserve any space for file-backed pages, or in-kernel allocations such as kernel code, network buffers, etc.
In-kernel structures will be allocated regardless, causing overcommit. They're generally limited individually (e.g. there's a setting for network buffers). I don't think there's an overall limit of 50%, although an overall limit is something that's been worked on for the purpose of hosting containers.
File-backed pages are where you usually run userspace code from, so you need space for that too.
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
add a comment |
Setting the ratio to 100% will not reserve any space for file-backed pages, or in-kernel allocations such as kernel code, network buffers, etc.
In-kernel structures will be allocated regardless, causing overcommit. They're generally limited individually (e.g. there's a setting for network buffers). I don't think there's an overall limit of 50%, although an overall limit is something that's been worked on for the purpose of hosting containers.
File-backed pages are where you usually run userspace code from, so you need space for that too.
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
Setting the ratio to 100% will not reserve any space for file-backed pages, or in-kernel allocations such as kernel code, network buffers, etc.
In-kernel structures will be allocated regardless, causing overcommit. They're generally limited individually (e.g. there's a setting for network buffers). I don't think there's an overall limit of 50%, although an overall limit is something that's been worked on for the purpose of hosting containers.
File-backed pages are where you usually run userspace code from, so you need space for that too.
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
edited Jan 25 at 18:47
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
answered Jul 9 '16 at 17:30
sourcejedisourcejedi
24.3k440107
24.3k440107
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f294180%2fwhere-the-remaining-memory-of-vm-overcommit-ratio-goes%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
