Can an application explicitly commit and decommit memory?
Clash Royale CLAN TAG#URR8PPP
up vote
2
down vote
favorite
Is it possible - from an application - to explicitly control memory page commits? In a Windows app many years ago, I wrote an "alarm system" heap manager to help me find a random dangling reference/heap corruption bug. I was able to reserve a large area of virtual memory, and then commit/decommit pages at will. The idea at the time was to create a set containing an uncommitted page, followed by one or more committed pages (sufficient to fill the request), followed by another uncommitted page; with a compile-time flag that controlled whether the pointer returned to the caller was at the beginning or end of the committed region (to instantly fire an exception on an underwrite or overwrite, respectively). Windows provides an API for reserving memory regions and committing pages. I have another use for such a book-ended buffer in a Linux program I'm writing, and I'm wondering if the kernel provides the same kind of functionality.
memory virtual-memory
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
asked Nov 21 '17 at 1:39
JackLThornton
112
add a comment |Â
up vote
2
down vote
favorite
Is it possible - from an application - to explicitly control memory page commits? In a Windows app many years ago, I wrote an "alarm system" heap manager to help me find a random dangling reference/heap corruption bug. I was able to reserve a large area of virtual memory, and then commit/decommit pages at will. The idea at the time was to create a set containing an uncommitted page, followed by one or more committed pages (sufficient to fill the request), followed by another uncommitted page; with a compile-time flag that controlled whether the pointer returned to the caller was at the beginning or end of the committed region (to instantly fire an exception on an underwrite or overwrite, respectively). Windows provides an API for reserving memory regions and committing pages. I have another use for such a book-ended buffer in a Linux program I'm writing, and I'm wondering if the kernel provides the same kind of functionality.
memory virtual-memory
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
asked Nov 21 '17 at 1:39
JackLThornton
112
The short answer is yes, Unix/Linux providesmalloc(memory allocate) andfreefunctions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.
– G-Man
Nov 21 '17 at 1:52
@G-Man:mallocandfreeare totally different from the virtual memory commit in windows. Virtual memory is reserved viammapin Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.
– dirkt
Nov 21 '17 at 18:46
add a comment |Â
up vote
2
down vote
favorite
up vote
2
down vote
favorite
Is it possible - from an application - to explicitly control memory page commits? In a Windows app many years ago, I wrote an "alarm system" heap manager to help me find a random dangling reference/heap corruption bug. I was able to reserve a large area of virtual memory, and then commit/decommit pages at will. The idea at the time was to create a set containing an uncommitted page, followed by one or more committed pages (sufficient to fill the request), followed by another uncommitted page; with a compile-time flag that controlled whether the pointer returned to the caller was at the beginning or end of the committed region (to instantly fire an exception on an underwrite or overwrite, respectively). Windows provides an API for reserving memory regions and committing pages. I have another use for such a book-ended buffer in a Linux program I'm writing, and I'm wondering if the kernel provides the same kind of functionality.
memory virtual-memory
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
asked Nov 21 '17 at 1:39
JackLThornton
112
Is it possible - from an application - to explicitly control memory page commits? In a Windows app many years ago, I wrote an "alarm system" heap manager to help me find a random dangling reference/heap corruption bug. I was able to reserve a large area of virtual memory, and then commit/decommit pages at will. The idea at the time was to create a set containing an uncommitted page, followed by one or more committed pages (sufficient to fill the request), followed by another uncommitted page; with a compile-time flag that controlled whether the pointer returned to the caller was at the beginning or end of the committed region (to instantly fire an exception on an underwrite or overwrite, respectively). Windows provides an API for reserving memory regions and committing pages. I have another use for such a book-ended buffer in a Linux program I'm writing, and I'm wondering if the kernel provides the same kind of functionality.
memory virtual-memory
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
asked Nov 21 '17 at 1:39
JackLThornton
112
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
edited Aug 21 at 1:49
Rui F Ribeiro
35.8k1271114
35.8k1271114
asked Nov 21 '17 at 1:39
JackLThornton
112
asked Nov 21 '17 at 1:39
JackLThornton
112
asked Nov 21 '17 at 1:39
JackLThornton
112
112
The short answer is yes, Unix/Linux providesmalloc(memory allocate) andfreefunctions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.
– G-Man
Nov 21 '17 at 1:52
@G-Man:mallocandfreeare totally different from the virtual memory commit in windows. Virtual memory is reserved viammapin Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.
– dirkt
Nov 21 '17 at 18:46
add a comment |Â
The short answer is yes, Unix/Linux providesmalloc(memory allocate) andfreefunctions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.
– G-Man
Nov 21 '17 at 1:52
@G-Man:mallocandfreeare totally different from the virtual memory commit in windows. Virtual memory is reserved viammapin Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.
– dirkt
Nov 21 '17 at 18:46
The short answer is yes, Unix/Linux provides
malloc (memory allocate) and free functions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.– G-Man
Nov 21 '17 at 1:52
The short answer is yes, Unix/Linux provides
malloc (memory allocate) and free functions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.– G-Man
Nov 21 '17 at 1:52
@G-Man:
malloc and free are totally different from the virtual memory commit in windows. Virtual memory is reserved via mmap in Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.– dirkt
Nov 21 '17 at 18:46
@G-Man:
malloc and free are totally different from the virtual memory commit in windows. Virtual memory is reserved via mmap in Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.– dirkt
Nov 21 '17 at 18:46
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
3
down vote
I take it you’re referring to VirtualAlloc and friends. There’s no direct equivalent in Linux as far as I’m aware, but you can achieve the same effect in a variety of ways. If you want to reserve memory, you can use mmap(2); to commit it, you need to write to it; to decommit it, you can use madvise(2)’s MADV_DONTNEED.
To implement guard pages, you might find mprotect(2) to be a better option, along with a SIGSEGV handler. You can also look at the overflow protection provided by your compiler (-fstack-protector etc., and the various -fsanitize options).
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
I take it you’re referring to VirtualAlloc and friends. There’s no direct equivalent in Linux as far as I’m aware, but you can achieve the same effect in a variety of ways. If you want to reserve memory, you can use mmap(2); to commit it, you need to write to it; to decommit it, you can use madvise(2)’s MADV_DONTNEED.
To implement guard pages, you might find mprotect(2) to be a better option, along with a SIGSEGV handler. You can also look at the overflow protection provided by your compiler (-fstack-protector etc., and the various -fsanitize options).
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
add a comment |Â
up vote
3
down vote
I take it you’re referring to VirtualAlloc and friends. There’s no direct equivalent in Linux as far as I’m aware, but you can achieve the same effect in a variety of ways. If you want to reserve memory, you can use mmap(2); to commit it, you need to write to it; to decommit it, you can use madvise(2)’s MADV_DONTNEED.
To implement guard pages, you might find mprotect(2) to be a better option, along with a SIGSEGV handler. You can also look at the overflow protection provided by your compiler (-fstack-protector etc., and the various -fsanitize options).
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
add a comment |Â
up vote
3
down vote
up vote
3
down vote
I take it you’re referring to VirtualAlloc and friends. There’s no direct equivalent in Linux as far as I’m aware, but you can achieve the same effect in a variety of ways. If you want to reserve memory, you can use mmap(2); to commit it, you need to write to it; to decommit it, you can use madvise(2)’s MADV_DONTNEED.
To implement guard pages, you might find mprotect(2) to be a better option, along with a SIGSEGV handler. You can also look at the overflow protection provided by your compiler (-fstack-protector etc., and the various -fsanitize options).
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
I take it you’re referring to VirtualAlloc and friends. There’s no direct equivalent in Linux as far as I’m aware, but you can achieve the same effect in a variety of ways. If you want to reserve memory, you can use mmap(2); to commit it, you need to write to it; to decommit it, you can use madvise(2)’s MADV_DONTNEED.
To implement guard pages, you might find mprotect(2) to be a better option, along with a SIGSEGV handler. You can also look at the overflow protection provided by your compiler (-fstack-protector etc., and the various -fsanitize options).
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
answered Nov 22 '17 at 9:59
Stephen Kitt
143k22312377
143k22312377
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f405883%2fcan-an-application-explicitly-commit-and-decommit-memory%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
The short answer is yes, Unix/Linux provides
malloc(memory allocate) andfreefunctions.  But it’s complicated, and I don’t really understand your objective, so I don’t know whether this will help you.– G-Man
Nov 21 '17 at 1:52
@G-Man:
mallocandfreeare totally different from the virtual memory commit in windows. Virtual memory is reserved viammapin Linux, but to my knowledge there's no explicit commit step in Linux. But I do think one can implement guard pages, e.g. the golang runtime does that. But I don't know details.– dirkt
Nov 21 '17 at 18:46