How does opening a file descriptor effect memory
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
Let’s say I have a 250MB file that I read from by opening a file descriptor. Does the entire file get dumped into memory and stay there until I close the file descriptor?
How does that differ from parsing data from the same file with a command like sed, grep, or awk?
text-processing awk sed files grep
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
add a comment |Â
up vote
0
down vote
favorite
Let’s say I have a 250MB file that I read from by opening a file descriptor. Does the entire file get dumped into memory and stay there until I close the file descriptor?
How does that differ from parsing data from the same file with a command like sed, grep, or awk?
text-processing awk sed files grep
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Let’s say I have a 250MB file that I read from by opening a file descriptor. Does the entire file get dumped into memory and stay there until I close the file descriptor?
How does that differ from parsing data from the same file with a command like sed, grep, or awk?
text-processing awk sed files grep
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
Let’s say I have a 250MB file that I read from by opening a file descriptor. Does the entire file get dumped into memory and stay there until I close the file descriptor?
How does that differ from parsing data from the same file with a command like sed, grep, or awk?
text-processing awk sed files grep
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
asked Dec 25 '17 at 2:57
EnterUserNameHere
4318
4318
add a comment |Â
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
The kernel will buffer the file in memory according to the kernel policy and memory demand situation. The whole file may end up in memory, and it will possibly stay in memory even after the file descriptor is closed until the buffers need to be reused, so if you run the same application with the same file again directly afterwards, it will be much faster. Or only parts may end up in memory, read after each other.
All applications behave the same in that respect, it doesn't matter what the application is. Though there are different ways of opening a file, and this may affect details of how everything works.
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
The kernel will buffer the file in memory according to the kernel policy and memory demand situation. The whole file may end up in memory, and it will possibly stay in memory even after the file descriptor is closed until the buffers need to be reused, so if you run the same application with the same file again directly afterwards, it will be much faster. Or only parts may end up in memory, read after each other.
All applications behave the same in that respect, it doesn't matter what the application is. Though there are different ways of opening a file, and this may affect details of how everything works.
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
add a comment |Â
up vote
2
down vote
accepted
The kernel will buffer the file in memory according to the kernel policy and memory demand situation. The whole file may end up in memory, and it will possibly stay in memory even after the file descriptor is closed until the buffers need to be reused, so if you run the same application with the same file again directly afterwards, it will be much faster. Or only parts may end up in memory, read after each other.
All applications behave the same in that respect, it doesn't matter what the application is. Though there are different ways of opening a file, and this may affect details of how everything works.
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
add a comment |Â
up vote
2
down vote
accepted
up vote
2
down vote
accepted
The kernel will buffer the file in memory according to the kernel policy and memory demand situation. The whole file may end up in memory, and it will possibly stay in memory even after the file descriptor is closed until the buffers need to be reused, so if you run the same application with the same file again directly afterwards, it will be much faster. Or only parts may end up in memory, read after each other.
All applications behave the same in that respect, it doesn't matter what the application is. Though there are different ways of opening a file, and this may affect details of how everything works.
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
The kernel will buffer the file in memory according to the kernel policy and memory demand situation. The whole file may end up in memory, and it will possibly stay in memory even after the file descriptor is closed until the buffers need to be reused, so if you run the same application with the same file again directly afterwards, it will be much faster. Or only parts may end up in memory, read after each other.
All applications behave the same in that respect, it doesn't matter what the application is. Though there are different ways of opening a file, and this may affect details of how everything works.
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
answered Dec 25 '17 at 8:54
dirkt
14.1k2931
14.1k2931
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f412880%2fhow-does-opening-a-file-descriptor-effect-memory%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password