Linux freezes during I/O load [closed]
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I use Linux Mint with Dell XPS 15 as work laptop and build a large Android application using gradle.
With all optimizations it still uses almost all available RAM (16 GB) and starting to use SWAP from swapfile (maybe, I should use swap partition instead?). It causes a high I/O rate and Cinnamon (and all other applications) have freezes.
Also, I tried to test, that I/O is a reason of these problems and found, that running dd command for creating load to my I/O system also cause the same problems. (I couldn't reproduce this behavior on Mac OS, for example)
I found that I should change I/O scheduler type to deadline, but my /sys/block/nvme0n1/queue/scheduler file contains just none option. And as I understand right it means that https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq) framework is used and I shouldn't to change anything.
Question: How I can solve these freezes during I/O load? Maybe, what metrics should I analyze for getting more information?
Environment:
OS: Linux Mint 18.3 Cinnamon 64-bit
Cinnamon: 3.6.7
Linux Kernel: 4.13.0-38-generic
linux debian linux-mint cinnamon freeze
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
closed as too broad by Rui F Ribeiro, G-Man, Patrick, umläute, Timothy Martin Apr 24 at 21:24
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |Â
up vote
1
down vote
favorite
I use Linux Mint with Dell XPS 15 as work laptop and build a large Android application using gradle.
With all optimizations it still uses almost all available RAM (16 GB) and starting to use SWAP from swapfile (maybe, I should use swap partition instead?). It causes a high I/O rate and Cinnamon (and all other applications) have freezes.
Also, I tried to test, that I/O is a reason of these problems and found, that running dd command for creating load to my I/O system also cause the same problems. (I couldn't reproduce this behavior on Mac OS, for example)
I found that I should change I/O scheduler type to deadline, but my /sys/block/nvme0n1/queue/scheduler file contains just none option. And as I understand right it means that https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq) framework is used and I shouldn't to change anything.
Question: How I can solve these freezes during I/O load? Maybe, what metrics should I analyze for getting more information?
Environment:
OS: Linux Mint 18.3 Cinnamon 64-bit
Cinnamon: 3.6.7
Linux Kernel: 4.13.0-38-generic
linux debian linux-mint cinnamon freeze
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
closed as too broad by Rui F Ribeiro, G-Man, Patrick, umläute, Timothy Martin Apr 24 at 21:24
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
I can keep addin the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try usingioniceto allow other programs to have I/O priority.
– dsstorefile1
Apr 18 at 22:54
please show the exactddcommand you are using to test. Also if you could show thetopheaders (first 5 lines) that would help as well.
– Patrick
Apr 18 at 23:34
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I use Linux Mint with Dell XPS 15 as work laptop and build a large Android application using gradle.
With all optimizations it still uses almost all available RAM (16 GB) and starting to use SWAP from swapfile (maybe, I should use swap partition instead?). It causes a high I/O rate and Cinnamon (and all other applications) have freezes.
Also, I tried to test, that I/O is a reason of these problems and found, that running dd command for creating load to my I/O system also cause the same problems. (I couldn't reproduce this behavior on Mac OS, for example)
I found that I should change I/O scheduler type to deadline, but my /sys/block/nvme0n1/queue/scheduler file contains just none option. And as I understand right it means that https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq) framework is used and I shouldn't to change anything.
Question: How I can solve these freezes during I/O load? Maybe, what metrics should I analyze for getting more information?
Environment:
OS: Linux Mint 18.3 Cinnamon 64-bit
Cinnamon: 3.6.7
Linux Kernel: 4.13.0-38-generic
linux debian linux-mint cinnamon freeze
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
I use Linux Mint with Dell XPS 15 as work laptop and build a large Android application using gradle.
With all optimizations it still uses almost all available RAM (16 GB) and starting to use SWAP from swapfile (maybe, I should use swap partition instead?). It causes a high I/O rate and Cinnamon (and all other applications) have freezes.
Also, I tried to test, that I/O is a reason of these problems and found, that running dd command for creating load to my I/O system also cause the same problems. (I couldn't reproduce this behavior on Mac OS, for example)
I found that I should change I/O scheduler type to deadline, but my /sys/block/nvme0n1/queue/scheduler file contains just none option. And as I understand right it means that https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq) framework is used and I shouldn't to change anything.
Question: How I can solve these freezes during I/O load? Maybe, what metrics should I analyze for getting more information?
Environment:
OS: Linux Mint 18.3 Cinnamon 64-bit
Cinnamon: 3.6.7
Linux Kernel: 4.13.0-38-generic
linux debian linux-mint cinnamon freeze
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
edited Apr 18 at 21:32
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
asked Apr 18 at 21:23
Ôøüð Ãœõрúурьõò
62
62
closed as too broad by Rui F Ribeiro, G-Man, Patrick, umläute, Timothy Martin Apr 24 at 21:24
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as too broad by Rui F Ribeiro, G-Man, Patrick, umläute, Timothy Martin Apr 24 at 21:24
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
I can keep addin the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try usingioniceto allow other programs to have I/O priority.
– dsstorefile1
Apr 18 at 22:54
please show the exactddcommand you are using to test. Also if you could show thetopheaders (first 5 lines) that would help as well.
– Patrick
Apr 18 at 23:34
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10
add a comment |Â
I can keep addin the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try usingioniceto allow other programs to have I/O priority.
– dsstorefile1
Apr 18 at 22:54
please show the exactddcommand you are using to test. Also if you could show thetopheaders (first 5 lines) that would help as well.
– Patrick
Apr 18 at 23:34
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10
I can keep a
dd in the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try using ionice to allow other programs to have I/O priority.– dsstorefile1
Apr 18 at 22:54
I can keep a
dd in the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try using ionice to allow other programs to have I/O priority.– dsstorefile1
Apr 18 at 22:54
please show the exact
dd command you are using to test. Also if you could show the top headers (first 5 lines) that would help as well.– Patrick
Apr 18 at 23:34
please show the exact
dd command you are using to test. Also if you could show the top headers (first 5 lines) that would help as well.– Patrick
Apr 18 at 23:34
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
0
down vote
With all optimizations it still uses almost all available RAM (16 GB)
it — who? Typically every general purpose OS designed since 1970s uses all available RAM or its huge part for slow storage content caching intensively. If it means VM cache what troubles are there? Run free -m and study its output, it used to have special indication of "+/- cache/buffers" hinting that it's not irrevocable used — cache would shrink if there's memory pressure.
It's pretty typical misconception a novice would have…
I found that I should change I/O scheduler type to deadline
Schedulers are there for slow devices like HDDs. With SSDs/NVME it's just extra overhead — you don't need to have queue for requests because there's no gain in re-ordering them — contrary to HDDs where it plays significant role reducing seek times.
How I can solve these freezes during I/O load?
There're no mindreaders here (being an exception I prefer to keep my talent hidden so others saved from envy), dd can be run in different ways, why didn't you just add a snippet showing how exactly it was run?
P. S. Generally I can advice updating kernel, because it could be specific driver quirks that got (or didn't) resolved.
answered Apr 20 at 9:39
poige
3,8571438
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
With all optimizations it still uses almost all available RAM (16 GB)
it — who? Typically every general purpose OS designed since 1970s uses all available RAM or its huge part for slow storage content caching intensively. If it means VM cache what troubles are there? Run free -m and study its output, it used to have special indication of "+/- cache/buffers" hinting that it's not irrevocable used — cache would shrink if there's memory pressure.
It's pretty typical misconception a novice would have…
I found that I should change I/O scheduler type to deadline
Schedulers are there for slow devices like HDDs. With SSDs/NVME it's just extra overhead — you don't need to have queue for requests because there's no gain in re-ordering them — contrary to HDDs where it plays significant role reducing seek times.
How I can solve these freezes during I/O load?
There're no mindreaders here (being an exception I prefer to keep my talent hidden so others saved from envy), dd can be run in different ways, why didn't you just add a snippet showing how exactly it was run?
P. S. Generally I can advice updating kernel, because it could be specific driver quirks that got (or didn't) resolved.
answered Apr 20 at 9:39
poige
3,8571438
add a comment |Â
up vote
0
down vote
With all optimizations it still uses almost all available RAM (16 GB)
it — who? Typically every general purpose OS designed since 1970s uses all available RAM or its huge part for slow storage content caching intensively. If it means VM cache what troubles are there? Run free -m and study its output, it used to have special indication of "+/- cache/buffers" hinting that it's not irrevocable used — cache would shrink if there's memory pressure.
It's pretty typical misconception a novice would have…
I found that I should change I/O scheduler type to deadline
Schedulers are there for slow devices like HDDs. With SSDs/NVME it's just extra overhead — you don't need to have queue for requests because there's no gain in re-ordering them — contrary to HDDs where it plays significant role reducing seek times.
How I can solve these freezes during I/O load?
There're no mindreaders here (being an exception I prefer to keep my talent hidden so others saved from envy), dd can be run in different ways, why didn't you just add a snippet showing how exactly it was run?
P. S. Generally I can advice updating kernel, because it could be specific driver quirks that got (or didn't) resolved.
answered Apr 20 at 9:39
poige
3,8571438
add a comment |Â
up vote
0
down vote
up vote
0
down vote
With all optimizations it still uses almost all available RAM (16 GB)
it — who? Typically every general purpose OS designed since 1970s uses all available RAM or its huge part for slow storage content caching intensively. If it means VM cache what troubles are there? Run free -m and study its output, it used to have special indication of "+/- cache/buffers" hinting that it's not irrevocable used — cache would shrink if there's memory pressure.
It's pretty typical misconception a novice would have…
I found that I should change I/O scheduler type to deadline
Schedulers are there for slow devices like HDDs. With SSDs/NVME it's just extra overhead — you don't need to have queue for requests because there's no gain in re-ordering them — contrary to HDDs where it plays significant role reducing seek times.
How I can solve these freezes during I/O load?
There're no mindreaders here (being an exception I prefer to keep my talent hidden so others saved from envy), dd can be run in different ways, why didn't you just add a snippet showing how exactly it was run?
P. S. Generally I can advice updating kernel, because it could be specific driver quirks that got (or didn't) resolved.
answered Apr 20 at 9:39
poige
3,8571438
With all optimizations it still uses almost all available RAM (16 GB)
it — who? Typically every general purpose OS designed since 1970s uses all available RAM or its huge part for slow storage content caching intensively. If it means VM cache what troubles are there? Run free -m and study its output, it used to have special indication of "+/- cache/buffers" hinting that it's not irrevocable used — cache would shrink if there's memory pressure.
It's pretty typical misconception a novice would have…
I found that I should change I/O scheduler type to deadline
Schedulers are there for slow devices like HDDs. With SSDs/NVME it's just extra overhead — you don't need to have queue for requests because there's no gain in re-ordering them — contrary to HDDs where it plays significant role reducing seek times.
How I can solve these freezes during I/O load?
There're no mindreaders here (being an exception I prefer to keep my talent hidden so others saved from envy), dd can be run in different ways, why didn't you just add a snippet showing how exactly it was run?
P. S. Generally I can advice updating kernel, because it could be specific driver quirks that got (or didn't) resolved.
answered Apr 20 at 9:39
poige
3,8571438
edited Apr 20 at 11:18
answered Apr 20 at 9:39
poige
3,8571438
answered Apr 20 at 9:39
poige
3,8571438
answered Apr 20 at 9:39
poige
3,8571438
3,8571438
add a comment |Â
add a comment |Â
I can keep a
ddin the background writing to disk with no problems. Assuming your hard drive isn't failing, you could try usingioniceto allow other programs to have I/O priority.– dsstorefile1
Apr 18 at 22:54
please show the exact
ddcommand you are using to test. Also if you could show thetopheaders (first 5 lines) that would help as well.– Patrick
Apr 18 at 23:34
I cannot remember anymore how many exchanges I had with Linus and Alan on this. Over the past 15+ years I have been going back and forth in many bugzillas trying to hammer home the idea that scheduler freezing the OS when someone copies a large file is a bad, bad implementation. These bug reports have for the most part been closed due to "end of life" and never looked into. Linux freezing during prolonged IO operations is a black eye for both Torvalds and Cox for the last couple decades.
– ajeh
Apr 19 at 16:10