Rule out hard disk performance as bottleneck in elasticsearch indexing [on hold]
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
our application is using elasticsearch for log indexing, and we've got very different performance on similar machines (All CentOs7). So my question is, how to i rule out the bottleneck is the hard disk? there is barely IO wait in the slow installation (very high load + cpu usage).
Here's a snapshot:
iostat 5
Linux 3.10.0-693.2.2.el7.centos.plus.x86_64 (tracer2) 2018-10-02 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
35,51 22,29 2,01 0,22 0,00 39,96
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 181,99 125,30 11490,17 129009633 11830256952
avg-cpu: %user %nice %system %iowait %steal %idle
43,15 50,56 2,83 0,30 0,00 3,16
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 322,60 0,00 14434,40 0 72172
avg-cpu: %user %nice %system %iowait %steal %idle
52,43 41,70 2,73 0,03 0,00 3,11
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 283,40 0,00 5685,60 0 28428
avg-cpu: %user %nice %system %iowait %steal %idle
52,77 43,05 2,36 0,03 0,00 1,80
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 320,40 0,00 31557,60 0 157788
so is this enough to rule out HD as the troublemaker?
performance elasticsearch
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as too broad by Rui F Ribeiro, Kiwy, Goro, RalfFriedl, Thomas 2 days ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |Â
up vote
0
down vote
favorite
our application is using elasticsearch for log indexing, and we've got very different performance on similar machines (All CentOs7). So my question is, how to i rule out the bottleneck is the hard disk? there is barely IO wait in the slow installation (very high load + cpu usage).
Here's a snapshot:
iostat 5
Linux 3.10.0-693.2.2.el7.centos.plus.x86_64 (tracer2) 2018-10-02 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
35,51 22,29 2,01 0,22 0,00 39,96
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 181,99 125,30 11490,17 129009633 11830256952
avg-cpu: %user %nice %system %iowait %steal %idle
43,15 50,56 2,83 0,30 0,00 3,16
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 322,60 0,00 14434,40 0 72172
avg-cpu: %user %nice %system %iowait %steal %idle
52,43 41,70 2,73 0,03 0,00 3,11
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 283,40 0,00 5685,60 0 28428
avg-cpu: %user %nice %system %iowait %steal %idle
52,77 43,05 2,36 0,03 0,00 1,80
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 320,40 0,00 31557,60 0 157788
so is this enough to rule out HD as the troublemaker?
performance elasticsearch
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as too broad by Rui F Ribeiro, Kiwy, Goro, RalfFriedl, Thomas 2 days ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
1
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to useatsarit's a great monitoring tool.
– Kiwy
Oct 4 at 11:39
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
our application is using elasticsearch for log indexing, and we've got very different performance on similar machines (All CentOs7). So my question is, how to i rule out the bottleneck is the hard disk? there is barely IO wait in the slow installation (very high load + cpu usage).
Here's a snapshot:
iostat 5
Linux 3.10.0-693.2.2.el7.centos.plus.x86_64 (tracer2) 2018-10-02 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
35,51 22,29 2,01 0,22 0,00 39,96
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 181,99 125,30 11490,17 129009633 11830256952
avg-cpu: %user %nice %system %iowait %steal %idle
43,15 50,56 2,83 0,30 0,00 3,16
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 322,60 0,00 14434,40 0 72172
avg-cpu: %user %nice %system %iowait %steal %idle
52,43 41,70 2,73 0,03 0,00 3,11
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 283,40 0,00 5685,60 0 28428
avg-cpu: %user %nice %system %iowait %steal %idle
52,77 43,05 2,36 0,03 0,00 1,80
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 320,40 0,00 31557,60 0 157788
so is this enough to rule out HD as the troublemaker?
performance elasticsearch
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
our application is using elasticsearch for log indexing, and we've got very different performance on similar machines (All CentOs7). So my question is, how to i rule out the bottleneck is the hard disk? there is barely IO wait in the slow installation (very high load + cpu usage).
Here's a snapshot:
iostat 5
Linux 3.10.0-693.2.2.el7.centos.plus.x86_64 (tracer2) 2018-10-02 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
35,51 22,29 2,01 0,22 0,00 39,96
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 181,99 125,30 11490,17 129009633 11830256952
avg-cpu: %user %nice %system %iowait %steal %idle
43,15 50,56 2,83 0,30 0,00 3,16
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 322,60 0,00 14434,40 0 72172
avg-cpu: %user %nice %system %iowait %steal %idle
52,43 41,70 2,73 0,03 0,00 3,11
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 283,40 0,00 5685,60 0 28428
avg-cpu: %user %nice %system %iowait %steal %idle
52,77 43,05 2,36 0,03 0,00 1,80
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 320,40 0,00 31557,60 0 157788
so is this enough to rule out HD as the troublemaker?
performance elasticsearch
performance elasticsearch
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Oct 4 at 8:58
kertal
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Oct 4 at 8:58
kertal
42
asked Oct 4 at 8:58
kertal
42
42
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
kertal is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as too broad by Rui F Ribeiro, Kiwy, Goro, RalfFriedl, Thomas 2 days ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
put on hold as too broad by Rui F Ribeiro, Kiwy, Goro, RalfFriedl, Thomas 2 days ago
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
1
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to useatsarit's a great monitoring tool.
– Kiwy
Oct 4 at 11:39
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46
add a comment |Â
1
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to useatsarit's a great monitoring tool.
– Kiwy
Oct 4 at 11:39
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46
1
1
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to use
atsar it's a great monitoring tool.– Kiwy
Oct 4 at 11:39
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to use
atsar it's a great monitoring tool.– Kiwy
Oct 4 at 11:39
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
That kind of question is barely answerable. In this case we would need the following information: complete description of the hardware including motherboard / connector to disk / type of disk / how much are they ? in which Raid configuration, What are the raw disk IO ? I could Ask a dozen more question and if you answered them perfectly, I would probably not be able to answer you yet. I think your question needs too much expertise and debate to be answered in that kind of site. Finding bottle neck in a system can be real hard. I would advise you to use
atsarit's a great monitoring tool.– Kiwy
Oct 4 at 11:39
Thank you, can't find anything about atsar, could you provide a link? what's a good tool for measuring raw disk IO?
– kertal
Oct 4 at 12:46