Get CPU usage and run a command if it is higher than 80%

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
2
down vote

favorite












My VPS was hacked several times, hackers put a CPU miner. My hosting provider shutdowns VPS if miner is detected and I did not reactin the next 12 hours. But they can send me notice at 21.00 PM on Saturday :) , and shutdown server at 9.00 AM on Sunday.



So I want to monitor CPU usage and block folder where miner is always revelead from writing.



I'm not very familiar with Linux, so please suggest with such script



  1. Check CPU usage, if it is higher than 80% (as example) Do something.

  2. In my case - delete all from install folder and make it read only.

Actually I've no idea how to implement item №1.







share|improve this question






















  • You may wish to take a look at uptime
    – Weijun Zhou
    Jan 15 at 9:43










  • And how would that help?
    – Murphy
    Jan 15 at 9:46










  • The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
    – Weijun Zhou
    Jan 15 at 9:49






  • 1




    Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
    – Weijun Zhou
    Jan 15 at 9:52






  • 1




    If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
    – Gerard H. Pille
    Jan 15 at 9:53














up vote
2
down vote

favorite












My VPS was hacked several times, hackers put a CPU miner. My hosting provider shutdowns VPS if miner is detected and I did not reactin the next 12 hours. But they can send me notice at 21.00 PM on Saturday :) , and shutdown server at 9.00 AM on Sunday.



So I want to monitor CPU usage and block folder where miner is always revelead from writing.



I'm not very familiar with Linux, so please suggest with such script



  1. Check CPU usage, if it is higher than 80% (as example) Do something.

  2. In my case - delete all from install folder and make it read only.

Actually I've no idea how to implement item №1.







share|improve this question






















  • You may wish to take a look at uptime
    – Weijun Zhou
    Jan 15 at 9:43










  • And how would that help?
    – Murphy
    Jan 15 at 9:46










  • The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
    – Weijun Zhou
    Jan 15 at 9:49






  • 1




    Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
    – Weijun Zhou
    Jan 15 at 9:52






  • 1




    If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
    – Gerard H. Pille
    Jan 15 at 9:53












up vote
2
down vote

favorite









up vote
2
down vote

favorite











My VPS was hacked several times, hackers put a CPU miner. My hosting provider shutdowns VPS if miner is detected and I did not reactin the next 12 hours. But they can send me notice at 21.00 PM on Saturday :) , and shutdown server at 9.00 AM on Sunday.



So I want to monitor CPU usage and block folder where miner is always revelead from writing.



I'm not very familiar with Linux, so please suggest with such script



  1. Check CPU usage, if it is higher than 80% (as example) Do something.

  2. In my case - delete all from install folder and make it read only.

Actually I've no idea how to implement item №1.







share|improve this question














My VPS was hacked several times, hackers put a CPU miner. My hosting provider shutdowns VPS if miner is detected and I did not reactin the next 12 hours. But they can send me notice at 21.00 PM on Saturday :) , and shutdown server at 9.00 AM on Sunday.



So I want to monitor CPU usage and block folder where miner is always revelead from writing.



I'm not very familiar with Linux, so please suggest with such script



  1. Check CPU usage, if it is higher than 80% (as example) Do something.

  2. In my case - delete all from install folder and make it read only.

Actually I've no idea how to implement item №1.









share|improve this question













share|improve this question




share|improve this question








edited Jan 15 at 10:46









Jeff Schaller

31.8k848109




31.8k848109










asked Jan 15 at 9:37









Alexey Shatrov

1114




1114











  • You may wish to take a look at uptime
    – Weijun Zhou
    Jan 15 at 9:43










  • And how would that help?
    – Murphy
    Jan 15 at 9:46










  • The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
    – Weijun Zhou
    Jan 15 at 9:49






  • 1




    Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
    – Weijun Zhou
    Jan 15 at 9:52






  • 1




    If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
    – Gerard H. Pille
    Jan 15 at 9:53
















  • You may wish to take a look at uptime
    – Weijun Zhou
    Jan 15 at 9:43










  • And how would that help?
    – Murphy
    Jan 15 at 9:46










  • The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
    – Weijun Zhou
    Jan 15 at 9:49






  • 1




    Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
    – Weijun Zhou
    Jan 15 at 9:52






  • 1




    If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
    – Gerard H. Pille
    Jan 15 at 9:53















You may wish to take a look at uptime
– Weijun Zhou
Jan 15 at 9:43




You may wish to take a look at uptime
– Weijun Zhou
Jan 15 at 9:43












And how would that help?
– Murphy
Jan 15 at 9:46




And how would that help?
– Murphy
Jan 15 at 9:46












The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
– Weijun Zhou
Jan 15 at 9:49




The load average roughly tells you to what extent the CPU is occupied, although they are not the same.
– Weijun Zhou
Jan 15 at 9:49




1




1




Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
– Weijun Zhou
Jan 15 at 9:52




Another solution. If you want to monitor the system and run a specific command when a certain condition about system load is satisfied, you can try installing monit. This command is able to handle CPU usage and disk IO and many others.
– Weijun Zhou
Jan 15 at 9:52




1




1




If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
– Gerard H. Pille
Jan 15 at 9:53




If you go for uptime, take the number of processors into account. From the man's page: Load averages are not normalized for the number of CPUs in a system, so a load average of 1 means a single CPU system is loaded all the time while on a 4 CPU system it means it was idle 75% of the time.
– Gerard H. Pille
Jan 15 at 9:53










2 Answers
2






active

oldest

votes

















up vote
3
down vote













I had a similar issue and had this short bash script already done.
It is calculating the load average for the last 15 minutes, if you want a different timeframe, it shold be change (to check the load avg for last 5 min, change the awk to print $1).

This will tell you the relative usage of the CPUs :



#!/bin/bash
cores=$(nproc)
load=$(awk 'print $3'< /proc/loadavg)
echo | awk -v c="$cores" -v l="$load" 'print "relative load is " l*100/c "%"'


Should run on Ubuntu and Centos.



To get to the point where you check if load is above 80% and 'do something' you should add to this script :



usage=$(echo | awk -v c="$cores" -v l="$load" 'print l*100/c' | awk -F. 'print $1')
if [[ $usage -ge 80 ]]; then
echo "delete all from install folder and make it read only"
fi





share|improve this answer


















  • 1




    good idea, done
    – mazs
    Jan 15 at 10:55










  • I'll try it in a couple of days. Thanks.
    – Alexey Shatrov
    Jan 15 at 18:16

















up vote
0
down vote














Check CPU usage, if it is higher than 80% (as example) Do something.




Modern CPUs have multiple cores and often each core supports multiple threads. Moreover, cores have often variable clock speed. It is then not that simple to define how to compute a CPU load. Not to mention a single vCPU can only be either 100% idle or 100% busy at any given time. There is no such thing as instantaneous 80% busy CPU.



What you can do is get the average load for each vCPU during a period of time (mpstat provides that) or the average for all vCPUs combined (vmstat).



Even fully CPU bound, if the hostile CPU consumer is single-threaded, it might not blatantly show up in the latter case because other vCPUs might be idle.



If multi threaded and CPU bound, it will be detected by both commands, but you have to make sure it is not a legit application or daemon which is loading your machine.



Another more useful metric is derived from the CPU contention, i.e. measuring how many threads are using and competing for the vCPU resources. This is what the load average is designed to show. Unfortunately, on Linux, the load average is considering a thread uninterruptible state to be CPU load while in fact, the CPU is idle and free to do other tasks so you should pay attention to that factor and identify potential cases where the load average is high but the actual contention is low.



Finally, there might have situations where the run queue is very high but for a very limited period of time. If the load average calculation which use sampling to get the run queue value happen to pick the number at this very peak moment, the load average value will be strongly biased for several minutes or dozens of minutes.






share|improve this answer






















  • Mining process takes all avaible CPU to 100%.
    – Alexey Shatrov
    Jan 15 at 18:15






  • 1




    Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
    – jlliagre
    Jan 15 at 20:38










Your Answer







StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);








 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f417209%2fget-cpu-usage-and-run-a-command-if-it-is-higher-than-80%23new-answer', 'question_page');

);

Post as a guest






























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
3
down vote













I had a similar issue and had this short bash script already done.
It is calculating the load average for the last 15 minutes, if you want a different timeframe, it shold be change (to check the load avg for last 5 min, change the awk to print $1).

This will tell you the relative usage of the CPUs :



#!/bin/bash
cores=$(nproc)
load=$(awk 'print $3'< /proc/loadavg)
echo | awk -v c="$cores" -v l="$load" 'print "relative load is " l*100/c "%"'


Should run on Ubuntu and Centos.



To get to the point where you check if load is above 80% and 'do something' you should add to this script :



usage=$(echo | awk -v c="$cores" -v l="$load" 'print l*100/c' | awk -F. 'print $1')
if [[ $usage -ge 80 ]]; then
echo "delete all from install folder and make it read only"
fi





share|improve this answer


















  • 1




    good idea, done
    – mazs
    Jan 15 at 10:55










  • I'll try it in a couple of days. Thanks.
    – Alexey Shatrov
    Jan 15 at 18:16














up vote
3
down vote













I had a similar issue and had this short bash script already done.
It is calculating the load average for the last 15 minutes, if you want a different timeframe, it shold be change (to check the load avg for last 5 min, change the awk to print $1).

This will tell you the relative usage of the CPUs :



#!/bin/bash
cores=$(nproc)
load=$(awk 'print $3'< /proc/loadavg)
echo | awk -v c="$cores" -v l="$load" 'print "relative load is " l*100/c "%"'


Should run on Ubuntu and Centos.



To get to the point where you check if load is above 80% and 'do something' you should add to this script :



usage=$(echo | awk -v c="$cores" -v l="$load" 'print l*100/c' | awk -F. 'print $1')
if [[ $usage -ge 80 ]]; then
echo "delete all from install folder and make it read only"
fi





share|improve this answer


















  • 1




    good idea, done
    – mazs
    Jan 15 at 10:55










  • I'll try it in a couple of days. Thanks.
    – Alexey Shatrov
    Jan 15 at 18:16












up vote
3
down vote










up vote
3
down vote









I had a similar issue and had this short bash script already done.
It is calculating the load average for the last 15 minutes, if you want a different timeframe, it shold be change (to check the load avg for last 5 min, change the awk to print $1).

This will tell you the relative usage of the CPUs :



#!/bin/bash
cores=$(nproc)
load=$(awk 'print $3'< /proc/loadavg)
echo | awk -v c="$cores" -v l="$load" 'print "relative load is " l*100/c "%"'


Should run on Ubuntu and Centos.



To get to the point where you check if load is above 80% and 'do something' you should add to this script :



usage=$(echo | awk -v c="$cores" -v l="$load" 'print l*100/c' | awk -F. 'print $1')
if [[ $usage -ge 80 ]]; then
echo "delete all from install folder and make it read only"
fi





share|improve this answer














I had a similar issue and had this short bash script already done.
It is calculating the load average for the last 15 minutes, if you want a different timeframe, it shold be change (to check the load avg for last 5 min, change the awk to print $1).

This will tell you the relative usage of the CPUs :



#!/bin/bash
cores=$(nproc)
load=$(awk 'print $3'< /proc/loadavg)
echo | awk -v c="$cores" -v l="$load" 'print "relative load is " l*100/c "%"'


Should run on Ubuntu and Centos.



To get to the point where you check if load is above 80% and 'do something' you should add to this script :



usage=$(echo | awk -v c="$cores" -v l="$load" 'print l*100/c' | awk -F. 'print $1')
if [[ $usage -ge 80 ]]; then
echo "delete all from install folder and make it read only"
fi






share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 15 at 11:34

























answered Jan 15 at 10:22









mazs

2,5151522




2,5151522







  • 1




    good idea, done
    – mazs
    Jan 15 at 10:55










  • I'll try it in a couple of days. Thanks.
    – Alexey Shatrov
    Jan 15 at 18:16












  • 1




    good idea, done
    – mazs
    Jan 15 at 10:55










  • I'll try it in a couple of days. Thanks.
    – Alexey Shatrov
    Jan 15 at 18:16







1




1




good idea, done
– mazs
Jan 15 at 10:55




good idea, done
– mazs
Jan 15 at 10:55












I'll try it in a couple of days. Thanks.
– Alexey Shatrov
Jan 15 at 18:16




I'll try it in a couple of days. Thanks.
– Alexey Shatrov
Jan 15 at 18:16












up vote
0
down vote














Check CPU usage, if it is higher than 80% (as example) Do something.




Modern CPUs have multiple cores and often each core supports multiple threads. Moreover, cores have often variable clock speed. It is then not that simple to define how to compute a CPU load. Not to mention a single vCPU can only be either 100% idle or 100% busy at any given time. There is no such thing as instantaneous 80% busy CPU.



What you can do is get the average load for each vCPU during a period of time (mpstat provides that) or the average for all vCPUs combined (vmstat).



Even fully CPU bound, if the hostile CPU consumer is single-threaded, it might not blatantly show up in the latter case because other vCPUs might be idle.



If multi threaded and CPU bound, it will be detected by both commands, but you have to make sure it is not a legit application or daemon which is loading your machine.



Another more useful metric is derived from the CPU contention, i.e. measuring how many threads are using and competing for the vCPU resources. This is what the load average is designed to show. Unfortunately, on Linux, the load average is considering a thread uninterruptible state to be CPU load while in fact, the CPU is idle and free to do other tasks so you should pay attention to that factor and identify potential cases where the load average is high but the actual contention is low.



Finally, there might have situations where the run queue is very high but for a very limited period of time. If the load average calculation which use sampling to get the run queue value happen to pick the number at this very peak moment, the load average value will be strongly biased for several minutes or dozens of minutes.






share|improve this answer






















  • Mining process takes all avaible CPU to 100%.
    – Alexey Shatrov
    Jan 15 at 18:15






  • 1




    Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
    – jlliagre
    Jan 15 at 20:38














up vote
0
down vote














Check CPU usage, if it is higher than 80% (as example) Do something.




Modern CPUs have multiple cores and often each core supports multiple threads. Moreover, cores have often variable clock speed. It is then not that simple to define how to compute a CPU load. Not to mention a single vCPU can only be either 100% idle or 100% busy at any given time. There is no such thing as instantaneous 80% busy CPU.



What you can do is get the average load for each vCPU during a period of time (mpstat provides that) or the average for all vCPUs combined (vmstat).



Even fully CPU bound, if the hostile CPU consumer is single-threaded, it might not blatantly show up in the latter case because other vCPUs might be idle.



If multi threaded and CPU bound, it will be detected by both commands, but you have to make sure it is not a legit application or daemon which is loading your machine.



Another more useful metric is derived from the CPU contention, i.e. measuring how many threads are using and competing for the vCPU resources. This is what the load average is designed to show. Unfortunately, on Linux, the load average is considering a thread uninterruptible state to be CPU load while in fact, the CPU is idle and free to do other tasks so you should pay attention to that factor and identify potential cases where the load average is high but the actual contention is low.



Finally, there might have situations where the run queue is very high but for a very limited period of time. If the load average calculation which use sampling to get the run queue value happen to pick the number at this very peak moment, the load average value will be strongly biased for several minutes or dozens of minutes.






share|improve this answer






















  • Mining process takes all avaible CPU to 100%.
    – Alexey Shatrov
    Jan 15 at 18:15






  • 1




    Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
    – jlliagre
    Jan 15 at 20:38












up vote
0
down vote










up vote
0
down vote










Check CPU usage, if it is higher than 80% (as example) Do something.




Modern CPUs have multiple cores and often each core supports multiple threads. Moreover, cores have often variable clock speed. It is then not that simple to define how to compute a CPU load. Not to mention a single vCPU can only be either 100% idle or 100% busy at any given time. There is no such thing as instantaneous 80% busy CPU.



What you can do is get the average load for each vCPU during a period of time (mpstat provides that) or the average for all vCPUs combined (vmstat).



Even fully CPU bound, if the hostile CPU consumer is single-threaded, it might not blatantly show up in the latter case because other vCPUs might be idle.



If multi threaded and CPU bound, it will be detected by both commands, but you have to make sure it is not a legit application or daemon which is loading your machine.



Another more useful metric is derived from the CPU contention, i.e. measuring how many threads are using and competing for the vCPU resources. This is what the load average is designed to show. Unfortunately, on Linux, the load average is considering a thread uninterruptible state to be CPU load while in fact, the CPU is idle and free to do other tasks so you should pay attention to that factor and identify potential cases where the load average is high but the actual contention is low.



Finally, there might have situations where the run queue is very high but for a very limited period of time. If the load average calculation which use sampling to get the run queue value happen to pick the number at this very peak moment, the load average value will be strongly biased for several minutes or dozens of minutes.






share|improve this answer















Check CPU usage, if it is higher than 80% (as example) Do something.




Modern CPUs have multiple cores and often each core supports multiple threads. Moreover, cores have often variable clock speed. It is then not that simple to define how to compute a CPU load. Not to mention a single vCPU can only be either 100% idle or 100% busy at any given time. There is no such thing as instantaneous 80% busy CPU.



What you can do is get the average load for each vCPU during a period of time (mpstat provides that) or the average for all vCPUs combined (vmstat).



Even fully CPU bound, if the hostile CPU consumer is single-threaded, it might not blatantly show up in the latter case because other vCPUs might be idle.



If multi threaded and CPU bound, it will be detected by both commands, but you have to make sure it is not a legit application or daemon which is loading your machine.



Another more useful metric is derived from the CPU contention, i.e. measuring how many threads are using and competing for the vCPU resources. This is what the load average is designed to show. Unfortunately, on Linux, the load average is considering a thread uninterruptible state to be CPU load while in fact, the CPU is idle and free to do other tasks so you should pay attention to that factor and identify potential cases where the load average is high but the actual contention is low.



Finally, there might have situations where the run queue is very high but for a very limited period of time. If the load average calculation which use sampling to get the run queue value happen to pick the number at this very peak moment, the load average value will be strongly biased for several minutes or dozens of minutes.







share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 15 at 14:27

























answered Jan 15 at 11:34









jlliagre

44.9k578124




44.9k578124











  • Mining process takes all avaible CPU to 100%.
    – Alexey Shatrov
    Jan 15 at 18:15






  • 1




    Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
    – jlliagre
    Jan 15 at 20:38
















  • Mining process takes all avaible CPU to 100%.
    – Alexey Shatrov
    Jan 15 at 18:15






  • 1




    Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
    – jlliagre
    Jan 15 at 20:38















Mining process takes all avaible CPU to 100%.
– Alexey Shatrov
Jan 15 at 18:15




Mining process takes all avaible CPU to 100%.
– Alexey Shatrov
Jan 15 at 18:15




1




1




Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
– jlliagre
Jan 15 at 20:38




Okay, that simplify the detection. However, given the fact your machine was hacked several times, odds are high hackers still have access to it. The next step would be to reinstall it from scratch with up to date applications/libraries/whatever you run on it, change all passwords and limit the reachable services to what you really need.
– jlliagre
Jan 15 at 20:38












 

draft saved


draft discarded


























 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f417209%2fget-cpu-usage-and-run-a-command-if-it-is-higher-than-80%23new-answer', 'question_page');

);

Post as a guest













































































Popular posts from this blog

Peggy Mitchell

Palaiologos

The Forum (Inglewood, California)