Safe reboot cron job with fallback on SysRq reset issue
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I'm trying to setup a cron job to reboot devices daily. With a safe callback to a SysRq reset if for some reason the reboot does hang (issue being that SSH gets killed and the device never reboots so it is lost and requires costly human intervention to restart).
The script that used to work for a while:
5 5 * * * root /sbin/reboot -f; sleep 30; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log; echo 1 > /proc/sys/kernel/sysrq; sync; echo b > /proc/sysrq-trigger
However it's pretty brutal (hard reboot -f) and some of our devices did not recover recently (a couple over thousands every day).
Not sure what hangs (looks like the file is never written so I'd say either the reboot itself or the echo hangs?
Was looking to use ampersands & to never "lock" and be sure that a proper reset will happen eventually, however it does not seem to work at all (no more reboots):
5 5 * * * root /sbin/shutdown -r +2 &; sleep 240; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log &; echo 1 > /proc/sys/kernel/sysrq; sleep 1; echo b > /proc/sysrq-trigger
Can I use the ampersand in a cron script?
Do you know another smarter way to achieve the desired results?
Thanks!
cron reboot
asked Nov 27 '17 at 14:03
Olivier
101
add a comment |Â
up vote
0
down vote
favorite
I'm trying to setup a cron job to reboot devices daily. With a safe callback to a SysRq reset if for some reason the reboot does hang (issue being that SSH gets killed and the device never reboots so it is lost and requires costly human intervention to restart).
The script that used to work for a while:
5 5 * * * root /sbin/reboot -f; sleep 30; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log; echo 1 > /proc/sys/kernel/sysrq; sync; echo b > /proc/sysrq-trigger
However it's pretty brutal (hard reboot -f) and some of our devices did not recover recently (a couple over thousands every day).
Not sure what hangs (looks like the file is never written so I'd say either the reboot itself or the echo hangs?
Was looking to use ampersands & to never "lock" and be sure that a proper reset will happen eventually, however it does not seem to work at all (no more reboots):
5 5 * * * root /sbin/shutdown -r +2 &; sleep 240; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log &; echo 1 > /proc/sys/kernel/sysrq; sleep 1; echo b > /proc/sysrq-trigger
Can I use the ampersand in a cron script?
Do you know another smarter way to achieve the desired results?
Thanks!
cron reboot
asked Nov 27 '17 at 14:03
Olivier
101
1
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to setup a cron job to reboot devices daily. With a safe callback to a SysRq reset if for some reason the reboot does hang (issue being that SSH gets killed and the device never reboots so it is lost and requires costly human intervention to restart).
The script that used to work for a while:
5 5 * * * root /sbin/reboot -f; sleep 30; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log; echo 1 > /proc/sys/kernel/sysrq; sync; echo b > /proc/sysrq-trigger
However it's pretty brutal (hard reboot -f) and some of our devices did not recover recently (a couple over thousands every day).
Not sure what hangs (looks like the file is never written so I'd say either the reboot itself or the echo hangs?
Was looking to use ampersands & to never "lock" and be sure that a proper reset will happen eventually, however it does not seem to work at all (no more reboots):
5 5 * * * root /sbin/shutdown -r +2 &; sleep 240; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log &; echo 1 > /proc/sys/kernel/sysrq; sleep 1; echo b > /proc/sysrq-trigger
Can I use the ampersand in a cron script?
Do you know another smarter way to achieve the desired results?
Thanks!
cron reboot
asked Nov 27 '17 at 14:03
Olivier
101
I'm trying to setup a cron job to reboot devices daily. With a safe callback to a SysRq reset if for some reason the reboot does hang (issue being that SSH gets killed and the device never reboots so it is lost and requires costly human intervention to restart).
The script that used to work for a while:
5 5 * * * root /sbin/reboot -f; sleep 30; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log; echo 1 > /proc/sys/kernel/sysrq; sync; echo b > /proc/sysrq-trigger
However it's pretty brutal (hard reboot -f) and some of our devices did not recover recently (a couple over thousands every day).
Not sure what hangs (looks like the file is never written so I'd say either the reboot itself or the echo hangs?
Was looking to use ampersands & to never "lock" and be sure that a proper reset will happen eventually, however it does not seem to work at all (no more reboots):
5 5 * * * root /sbin/shutdown -r +2 &; sleep 240; /bin/echo `date -u +'%Y-%m-%dT%H:%M:%SZ'` >> /var/log/player-reboot.error.log &; echo 1 > /proc/sys/kernel/sysrq; sleep 1; echo b > /proc/sysrq-trigger
Can I use the ampersand in a cron script?
Do you know another smarter way to achieve the desired results?
Thanks!
cron reboot
asked Nov 27 '17 at 14:03
Olivier
101
asked Nov 27 '17 at 14:03
Olivier
101
asked Nov 27 '17 at 14:03
Olivier
101
asked Nov 27 '17 at 14:03
Olivier
101
101
1
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01
add a comment |Â
1
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01
1
1
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f407291%2fsafe-reboot-cron-job-with-fallback-on-sysrq-reset-issue%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Not sure what the current issue is, but I'd be tempted to write a single script, which triggers the various things. I'm always wary of essentially writing mini-scripts as the cron command itself, easier IMO to write a proper script and just call that.
– EightBitTony
Nov 27 '17 at 14:25
Assuming your machine uses systemd, I don't think you can background (&) a call to '/sbin/shutdown'. It's essentially an internal call to an AF_UNIX socket that implements the shutdown process. If you're on a machine that uses 'telinit' (i.e., Upstart or SysV init), you could use '/sbin/telinit 6' followed by the rest of your invocation. But a script would be better.
– Thomas N
Nov 27 '17 at 18:01