Find command to check only for the last 10 minutes and not the whole folder
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I have an nvr system that recording video surveillance into a file server (Debian). I made a Nagios plugin to check if the NVR system is recording correctly and send me a notification when it stops doing that. The issue is I'm using a find command:
find /srv/unifi-video/videos/ -name '*.ts' -mmin -10 | wc -l
and since it's checking through 400 Gb+ of files Nagios keeps timing out and sending "NRPE Socket timed out" messages.
Is it possible to make the find command to search only for the files created in the passed 10 minutes and not only the whole folder ?
linux find nagios nrpe
asked Sep 22 '15 at 16:16
Youssef Karami
111
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |Â
up vote
0
down vote
favorite
I have an nvr system that recording video surveillance into a file server (Debian). I made a Nagios plugin to check if the NVR system is recording correctly and send me a notification when it stops doing that. The issue is I'm using a find command:
find /srv/unifi-video/videos/ -name '*.ts' -mmin -10 | wc -l
and since it's checking through 400 Gb+ of files Nagios keeps timing out and sending "NRPE Socket timed out" messages.
Is it possible to make the find command to search only for the files created in the passed 10 minutes and not only the whole folder ?
linux find nagios nrpe
asked Sep 22 '15 at 16:16
Youssef Karami
111
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
What do you mean by “not the whole folderâ€� Do you mean that the files are created in/srv/unifi-video/videos, not a subdirectory, and you don't wantfindto explore the subdirectories? If it's not that, what do you mean?
– Gilles
Sep 22 '15 at 22:11
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have an nvr system that recording video surveillance into a file server (Debian). I made a Nagios plugin to check if the NVR system is recording correctly and send me a notification when it stops doing that. The issue is I'm using a find command:
find /srv/unifi-video/videos/ -name '*.ts' -mmin -10 | wc -l
and since it's checking through 400 Gb+ of files Nagios keeps timing out and sending "NRPE Socket timed out" messages.
Is it possible to make the find command to search only for the files created in the passed 10 minutes and not only the whole folder ?
linux find nagios nrpe
asked Sep 22 '15 at 16:16
Youssef Karami
111
I have an nvr system that recording video surveillance into a file server (Debian). I made a Nagios plugin to check if the NVR system is recording correctly and send me a notification when it stops doing that. The issue is I'm using a find command:
find /srv/unifi-video/videos/ -name '*.ts' -mmin -10 | wc -l
and since it's checking through 400 Gb+ of files Nagios keeps timing out and sending "NRPE Socket timed out" messages.
Is it possible to make the find command to search only for the files created in the passed 10 minutes and not only the whole folder ?
linux find nagios nrpe
linux find nagios nrpe
asked Sep 22 '15 at 16:16
Youssef Karami
111
asked Sep 22 '15 at 16:16
Youssef Karami
111
asked Sep 22 '15 at 16:16
Youssef Karami
111
asked Sep 22 '15 at 16:16
Youssef Karami
111
asked Sep 22 '15 at 16:16
Youssef Karami
111
111
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
What do you mean by “not the whole folderâ€� Do you mean that the files are created in/srv/unifi-video/videos, not a subdirectory, and you don't wantfindto explore the subdirectories? If it's not that, what do you mean?
– Gilles
Sep 22 '15 at 22:11
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57
add a comment |Â
What do you mean by “not the whole folderâ€� Do you mean that the files are created in/srv/unifi-video/videos, not a subdirectory, and you don't wantfindto explore the subdirectories? If it's not that, what do you mean?
– Gilles
Sep 22 '15 at 22:11
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57
What do you mean by “not the whole folderâ€� Do you mean that the files are created in
/srv/unifi-video/videos, not a subdirectory, and you don't want find to explore the subdirectories? If it's not that, what do you mean?– Gilles
Sep 22 '15 at 22:11
What do you mean by “not the whole folderâ€� Do you mean that the files are created in
/srv/unifi-video/videos, not a subdirectory, and you don't want find to explore the subdirectories? If it's not that, what do you mean?– Gilles
Sep 22 '15 at 22:11
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
0
down vote
Your code already seems correct, -mmin -10 narrows results to "modified less than 10 minutes ago". If your videos/ also has other items such as subdirectories, you might be able to shave some time time by ignoring directories and focusing only on files, adding -type f:
find /srv/unifi-video/videos/ -type f -name '*.ts' -mmin -10 | wc -l
If you have any finer control over how your nvr system saves files, maybe try to make it save only *.ts files in a dedicated directory eg videos/ts somehow, so find only has to deal with ts files and nothing else.
Other Ideas
Hopefully you already tried looking into documentation for your nvr system to see if there was some logging feature that just needed to be enabled. Why I mention logging is because it would be faster to get status information if you just had to make a script to read logs, compared to a script using find to examine file system records.
answered Sep 22 '15 at 17:47
estudiante27
413
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try-type fso find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Usingtimelike thistime find -mmin -10, compared totime find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.
– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would usingrelatimeas a mount option work any better ? just a vague idea. though not sure you canfindusing access time
– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
add a comment |Â
up vote
0
down vote
I would first find the latest directory used and then check only that directory:
find $(ls -rt /srv/unifi-video/videos/* | tail -1) -type f -name '*.ts' -mmin -10 | wc -l
answered Nov 21 '17 at 7:46
xenoid
1,7051620
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Your code already seems correct, -mmin -10 narrows results to "modified less than 10 minutes ago". If your videos/ also has other items such as subdirectories, you might be able to shave some time time by ignoring directories and focusing only on files, adding -type f:
find /srv/unifi-video/videos/ -type f -name '*.ts' -mmin -10 | wc -l
If you have any finer control over how your nvr system saves files, maybe try to make it save only *.ts files in a dedicated directory eg videos/ts somehow, so find only has to deal with ts files and nothing else.
Other Ideas
Hopefully you already tried looking into documentation for your nvr system to see if there was some logging feature that just needed to be enabled. Why I mention logging is because it would be faster to get status information if you just had to make a script to read logs, compared to a script using find to examine file system records.
answered Sep 22 '15 at 17:47
estudiante27
413
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try-type fso find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Usingtimelike thistime find -mmin -10, compared totime find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.
– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would usingrelatimeas a mount option work any better ? just a vague idea. though not sure you canfindusing access time
– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
add a comment |Â
up vote
0
down vote
Your code already seems correct, -mmin -10 narrows results to "modified less than 10 minutes ago". If your videos/ also has other items such as subdirectories, you might be able to shave some time time by ignoring directories and focusing only on files, adding -type f:
find /srv/unifi-video/videos/ -type f -name '*.ts' -mmin -10 | wc -l
If you have any finer control over how your nvr system saves files, maybe try to make it save only *.ts files in a dedicated directory eg videos/ts somehow, so find only has to deal with ts files and nothing else.
Other Ideas
Hopefully you already tried looking into documentation for your nvr system to see if there was some logging feature that just needed to be enabled. Why I mention logging is because it would be faster to get status information if you just had to make a script to read logs, compared to a script using find to examine file system records.
answered Sep 22 '15 at 17:47
estudiante27
413
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try-type fso find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Usingtimelike thistime find -mmin -10, compared totime find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.
– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would usingrelatimeas a mount option work any better ? just a vague idea. though not sure you canfindusing access time
– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
add a comment |Â
up vote
0
down vote
up vote
0
down vote
Your code already seems correct, -mmin -10 narrows results to "modified less than 10 minutes ago". If your videos/ also has other items such as subdirectories, you might be able to shave some time time by ignoring directories and focusing only on files, adding -type f:
find /srv/unifi-video/videos/ -type f -name '*.ts' -mmin -10 | wc -l
If you have any finer control over how your nvr system saves files, maybe try to make it save only *.ts files in a dedicated directory eg videos/ts somehow, so find only has to deal with ts files and nothing else.
Other Ideas
Hopefully you already tried looking into documentation for your nvr system to see if there was some logging feature that just needed to be enabled. Why I mention logging is because it would be faster to get status information if you just had to make a script to read logs, compared to a script using find to examine file system records.
answered Sep 22 '15 at 17:47
estudiante27
413
Your code already seems correct, -mmin -10 narrows results to "modified less than 10 minutes ago". If your videos/ also has other items such as subdirectories, you might be able to shave some time time by ignoring directories and focusing only on files, adding -type f:
find /srv/unifi-video/videos/ -type f -name '*.ts' -mmin -10 | wc -l
If you have any finer control over how your nvr system saves files, maybe try to make it save only *.ts files in a dedicated directory eg videos/ts somehow, so find only has to deal with ts files and nothing else.
Other Ideas
Hopefully you already tried looking into documentation for your nvr system to see if there was some logging feature that just needed to be enabled. Why I mention logging is because it would be faster to get status information if you just had to make a script to read logs, compared to a script using find to examine file system records.
answered Sep 22 '15 at 17:47
estudiante27
413
answered Sep 22 '15 at 17:47
estudiante27
413
answered Sep 22 '15 at 17:47
estudiante27
413
answered Sep 22 '15 at 17:47
estudiante27
413
413
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try-type fso find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Usingtimelike thistime find -mmin -10, compared totime find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.
– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would usingrelatimeas a mount option work any better ? just a vague idea. though not sure you canfindusing access time
– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
add a comment |Â
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try-type fso find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Usingtimelike thistime find -mmin -10, compared totime find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.
– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would usingrelatimeas a mount option work any better ? just a vague idea. though not sure you canfindusing access time
– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
I checked and this NVR system is creating random folder after the path like 00-558-5282.... and then sub-directories for dates... I think what makes the command to take forever to execute is because it's accessing all folders that available and it's supposed to do that since I'm not sure which one is NVR using...
– Youssef Karami
Sep 22 '15 at 18:06
In that case yes, try
-type f so find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Using time like this time find -mmin -10, compared to time find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.– estudiante27
Sep 22 '15 at 18:17
In that case yes, try
-type f so find will only focus on files. I did a test on a local directory with many, many subdirectories, and I only modified a file deep in the structure less than 10 minutes ago. Using time like this time find -mmin -10, compared to time find -type f -mmin -10, both found my file, and the later one seems to be 3s faster. Only 3s for me but if you have more folders/bigger files, it should be a greater improvement.– estudiante27
Sep 22 '15 at 18:17
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
I did that but it's not really helping. I wish the find command could have more options
– Youssef Karami
Sep 22 '15 at 20:10
would using
relatime as a mount option work any better ? just a vague idea. though not sure you can find using access time– infinite-etcetera
Jan 30 '17 at 8:07
would using
relatime as a mount option work any better ? just a vague idea. though not sure you can find using access time– infinite-etcetera
Jan 30 '17 at 8:07
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
Accessing a folder with few files is often quicker than accessing a folder with millions of files.
– Kusalananda
Jun 8 at 8:10
add a comment |Â
up vote
0
down vote
I would first find the latest directory used and then check only that directory:
find $(ls -rt /srv/unifi-video/videos/* | tail -1) -type f -name '*.ts' -mmin -10 | wc -l
answered Nov 21 '17 at 7:46
xenoid
1,7051620
add a comment |Â
up vote
0
down vote
I would first find the latest directory used and then check only that directory:
find $(ls -rt /srv/unifi-video/videos/* | tail -1) -type f -name '*.ts' -mmin -10 | wc -l
answered Nov 21 '17 at 7:46
xenoid
1,7051620
add a comment |Â
up vote
0
down vote
up vote
0
down vote
I would first find the latest directory used and then check only that directory:
find $(ls -rt /srv/unifi-video/videos/* | tail -1) -type f -name '*.ts' -mmin -10 | wc -l
answered Nov 21 '17 at 7:46
xenoid
1,7051620
I would first find the latest directory used and then check only that directory:
find $(ls -rt /srv/unifi-video/videos/* | tail -1) -type f -name '*.ts' -mmin -10 | wc -l
answered Nov 21 '17 at 7:46
xenoid
1,7051620
answered Nov 21 '17 at 7:46
xenoid
1,7051620
answered Nov 21 '17 at 7:46
xenoid
1,7051620
answered Nov 21 '17 at 7:46
xenoid
1,7051620
1,7051620
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f231371%2ffind-command-to-check-only-for-the-last-10-minutes-and-not-the-whole-folder%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
What do you mean by “not the whole folderâ€� Do you mean that the files are created in
/srv/unifi-video/videos, not a subdirectory, and you don't wantfindto explore the subdirectories? If it's not that, what do you mean?– Gilles
Sep 22 '15 at 22:11
I meant to check inside the folder and sub-folders but only for a specific time frame. Like only for the passed 10 minutes and see if there's files or not. The way find is working takes a very long time to go through 400 Gb+ of files and that causes my plugin to timeout. I figured a solution in Nagios by adjusting the interval time and increase the timeout.
– Youssef Karami
Sep 23 '15 at 14:58
I don't think the size of files should matter most, it's probably just the sheer amount of files.
– phk
Dec 25 '16 at 16:57