Multiple >> redirects to same file by 2 scripts, mid-sentence breaking
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I have 2 programs, both writing to the same file (/tmp/outfile). Started by cron at the same time.
Basically this is what is happening:
echo -n "1111111111" >> /tmp/outfile
And at the same time:
echo -n "2222222222" >> /tmp/outfile
The output file says "11111222222222211111". This is an example, I am talking about hundreds of lines, where one line is "cut" mid-sentence, but simply put, above thing is happening.
How to prevent this behavior?
linux files cron io-redirection
edited Mar 15 at 13:30
galoget
36319
asked Mar 15 at 10:16
Karlo
8521617
add a comment |Â
up vote
1
down vote
favorite
I have 2 programs, both writing to the same file (/tmp/outfile). Started by cron at the same time.
Basically this is what is happening:
echo -n "1111111111" >> /tmp/outfile
And at the same time:
echo -n "2222222222" >> /tmp/outfile
The output file says "11111222222222211111". This is an example, I am talking about hundreds of lines, where one line is "cut" mid-sentence, but simply put, above thing is happening.
How to prevent this behavior?
linux files cron io-redirection
edited Mar 15 at 13:30
galoget
36319
asked Mar 15 at 10:16
Karlo
8521617
1
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look atflock-- it provides the necessary mechanisms to do so.
– Malte Skoruppa
Mar 15 at 10:30
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Prevent it and do what?
– muru
Mar 15 at 11:01
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have 2 programs, both writing to the same file (/tmp/outfile). Started by cron at the same time.
Basically this is what is happening:
echo -n "1111111111" >> /tmp/outfile
And at the same time:
echo -n "2222222222" >> /tmp/outfile
The output file says "11111222222222211111". This is an example, I am talking about hundreds of lines, where one line is "cut" mid-sentence, but simply put, above thing is happening.
How to prevent this behavior?
linux files cron io-redirection
edited Mar 15 at 13:30
galoget
36319
asked Mar 15 at 10:16
Karlo
8521617
I have 2 programs, both writing to the same file (/tmp/outfile). Started by cron at the same time.
Basically this is what is happening:
echo -n "1111111111" >> /tmp/outfile
And at the same time:
echo -n "2222222222" >> /tmp/outfile
The output file says "11111222222222211111". This is an example, I am talking about hundreds of lines, where one line is "cut" mid-sentence, but simply put, above thing is happening.
How to prevent this behavior?
linux files cron io-redirection
edited Mar 15 at 13:30
galoget
36319
asked Mar 15 at 10:16
Karlo
8521617
edited Mar 15 at 13:30
galoget
36319
edited Mar 15 at 13:30
galoget
36319
edited Mar 15 at 13:30
galoget
36319
36319
asked Mar 15 at 10:16
Karlo
8521617
asked Mar 15 at 10:16
Karlo
8521617
asked Mar 15 at 10:16
Karlo
8521617
8521617
1
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look atflock-- it provides the necessary mechanisms to do so.
– Malte Skoruppa
Mar 15 at 10:30
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Prevent it and do what?
– muru
Mar 15 at 11:01
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10
add a comment |Â
1
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look atflock-- it provides the necessary mechanisms to do so.
– Malte Skoruppa
Mar 15 at 10:30
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Prevent it and do what?
– muru
Mar 15 at 11:01
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10
1
1
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look at
flock -- it provides the necessary mechanisms to do so.– Malte Skoruppa
Mar 15 at 10:30
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look at
flock -- it provides the necessary mechanisms to do so.– Malte Skoruppa
Mar 15 at 10:30
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Prevent it and do what?
– muru
Mar 15 at 11:01
Prevent it and do what?
– muru
Mar 15 at 11:01
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
There are two immediately obvious ways to solve this:
Serialize the tasks. Instead of scheduling the two tasks at the same time, schedule a script that runs the tasks one after the other.
Use a advisory locking scheme to lock the writing operation of the tasks in such a way that only one task can write at a time. See questions tagged with lock and flock.
These two may be combined into a single script that runs both tasks in the background while the tasks themselves uses some form of locking as to not produce garbled/intermingled output.
answered Mar 16 at 7:44
Kusalananda
103k13201318
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
There are two immediately obvious ways to solve this:
Serialize the tasks. Instead of scheduling the two tasks at the same time, schedule a script that runs the tasks one after the other.
Use a advisory locking scheme to lock the writing operation of the tasks in such a way that only one task can write at a time. See questions tagged with lock and flock.
These two may be combined into a single script that runs both tasks in the background while the tasks themselves uses some form of locking as to not produce garbled/intermingled output.
answered Mar 16 at 7:44
Kusalananda
103k13201318
add a comment |Â
up vote
2
down vote
accepted
There are two immediately obvious ways to solve this:
Serialize the tasks. Instead of scheduling the two tasks at the same time, schedule a script that runs the tasks one after the other.
Use a advisory locking scheme to lock the writing operation of the tasks in such a way that only one task can write at a time. See questions tagged with lock and flock.
These two may be combined into a single script that runs both tasks in the background while the tasks themselves uses some form of locking as to not produce garbled/intermingled output.
answered Mar 16 at 7:44
Kusalananda
103k13201318
add a comment |Â
up vote
2
down vote
accepted
up vote
2
down vote
accepted
There are two immediately obvious ways to solve this:
Serialize the tasks. Instead of scheduling the two tasks at the same time, schedule a script that runs the tasks one after the other.
Use a advisory locking scheme to lock the writing operation of the tasks in such a way that only one task can write at a time. See questions tagged with lock and flock.
These two may be combined into a single script that runs both tasks in the background while the tasks themselves uses some form of locking as to not produce garbled/intermingled output.
answered Mar 16 at 7:44
Kusalananda
103k13201318
There are two immediately obvious ways to solve this:
Serialize the tasks. Instead of scheduling the two tasks at the same time, schedule a script that runs the tasks one after the other.
Use a advisory locking scheme to lock the writing operation of the tasks in such a way that only one task can write at a time. See questions tagged with lock and flock.
These two may be combined into a single script that runs both tasks in the background while the tasks themselves uses some form of locking as to not produce garbled/intermingled output.
answered Mar 16 at 7:44
Kusalananda
103k13201318
edited Mar 16 at 8:31
answered Mar 16 at 7:44
Kusalananda
103k13201318
answered Mar 16 at 7:44
Kusalananda
103k13201318
answered Mar 16 at 7:44
Kusalananda
103k13201318
103k13201318
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f430361%2fmultiple-redirects-to-same-file-by-2-scripts-mid-sentence-breaking%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Well, you have different threads with a handle on the same file, the behavior that you're seeing is perfectly logical. To prevent it, the different cron jobs that you have should each lock a file when they're writing to it, unlock it when they're done, and wait before writing to a file when it is locked. Have a look at
flock-- it provides the necessary mechanisms to do so.– Malte Skoruppa
Mar 15 at 10:30
Related: unix.stackexchange.com/questions/299627/…
– Tomasz
Mar 15 at 10:33
Prevent it and do what?
– muru
Mar 15 at 11:01
Flock sounds like an answer. I want to prevent sentences from being "broken" in 2 parts :)
– Karlo
Mar 15 at 13:10