How can I use a file in a command and redirect output to the same file without truncating it? [duplicate]

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite













This question already has an answer here:



  • Can I read and write to the same file in Linux without overwriting it? [duplicate]

    2 answers



  • Output to be saved in file named same as the input file [duplicate]

    3 answers



  • How can I make iconv replace the input file with the converted output?

    8 answers



cat package.yaml > package.yaml


The above will truncate package.yaml, and hence the file will be empty - cat will have empty output.



How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?







share|improve this question














marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles bash
Users with the  bash badge can single-handedly close bash questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Feb 27 at 7:19


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.










  • 1




    What are you trying to accomplish?
    – Jesse_b
    Feb 26 at 18:17










  • Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
    – Chris Stryczynski
    Feb 26 at 18:18






  • 2




    See: unix.stackexchange.com/questions/189473/…
    – Jeff Schaller
    Feb 26 at 18:19










  • In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
    – Jesse_b
    Feb 26 at 18:19










  • A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
    – Jesse_b
    Feb 26 at 18:22















up vote
0
down vote

favorite













This question already has an answer here:



  • Can I read and write to the same file in Linux without overwriting it? [duplicate]

    2 answers



  • Output to be saved in file named same as the input file [duplicate]

    3 answers



  • How can I make iconv replace the input file with the converted output?

    8 answers



cat package.yaml > package.yaml


The above will truncate package.yaml, and hence the file will be empty - cat will have empty output.



How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?







share|improve this question














marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles bash
Users with the  bash badge can single-handedly close bash questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Feb 27 at 7:19


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.










  • 1




    What are you trying to accomplish?
    – Jesse_b
    Feb 26 at 18:17










  • Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
    – Chris Stryczynski
    Feb 26 at 18:18






  • 2




    See: unix.stackexchange.com/questions/189473/…
    – Jeff Schaller
    Feb 26 at 18:19










  • In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
    – Jesse_b
    Feb 26 at 18:19










  • A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
    – Jesse_b
    Feb 26 at 18:22













up vote
0
down vote

favorite









up vote
0
down vote

favorite












This question already has an answer here:



  • Can I read and write to the same file in Linux without overwriting it? [duplicate]

    2 answers



  • Output to be saved in file named same as the input file [duplicate]

    3 answers



  • How can I make iconv replace the input file with the converted output?

    8 answers



cat package.yaml > package.yaml


The above will truncate package.yaml, and hence the file will be empty - cat will have empty output.



How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?







share|improve this question















This question already has an answer here:



  • Can I read and write to the same file in Linux without overwriting it? [duplicate]

    2 answers



  • Output to be saved in file named same as the input file [duplicate]

    3 answers



  • How can I make iconv replace the input file with the converted output?

    8 answers



cat package.yaml > package.yaml


The above will truncate package.yaml, and hence the file will be empty - cat will have empty output.



How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?





This question already has an answer here:



  • Can I read and write to the same file in Linux without overwriting it? [duplicate]

    2 answers



  • Output to be saved in file named same as the input file [duplicate]

    3 answers



  • How can I make iconv replace the input file with the converted output?

    8 answers









share|improve this question













share|improve this question




share|improve this question








edited Feb 26 at 18:34

























asked Feb 26 at 18:15









Chris Stryczynski

403216




403216




marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles bash
Users with the  bash badge can single-handedly close bash questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Feb 27 at 7:19


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.






marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles bash
Users with the  bash badge can single-handedly close bash questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Feb 27 at 7:19


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.









  • 1




    What are you trying to accomplish?
    – Jesse_b
    Feb 26 at 18:17










  • Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
    – Chris Stryczynski
    Feb 26 at 18:18






  • 2




    See: unix.stackexchange.com/questions/189473/…
    – Jeff Schaller
    Feb 26 at 18:19










  • In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
    – Jesse_b
    Feb 26 at 18:19










  • A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
    – Jesse_b
    Feb 26 at 18:22













  • 1




    What are you trying to accomplish?
    – Jesse_b
    Feb 26 at 18:17










  • Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
    – Chris Stryczynski
    Feb 26 at 18:18






  • 2




    See: unix.stackexchange.com/questions/189473/…
    – Jeff Schaller
    Feb 26 at 18:19










  • In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
    – Jesse_b
    Feb 26 at 18:19










  • A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
    – Jesse_b
    Feb 26 at 18:22








1




1




What are you trying to accomplish?
– Jesse_b
Feb 26 at 18:17




What are you trying to accomplish?
– Jesse_b
Feb 26 at 18:17












Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
– Chris Stryczynski
Feb 26 at 18:18




Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
– Chris Stryczynski
Feb 26 at 18:18




2




2




See: unix.stackexchange.com/questions/189473/…
– Jeff Schaller
Feb 26 at 18:19




See: unix.stackexchange.com/questions/189473/…
– Jeff Schaller
Feb 26 at 18:19












In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
– Jesse_b
Feb 26 at 18:19




In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
– Jesse_b
Feb 26 at 18:19












A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
– Jesse_b
Feb 26 at 18:22





A variable potentially could be used but is certainly not going to be the best or safest option: myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
– Jesse_b
Feb 26 at 18:22











1 Answer
1






active

oldest

votes

















up vote
1
down vote













All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.




Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:



$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt
hello!

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123


Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.



If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).



You wouldn't want to edit a file this way though.






share|improve this answer






















  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
    – Gilles
    Feb 27 at 7:22










  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
    – Stéphane Chazelas
    Mar 4 at 20:08











  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
    – Stéphane Chazelas
    Mar 4 at 20:10

















1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
1
down vote













All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.




Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:



$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt
hello!

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123


Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.



If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).



You wouldn't want to edit a file this way though.






share|improve this answer






















  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
    – Gilles
    Feb 27 at 7:22










  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
    – Stéphane Chazelas
    Mar 4 at 20:08











  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
    – Stéphane Chazelas
    Mar 4 at 20:10














up vote
1
down vote













All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.




Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:



$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt
hello!

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123


Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.



If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).



You wouldn't want to edit a file this way though.






share|improve this answer






















  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
    – Gilles
    Feb 27 at 7:22










  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
    – Stéphane Chazelas
    Mar 4 at 20:08











  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
    – Stéphane Chazelas
    Mar 4 at 20:10












up vote
1
down vote










up vote
1
down vote









All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.




Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:



$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt
hello!

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123


Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.



If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).



You wouldn't want to edit a file this way though.






share|improve this answer














All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.




Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd utility, for example:



$ cat file.txt
hello world
abc abc 123 123

$ cat insert.txt
hello!

$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)

$ cat file.txt
hello hello!
bc abc 123 123


Here, we insert what's in insert.txt into file.txt by first seeking 6 bytes forward in the file (stepping past hello and the space) and then modifying it. The conv=notrunc stops the output file from being truncated at the end of the writing operation.



If if=insert.txt was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt).



You wouldn't want to edit a file this way though.







share|improve this answer














share|improve this answer



share|improve this answer








edited Feb 26 at 19:11

























answered Feb 26 at 18:20









Kusalananda

103k13202318




103k13202318











  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
    – Gilles
    Feb 27 at 7:22










  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
    – Stéphane Chazelas
    Mar 4 at 20:08











  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
    – Stéphane Chazelas
    Mar 4 at 20:10
















  • Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
    – Gilles
    Feb 27 at 7:22










  • Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
    – Stéphane Chazelas
    Mar 4 at 20:08











  • Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
    – Stéphane Chazelas
    Mar 4 at 20:10















Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
– Gilles
Feb 27 at 7:22




Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
– Gilles
Feb 27 at 7:22












Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
– Stéphane Chazelas
Mar 4 at 20:08





Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like perl -pi (also sed -i, gawk -v inplace) but is not the norm. See also the <> redirection operator, $mapfile in zsh, moreutils sponge, mmap()...
– Stéphane Chazelas
Mar 4 at 20:08













Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
– Stéphane Chazelas
Mar 4 at 20:10




Some systems do support inserting data into file (see fallocate(FALLOC_FL_INSERT_RANGE) on Linux for instance, though that's only in multiples of the filesystem block size).
– Stéphane Chazelas
Mar 4 at 20:10


Popular posts from this blog

How to check contact read email or not when send email to Individual?

Bahrain

Postfix configuration issue with fips on centos 7; mailgun relay