How can I use a file in a command and redirect output to the same file without truncating it? [duplicate]
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
This question already has an answer here:
Can I read and write to the same file in Linux without overwriting it? [duplicate]
2 answers
Output to be saved in file named same as the input file [duplicate]
3 answers
How can I make iconv replace the input file with the converted output?
8 answers
cat package.yaml > package.yaml
The above will truncate package.yaml
, and hence the file will be empty - cat
will have empty output.
How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?
bash command-line zsh io-redirection
marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles
StackExchange.ready(function()
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();
);
);
);
Feb 27 at 7:19
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
 |Â
show 2 more comments
up vote
0
down vote
favorite
This question already has an answer here:
Can I read and write to the same file in Linux without overwriting it? [duplicate]
2 answers
Output to be saved in file named same as the input file [duplicate]
3 answers
How can I make iconv replace the input file with the converted output?
8 answers
cat package.yaml > package.yaml
The above will truncate package.yaml
, and hence the file will be empty - cat
will have empty output.
How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?
bash command-line zsh io-redirection
marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles
StackExchange.ready(function()
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();
);
);
);
Feb 27 at 7:19
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
2
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
A variable potentially could be used but is certainly not going to be the best or safest option:myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22
 |Â
show 2 more comments
up vote
0
down vote
favorite
up vote
0
down vote
favorite
This question already has an answer here:
Can I read and write to the same file in Linux without overwriting it? [duplicate]
2 answers
Output to be saved in file named same as the input file [duplicate]
3 answers
How can I make iconv replace the input file with the converted output?
8 answers
cat package.yaml > package.yaml
The above will truncate package.yaml
, and hence the file will be empty - cat
will have empty output.
How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?
bash command-line zsh io-redirection
This question already has an answer here:
Can I read and write to the same file in Linux without overwriting it? [duplicate]
2 answers
Output to be saved in file named same as the input file [duplicate]
3 answers
How can I make iconv replace the input file with the converted output?
8 answers
cat package.yaml > package.yaml
The above will truncate package.yaml
, and hence the file will be empty - cat
will have empty output.
How can I avoid truncating the file? Is there an alternative to using a temporary file? A variable maybe?
This question already has an answer here:
Can I read and write to the same file in Linux without overwriting it? [duplicate]
2 answers
Output to be saved in file named same as the input file [duplicate]
3 answers
How can I make iconv replace the input file with the converted output?
8 answers
bash command-line zsh io-redirection
edited Feb 26 at 18:34
asked Feb 26 at 18:15
Chris Stryczynski
403216
403216
marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles
StackExchange.ready(function()
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();
);
);
);
Feb 27 at 7:19
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
marked as duplicate by Jeff Schaller, G-Man, Romeo Ninov, Gilles
StackExchange.ready(function()
if (StackExchange.options.isMobile) return;
$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');
$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();
);
);
);
Feb 27 at 7:19
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
2
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
A variable potentially could be used but is certainly not going to be the best or safest option:myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22
 |Â
show 2 more comments
1
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
2
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
A variable potentially could be used but is certainly not going to be the best or safest option:myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22
1
1
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
2
2
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
A variable potentially could be used but is certainly not going to be the best or safest option:
myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22
A variable potentially could be used but is certainly not going to be the best or safest option:
myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22
 |Â
show 2 more comments
1 Answer
1
active
oldest
votes
up vote
1
down vote
All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.
Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd
utility, for example:
$ cat file.txt
hello world
abc abc 123 123
$ cat insert.txt
hello!
$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)
$ cat file.txt
hello hello!
bc abc 123 123
Here, we insert what's in insert.txt
into file.txt
by first seeking 6 bytes forward in the file (stepping past hello
and the space) and then modifying it. The conv=notrunc
stops the output file from being truncated at the end of the writing operation.
If if=insert.txt
was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt
).
You wouldn't want to edit a file this way though.
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more likeperl -pi
(alsosed -i
,gawk -v inplace
) but is not the norm. See also the<>
redirection operator,$mapfile
inzsh
, moreutilssponge
,mmap()
...
â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (seefallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).
â Stéphane Chazelas
Mar 4 at 20:10
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.
Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd
utility, for example:
$ cat file.txt
hello world
abc abc 123 123
$ cat insert.txt
hello!
$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)
$ cat file.txt
hello hello!
bc abc 123 123
Here, we insert what's in insert.txt
into file.txt
by first seeking 6 bytes forward in the file (stepping past hello
and the space) and then modifying it. The conv=notrunc
stops the output file from being truncated at the end of the writing operation.
If if=insert.txt
was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt
).
You wouldn't want to edit a file this way though.
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more likeperl -pi
(alsosed -i
,gawk -v inplace
) but is not the norm. See also the<>
redirection operator,$mapfile
inzsh
, moreutilssponge
,mmap()
...
â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (seefallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).
â Stéphane Chazelas
Mar 4 at 20:10
add a comment |Â
up vote
1
down vote
All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.
Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd
utility, for example:
$ cat file.txt
hello world
abc abc 123 123
$ cat insert.txt
hello!
$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)
$ cat file.txt
hello hello!
bc abc 123 123
Here, we insert what's in insert.txt
into file.txt
by first seeking 6 bytes forward in the file (stepping past hello
and the space) and then modifying it. The conv=notrunc
stops the output file from being truncated at the end of the writing operation.
If if=insert.txt
was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt
).
You wouldn't want to edit a file this way though.
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more likeperl -pi
(alsosed -i
,gawk -v inplace
) but is not the norm. See also the<>
redirection operator,$mapfile
inzsh
, moreutilssponge
,mmap()
...
â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (seefallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).
â Stéphane Chazelas
Mar 4 at 20:10
add a comment |Â
up vote
1
down vote
up vote
1
down vote
All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.
Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd
utility, for example:
$ cat file.txt
hello world
abc abc 123 123
$ cat insert.txt
hello!
$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)
$ cat file.txt
hello hello!
bc abc 123 123
Here, we insert what's in insert.txt
into file.txt
by first seeking 6 bytes forward in the file (stepping past hello
and the space) and then modifying it. The conv=notrunc
stops the output file from being truncated at the end of the writing operation.
If if=insert.txt
was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt
).
You wouldn't want to edit a file this way though.
All utilities that allows for in-place editing of files does so by internally writing the result to a temporary file and then replacing the original file by this temporary file once the operation is complete.
Files can be modified in place, but you would only be able to overwrite existing data and/or extend the length of the file. This may be done using the dd
utility, for example:
$ cat file.txt
hello world
abc abc 123 123
$ cat insert.txt
hello!
$ dd if=insert.txt of=file.txt bs=1 seek=6 conv=notrunc
7+0 records in
7+0 records out
7 bytes transferred in 0.000 secs (30918 bytes/sec)
$ cat file.txt
hello hello!
bc abc 123 123
Here, we insert what's in insert.txt
into file.txt
by first seeking 6 bytes forward in the file (stepping past hello
and the space) and then modifying it. The conv=notrunc
stops the output file from being truncated at the end of the writing operation.
If if=insert.txt
was left out, it would have been possible to insert any text from the keyboard. Note that "inserting" is the wrong term here really. "Overwriting" may better describe what's happening (see the first character of the second line being overwritten by the newline at the end of insert.txt
).
You wouldn't want to edit a file this way though.
edited Feb 26 at 19:11
answered Feb 26 at 18:20
Kusalananda
103k13202318
103k13202318
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more likeperl -pi
(alsosed -i
,gawk -v inplace
) but is not the norm. See also the<>
redirection operator,$mapfile
inzsh
, moreutilssponge
,mmap()
...
â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (seefallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).
â Stéphane Chazelas
Mar 4 at 20:10
add a comment |Â
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more likeperl -pi
(alsosed -i
,gawk -v inplace
) but is not the norm. See also the<>
redirection operator,$mapfile
inzsh
, moreutilssponge
,mmap()
...
â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (seefallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).
â Stéphane Chazelas
Mar 4 at 20:10
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Regarding your first sentence: there are other strategies. What you describes is the best strategy in terms of safety: if something fails, the original is untouched. But many programs do backup-then-overwrite, which results in a truncated file if interrupted but has the advantage of preserving permissins. And some dangerous programs do read-then-overwrite, which saves disk space but your data disappears if anything fails.
â Gilles
Feb 27 at 7:22
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like
perl -pi
(also sed -i
, gawk -v inplace
) but is not the norm. See also the <>
redirection operator, $mapfile
in zsh
, moreutils sponge
, mmap()
...â Stéphane Chazelas
Mar 4 at 20:08
Different text editors have different strategies to edit files (without having to store the whole content in memory for some), but few if any take the approach you describe. What you describe sounds more like
perl -pi
(also sed -i
, gawk -v inplace
) but is not the norm. See also the <>
redirection operator, $mapfile
in zsh
, moreutils sponge
, mmap()
...â Stéphane Chazelas
Mar 4 at 20:08
Some systems do support inserting data into file (see
fallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).â Stéphane Chazelas
Mar 4 at 20:10
Some systems do support inserting data into file (see
fallocate(FALLOC_FL_INSERT_RANGE)
on Linux for instance, though that's only in multiples of the filesystem block size).â Stéphane Chazelas
Mar 4 at 20:10
add a comment |Â
1
What are you trying to accomplish?
â Jesse_b
Feb 26 at 18:17
Sorry, the command is a simplified useless case. Essentially I'm modifying the file and would like to 'update' it.
â Chris Stryczynski
Feb 26 at 18:18
2
See: unix.stackexchange.com/questions/189473/â¦
â Jeff Schaller
Feb 26 at 18:19
In this case it would help to know how you are modifying it because some text processing programs (like sed) can modify files in place natively.
â Jesse_b
Feb 26 at 18:19
A variable potentially could be used but is certainly not going to be the best or safest option:
myvar=$(sed 's/my_string/new_string/' input) && echo "$myvar" > input
â Jesse_b
Feb 26 at 18:22