How to remove multiple newlines at EOF?
Clash Royale CLAN TAG#URR8PPP
up vote
21
down vote
favorite
I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools?
Example bad file:
1n
n
2n
n
n
3n
n
n
n
Example corrected file:
1n
n
2n
n
n
3n
In other words: There should be exactly one newline between the EOF and the last non-newline character of the file.
Reference Implementation
Read file contents, chop off a single newline till there no further two newlines at the end, write it back:
#! /bin/python
import sys
with open(sys.argv[1]) as infile:
lines = infile.read()
while lines.endswith("nn"):
lines = lines[:-1]
with open(sys.argv[2], 'w') as outfile:
for line in lines:
outfile.write(line)
Clarification: Of course, piping is allowed, if that is more elegant.
bash sed awk ed
add a comment |Â
up vote
21
down vote
favorite
I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools?
Example bad file:
1n
n
2n
n
n
3n
n
n
n
Example corrected file:
1n
n
2n
n
n
3n
In other words: There should be exactly one newline between the EOF and the last non-newline character of the file.
Reference Implementation
Read file contents, chop off a single newline till there no further two newlines at the end, write it back:
#! /bin/python
import sys
with open(sys.argv[1]) as infile:
lines = infile.read()
while lines.endswith("nn"):
lines = lines[:-1]
with open(sys.argv[2], 'w') as outfile:
for line in lines:
outfile.write(line)
Clarification: Of course, piping is allowed, if that is more elegant.
bash sed awk ed
add a comment |Â
up vote
21
down vote
favorite
up vote
21
down vote
favorite
I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools?
Example bad file:
1n
n
2n
n
n
3n
n
n
n
Example corrected file:
1n
n
2n
n
n
3n
In other words: There should be exactly one newline between the EOF and the last non-newline character of the file.
Reference Implementation
Read file contents, chop off a single newline till there no further two newlines at the end, write it back:
#! /bin/python
import sys
with open(sys.argv[1]) as infile:
lines = infile.read()
while lines.endswith("nn"):
lines = lines[:-1]
with open(sys.argv[2], 'w') as outfile:
for line in lines:
outfile.write(line)
Clarification: Of course, piping is allowed, if that is more elegant.
bash sed awk ed
I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools?
Example bad file:
1n
n
2n
n
n
3n
n
n
n
Example corrected file:
1n
n
2n
n
n
3n
In other words: There should be exactly one newline between the EOF and the last non-newline character of the file.
Reference Implementation
Read file contents, chop off a single newline till there no further two newlines at the end, write it back:
#! /bin/python
import sys
with open(sys.argv[1]) as infile:
lines = infile.read()
while lines.endswith("nn"):
lines = lines[:-1]
with open(sys.argv[2], 'w') as outfile:
for line in lines:
outfile.write(line)
Clarification: Of course, piping is allowed, if that is more elegant.
bash sed awk ed
bash sed awk ed
edited Jul 4 '13 at 14:57
slmâ¦
239k65494665
239k65494665
asked Jul 4 '13 at 0:20
Bengt
4041514
4041514
add a comment |Â
add a comment |Â
10 Answers
10
active
oldest
votes
up vote
15
down vote
accepted
awk '/^$/ nlstack=nlstack "n";next; printf "%s",nlstack; nlstack=""; print;' file
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw thesed
proposal I just thought OMG...
â Hauke Laging
Jul 4 '13 at 11:32
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors withawk: illegal statement
.brew install mawk
and changing the command tomawk
works though.
â tjmcewan
May 9 '14 at 5:02
add a comment |Â
up vote
15
down vote
From useful one-line scripts for sed.
# Delete all trailing blank lines at end of file (only).
sed -e :a -e '/^n*$/$d;N;;/n$/ba' file
3
Thanks, I used the following to do it in place for multiple files:find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
add a comment |Â
up vote
15
down vote
Since you already have answers with the more suitable tools sed and awk; you could take advantage of the fact that $(< file)
strips off trailing blank lines.
a=$(<file); printf '%sn' "$a" > file
That cheap hack wouldn't work to remove trailing blank lines which may contain spaces or other non-printing characters, only to remove trailing empty lines. It also won't work if the file contains null bytes.
In shells other than bash and zsh, use $(cat file)
instead of $(<file)
.
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
@OlivierDulac$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier:echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.
â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.
â davidchambers
Apr 14 '14 at 19:11
add a comment |Â
up vote
5
down vote
You can use this trick with cat
& printf
:
$ printf '%sn' "`cat file`"
For example
$ printf '%sn' "`cat ifile`" > ofile
$ cat -e ofile
1$
$
2$
$
$
3$
The $
denotes the end of a line.
References
- Removing trailing blank lines
1
may fail iffile
contains e.g.-n
. Useprintf
instead ofecho
.
â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that``
strips any newlines at EOF. See the other answer for a discussion.
â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
add a comment |Â
up vote
2
down vote
Here's a Perl solution that doesn't require reading more than one line into memory at a time:
my $n = 0;
while (<>)
if (/./)
print "n" x $n, $_;
$n = 0;
else
$n++;
or, as a one-liner:
perl -ne 'if (/./) print "n" x $n, $_; $n = 0 else $n++ '
This reads the file a line at a time and checks each line to see if contains a non-newline character. If it doesn't, it increments a counter; if it does, it prints the number of newlines indicated by the counter, followed by the line itself, and then resets the counter.
Technically, even buffering a single line in memory is unnecessary; it would be possible to solve this problem using a constant amount of memory by reading the file in fixed-length chunks and processing it character by character using a state machine. However, I suspect that would be needlessly complicated for the typical use case.
add a comment |Â
up vote
2
down vote
This question is tagged with ed
, but nobody has proposed an ed
solution (I wonder why?).
Here's one:
ed file <<ED_END
a
.
?^..*?+1,.d
w
ED_END
ed
will place you at the last line of the editing buffer by default upon startup.
The first command (a
) adds an empty line to the end of the buffer (the empty line in the editing script is this line, and the dot (.
) is just for coming back into command mode).
The second command (?
) looks for the nearest previous line that contains something (even white-space characters), and then deletes everything to the end of the buffer from the next line on.
The third command (w
) writes the file back to disk.
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
add a comment |Â
up vote
1
down vote
If your file is small enough to slurp into memory, you can use this
perl -e 'local($/);$f=<>; $f=~s/n*$/n/;print $f;' file
add a comment |Â
up vote
0
down vote
In python (I know it is not what you want, but it is much better as it is optimized, and a prelude to the bash version) without rewriting the file and without reading all the file (which is a good thing if the file is very large):
#!/bin/python
import sys
infile = open(sys.argv[1], 'r+')
infile.seek(-1, 2)
while infile.read(1) == 'n':
infile.seek(-2, 1)
infile.seek(1, 1)
infile.truncate()
infile.close()
Note that it does not work on files where the EOL character is not 'n'.
add a comment |Â
up vote
0
down vote
A bash version, implementing the python algorithm, but less efficient as it needs many processes:
#!/bin/bash
n=1
while test "$(tail -n $n "$1")" == ""; do
((n++))
done
((n--))
truncate -s $(($(stat -c "%s" "$1") - $n)) "$1"
add a comment |Â
up vote
0
down vote
This one is quick to type, and, if you know sed, easy to remember:
tac < file | sed '/[^[:blank:]]/,$!d' | tac
It uses the sed script to delete leading blank lines from useful one line scripts for sed, referenced by Alexey, above, and tac (reverse cat).
In a quick test, on an 18MB, 64,000 line file, Alexey's approach was faster, (0.036 vs 0.046 seconds).
add a comment |Â
10 Answers
10
active
oldest
votes
10 Answers
10
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
15
down vote
accepted
awk '/^$/ nlstack=nlstack "n";next; printf "%s",nlstack; nlstack=""; print;' file
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw thesed
proposal I just thought OMG...
â Hauke Laging
Jul 4 '13 at 11:32
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors withawk: illegal statement
.brew install mawk
and changing the command tomawk
works though.
â tjmcewan
May 9 '14 at 5:02
add a comment |Â
up vote
15
down vote
accepted
awk '/^$/ nlstack=nlstack "n";next; printf "%s",nlstack; nlstack=""; print;' file
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw thesed
proposal I just thought OMG...
â Hauke Laging
Jul 4 '13 at 11:32
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors withawk: illegal statement
.brew install mawk
and changing the command tomawk
works though.
â tjmcewan
May 9 '14 at 5:02
add a comment |Â
up vote
15
down vote
accepted
up vote
15
down vote
accepted
awk '/^$/ nlstack=nlstack "n";next; printf "%s",nlstack; nlstack=""; print;' file
awk '/^$/ nlstack=nlstack "n";next; printf "%s",nlstack; nlstack=""; print;' file
answered Jul 4 '13 at 0:40
Hauke Laging
53.9k1282130
53.9k1282130
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw thesed
proposal I just thought OMG...
â Hauke Laging
Jul 4 '13 at 11:32
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors withawk: illegal statement
.brew install mawk
and changing the command tomawk
works though.
â tjmcewan
May 9 '14 at 5:02
add a comment |Â
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw thesed
proposal I just thought OMG...
â Hauke Laging
Jul 4 '13 at 11:32
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors withawk: illegal statement
.brew install mawk
and changing the command tomawk
works though.
â tjmcewan
May 9 '14 at 5:02
2
2
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
+1: awk's solutions are (almost) always elegant and readable!
â Olivier Dulac
Jul 4 '13 at 9:37
@OlivierDulac Indeed. When I saw the
sed
proposal I just thought OMG...â Hauke Laging
Jul 4 '13 at 11:32
@OlivierDulac Indeed. When I saw the
sed
proposal I just thought OMG...â Hauke Laging
Jul 4 '13 at 11:32
1
1
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors with
awk: illegal statement
. brew install mawk
and changing the command to mawk
works though.â tjmcewan
May 9 '14 at 5:02
this doesn't work on OSX Mavericks using the latest available awk from Homebrew. It errors with
awk: illegal statement
. brew install mawk
and changing the command to mawk
works though.â tjmcewan
May 9 '14 at 5:02
add a comment |Â
up vote
15
down vote
From useful one-line scripts for sed.
# Delete all trailing blank lines at end of file (only).
sed -e :a -e '/^n*$/$d;N;;/n$/ba' file
3
Thanks, I used the following to do it in place for multiple files:find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
add a comment |Â
up vote
15
down vote
From useful one-line scripts for sed.
# Delete all trailing blank lines at end of file (only).
sed -e :a -e '/^n*$/$d;N;;/n$/ba' file
3
Thanks, I used the following to do it in place for multiple files:find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
add a comment |Â
up vote
15
down vote
up vote
15
down vote
From useful one-line scripts for sed.
# Delete all trailing blank lines at end of file (only).
sed -e :a -e '/^n*$/$d;N;;/n$/ba' file
From useful one-line scripts for sed.
# Delete all trailing blank lines at end of file (only).
sed -e :a -e '/^n*$/$d;N;;/n$/ba' file
answered Jul 4 '13 at 0:38
Alexey Shmalko
29915
29915
3
Thanks, I used the following to do it in place for multiple files:find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
add a comment |Â
3
Thanks, I used the following to do it in place for multiple files:find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
3
3
Thanks, I used the following to do it in place for multiple files:
find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
Thanks, I used the following to do it in place for multiple files:
find . -type f -name '*.js' -exec sed --in-place -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â jakub.g
Nov 22 '13 at 9:48
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
@jakub.g in place and recursive is exactly what I needed. thank you.
â Buttle Butkus
Dec 13 '15 at 10:41
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:
find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
To add to the excellent comment from @jakub.g you can invoke the command like this on OS X:
find . -type f -name '*.js' -exec sed -i '' -e :a -e '/^n*$/$d;N;;/n$/ba' ;
â davejagoda
Feb 19 at 18:35
add a comment |Â
up vote
15
down vote
Since you already have answers with the more suitable tools sed and awk; you could take advantage of the fact that $(< file)
strips off trailing blank lines.
a=$(<file); printf '%sn' "$a" > file
That cheap hack wouldn't work to remove trailing blank lines which may contain spaces or other non-printing characters, only to remove trailing empty lines. It also won't work if the file contains null bytes.
In shells other than bash and zsh, use $(cat file)
instead of $(<file)
.
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
@OlivierDulac$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier:echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.
â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.
â davidchambers
Apr 14 '14 at 19:11
add a comment |Â
up vote
15
down vote
Since you already have answers with the more suitable tools sed and awk; you could take advantage of the fact that $(< file)
strips off trailing blank lines.
a=$(<file); printf '%sn' "$a" > file
That cheap hack wouldn't work to remove trailing blank lines which may contain spaces or other non-printing characters, only to remove trailing empty lines. It also won't work if the file contains null bytes.
In shells other than bash and zsh, use $(cat file)
instead of $(<file)
.
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
@OlivierDulac$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier:echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.
â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.
â davidchambers
Apr 14 '14 at 19:11
add a comment |Â
up vote
15
down vote
up vote
15
down vote
Since you already have answers with the more suitable tools sed and awk; you could take advantage of the fact that $(< file)
strips off trailing blank lines.
a=$(<file); printf '%sn' "$a" > file
That cheap hack wouldn't work to remove trailing blank lines which may contain spaces or other non-printing characters, only to remove trailing empty lines. It also won't work if the file contains null bytes.
In shells other than bash and zsh, use $(cat file)
instead of $(<file)
.
Since you already have answers with the more suitable tools sed and awk; you could take advantage of the fact that $(< file)
strips off trailing blank lines.
a=$(<file); printf '%sn' "$a" > file
That cheap hack wouldn't work to remove trailing blank lines which may contain spaces or other non-printing characters, only to remove trailing empty lines. It also won't work if the file contains null bytes.
In shells other than bash and zsh, use $(cat file)
instead of $(<file)
.
edited Jul 5 '13 at 23:01
Gilles
511k12010091540
511k12010091540
answered Jul 4 '13 at 0:47
llua
4,5441220
4,5441220
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
@OlivierDulac$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier:echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.
â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.
â davidchambers
Apr 14 '14 at 19:11
add a comment |Â
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
@OlivierDulac$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier:echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.
â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.
â davidchambers
Apr 14 '14 at 19:11
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
+1 to point out what looks like a bug to me : $(<file) isn't really reading the file? why does it discard trailing newlines? (it does, i just tested, thanks for pointing it out!)
â Olivier Dulac
Jul 4 '13 at 9:23
2
2
@OlivierDulac
$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier: echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.â Hauke Laging
Jul 4 '13 at 11:31
@OlivierDulac
$()
discards trailing newlines. That's a design decision. I assume that this shall make the integration in other strings easier: echo "On $(date ...) we will meet."
would be evil with the newline that nearly every shell command outputs at the end.â Hauke Laging
Jul 4 '13 at 11:31
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
@HaukeLaging: good point, it's probably the source of that behaviour
â Olivier Dulac
Jul 4 '13 at 12:13
I added a special case to avoid appending "n" to empty files:
[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.â davidchambers
Apr 14 '14 at 19:11
I added a special case to avoid appending "n" to empty files:
[[ $a == '' ]] || printf '%sn' "$a" >"$file"
.â davidchambers
Apr 14 '14 at 19:11
add a comment |Â
up vote
5
down vote
You can use this trick with cat
& printf
:
$ printf '%sn' "`cat file`"
For example
$ printf '%sn' "`cat ifile`" > ofile
$ cat -e ofile
1$
$
2$
$
$
3$
The $
denotes the end of a line.
References
- Removing trailing blank lines
1
may fail iffile
contains e.g.-n
. Useprintf
instead ofecho
.
â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that``
strips any newlines at EOF. See the other answer for a discussion.
â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
add a comment |Â
up vote
5
down vote
You can use this trick with cat
& printf
:
$ printf '%sn' "`cat file`"
For example
$ printf '%sn' "`cat ifile`" > ofile
$ cat -e ofile
1$
$
2$
$
$
3$
The $
denotes the end of a line.
References
- Removing trailing blank lines
1
may fail iffile
contains e.g.-n
. Useprintf
instead ofecho
.
â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that``
strips any newlines at EOF. See the other answer for a discussion.
â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
add a comment |Â
up vote
5
down vote
up vote
5
down vote
You can use this trick with cat
& printf
:
$ printf '%sn' "`cat file`"
For example
$ printf '%sn' "`cat ifile`" > ofile
$ cat -e ofile
1$
$
2$
$
$
3$
The $
denotes the end of a line.
References
- Removing trailing blank lines
You can use this trick with cat
& printf
:
$ printf '%sn' "`cat file`"
For example
$ printf '%sn' "`cat ifile`" > ofile
$ cat -e ofile
1$
$
2$
$
$
3$
The $
denotes the end of a line.
References
- Removing trailing blank lines
edited Jul 7 '13 at 15:25
Bengt
4041514
4041514
answered Jul 4 '13 at 2:30
slmâ¦
239k65494665
239k65494665
1
may fail iffile
contains e.g.-n
. Useprintf
instead ofecho
.
â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that``
strips any newlines at EOF. See the other answer for a discussion.
â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
add a comment |Â
1
may fail iffile
contains e.g.-n
. Useprintf
instead ofecho
.
â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that``
strips any newlines at EOF. See the other answer for a discussion.
â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
1
1
may fail if
file
contains e.g. -n
. Use printf
instead of echo
.â Gilles
Jul 5 '13 at 22:59
may fail if
file
contains e.g. -n
. Use printf
instead of echo
.â Gilles
Jul 5 '13 at 22:59
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
@Gilles - thanks I've fixed per your feedback.
â slmâ¦
Jul 5 '13 at 23:15
This exploits the fact that
``
strips any newlines at EOF. See the other answer for a discussion.â Bengt
Jul 7 '13 at 14:52
This exploits the fact that
``
strips any newlines at EOF. See the other answer for a discussion.â Bengt
Jul 7 '13 at 14:52
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@Bengt - which discussion, the link you provided doesn't go to anything specific in this thread.
â slmâ¦
Jul 7 '13 at 23:19
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
@slm It should be equivalent to this one: unix.stackexchange.com/questions/81685/â¦
â Bengt
Jul 8 '13 at 0:00
add a comment |Â
up vote
2
down vote
Here's a Perl solution that doesn't require reading more than one line into memory at a time:
my $n = 0;
while (<>)
if (/./)
print "n" x $n, $_;
$n = 0;
else
$n++;
or, as a one-liner:
perl -ne 'if (/./) print "n" x $n, $_; $n = 0 else $n++ '
This reads the file a line at a time and checks each line to see if contains a non-newline character. If it doesn't, it increments a counter; if it does, it prints the number of newlines indicated by the counter, followed by the line itself, and then resets the counter.
Technically, even buffering a single line in memory is unnecessary; it would be possible to solve this problem using a constant amount of memory by reading the file in fixed-length chunks and processing it character by character using a state machine. However, I suspect that would be needlessly complicated for the typical use case.
add a comment |Â
up vote
2
down vote
Here's a Perl solution that doesn't require reading more than one line into memory at a time:
my $n = 0;
while (<>)
if (/./)
print "n" x $n, $_;
$n = 0;
else
$n++;
or, as a one-liner:
perl -ne 'if (/./) print "n" x $n, $_; $n = 0 else $n++ '
This reads the file a line at a time and checks each line to see if contains a non-newline character. If it doesn't, it increments a counter; if it does, it prints the number of newlines indicated by the counter, followed by the line itself, and then resets the counter.
Technically, even buffering a single line in memory is unnecessary; it would be possible to solve this problem using a constant amount of memory by reading the file in fixed-length chunks and processing it character by character using a state machine. However, I suspect that would be needlessly complicated for the typical use case.
add a comment |Â
up vote
2
down vote
up vote
2
down vote
Here's a Perl solution that doesn't require reading more than one line into memory at a time:
my $n = 0;
while (<>)
if (/./)
print "n" x $n, $_;
$n = 0;
else
$n++;
or, as a one-liner:
perl -ne 'if (/./) print "n" x $n, $_; $n = 0 else $n++ '
This reads the file a line at a time and checks each line to see if contains a non-newline character. If it doesn't, it increments a counter; if it does, it prints the number of newlines indicated by the counter, followed by the line itself, and then resets the counter.
Technically, even buffering a single line in memory is unnecessary; it would be possible to solve this problem using a constant amount of memory by reading the file in fixed-length chunks and processing it character by character using a state machine. However, I suspect that would be needlessly complicated for the typical use case.
Here's a Perl solution that doesn't require reading more than one line into memory at a time:
my $n = 0;
while (<>)
if (/./)
print "n" x $n, $_;
$n = 0;
else
$n++;
or, as a one-liner:
perl -ne 'if (/./) print "n" x $n, $_; $n = 0 else $n++ '
This reads the file a line at a time and checks each line to see if contains a non-newline character. If it doesn't, it increments a counter; if it does, it prints the number of newlines indicated by the counter, followed by the line itself, and then resets the counter.
Technically, even buffering a single line in memory is unnecessary; it would be possible to solve this problem using a constant amount of memory by reading the file in fixed-length chunks and processing it character by character using a state machine. However, I suspect that would be needlessly complicated for the typical use case.
answered Jul 4 '13 at 10:16
Ilmari Karonen
66559
66559
add a comment |Â
add a comment |Â
up vote
2
down vote
This question is tagged with ed
, but nobody has proposed an ed
solution (I wonder why?).
Here's one:
ed file <<ED_END
a
.
?^..*?+1,.d
w
ED_END
ed
will place you at the last line of the editing buffer by default upon startup.
The first command (a
) adds an empty line to the end of the buffer (the empty line in the editing script is this line, and the dot (.
) is just for coming back into command mode).
The second command (?
) looks for the nearest previous line that contains something (even white-space characters), and then deletes everything to the end of the buffer from the next line on.
The third command (w
) writes the file back to disk.
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
add a comment |Â
up vote
2
down vote
This question is tagged with ed
, but nobody has proposed an ed
solution (I wonder why?).
Here's one:
ed file <<ED_END
a
.
?^..*?+1,.d
w
ED_END
ed
will place you at the last line of the editing buffer by default upon startup.
The first command (a
) adds an empty line to the end of the buffer (the empty line in the editing script is this line, and the dot (.
) is just for coming back into command mode).
The second command (?
) looks for the nearest previous line that contains something (even white-space characters), and then deletes everything to the end of the buffer from the next line on.
The third command (w
) writes the file back to disk.
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
add a comment |Â
up vote
2
down vote
up vote
2
down vote
This question is tagged with ed
, but nobody has proposed an ed
solution (I wonder why?).
Here's one:
ed file <<ED_END
a
.
?^..*?+1,.d
w
ED_END
ed
will place you at the last line of the editing buffer by default upon startup.
The first command (a
) adds an empty line to the end of the buffer (the empty line in the editing script is this line, and the dot (.
) is just for coming back into command mode).
The second command (?
) looks for the nearest previous line that contains something (even white-space characters), and then deletes everything to the end of the buffer from the next line on.
The third command (w
) writes the file back to disk.
This question is tagged with ed
, but nobody has proposed an ed
solution (I wonder why?).
Here's one:
ed file <<ED_END
a
.
?^..*?+1,.d
w
ED_END
ed
will place you at the last line of the editing buffer by default upon startup.
The first command (a
) adds an empty line to the end of the buffer (the empty line in the editing script is this line, and the dot (.
) is just for coming back into command mode).
The second command (?
) looks for the nearest previous line that contains something (even white-space characters), and then deletes everything to the end of the buffer from the next line on.
The third command (w
) writes the file back to disk.
edited Dec 29 '16 at 17:15
answered Jul 8 '16 at 16:12
Kusalananda
107k14209331
107k14209331
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
add a comment |Â
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
@don_crissti Sorted. Thanks for the nudge! Sorry for the delay. Life, you know, keeps happening.
â Kusalananda
Dec 29 '16 at 17:16
add a comment |Â
up vote
1
down vote
If your file is small enough to slurp into memory, you can use this
perl -e 'local($/);$f=<>; $f=~s/n*$/n/;print $f;' file
add a comment |Â
up vote
1
down vote
If your file is small enough to slurp into memory, you can use this
perl -e 'local($/);$f=<>; $f=~s/n*$/n/;print $f;' file
add a comment |Â
up vote
1
down vote
up vote
1
down vote
If your file is small enough to slurp into memory, you can use this
perl -e 'local($/);$f=<>; $f=~s/n*$/n/;print $f;' file
If your file is small enough to slurp into memory, you can use this
perl -e 'local($/);$f=<>; $f=~s/n*$/n/;print $f;' file
answered Jul 4 '13 at 0:51
terdonâ¦
123k28232406
123k28232406
add a comment |Â
add a comment |Â
up vote
0
down vote
In python (I know it is not what you want, but it is much better as it is optimized, and a prelude to the bash version) without rewriting the file and without reading all the file (which is a good thing if the file is very large):
#!/bin/python
import sys
infile = open(sys.argv[1], 'r+')
infile.seek(-1, 2)
while infile.read(1) == 'n':
infile.seek(-2, 1)
infile.seek(1, 1)
infile.truncate()
infile.close()
Note that it does not work on files where the EOL character is not 'n'.
add a comment |Â
up vote
0
down vote
In python (I know it is not what you want, but it is much better as it is optimized, and a prelude to the bash version) without rewriting the file and without reading all the file (which is a good thing if the file is very large):
#!/bin/python
import sys
infile = open(sys.argv[1], 'r+')
infile.seek(-1, 2)
while infile.read(1) == 'n':
infile.seek(-2, 1)
infile.seek(1, 1)
infile.truncate()
infile.close()
Note that it does not work on files where the EOL character is not 'n'.
add a comment |Â
up vote
0
down vote
up vote
0
down vote
In python (I know it is not what you want, but it is much better as it is optimized, and a prelude to the bash version) without rewriting the file and without reading all the file (which is a good thing if the file is very large):
#!/bin/python
import sys
infile = open(sys.argv[1], 'r+')
infile.seek(-1, 2)
while infile.read(1) == 'n':
infile.seek(-2, 1)
infile.seek(1, 1)
infile.truncate()
infile.close()
Note that it does not work on files where the EOL character is not 'n'.
In python (I know it is not what you want, but it is much better as it is optimized, and a prelude to the bash version) without rewriting the file and without reading all the file (which is a good thing if the file is very large):
#!/bin/python
import sys
infile = open(sys.argv[1], 'r+')
infile.seek(-1, 2)
while infile.read(1) == 'n':
infile.seek(-2, 1)
infile.seek(1, 1)
infile.truncate()
infile.close()
Note that it does not work on files where the EOL character is not 'n'.
answered Jul 9 '13 at 10:19
jfg956
3,10711223
3,10711223
add a comment |Â
add a comment |Â
up vote
0
down vote
A bash version, implementing the python algorithm, but less efficient as it needs many processes:
#!/bin/bash
n=1
while test "$(tail -n $n "$1")" == ""; do
((n++))
done
((n--))
truncate -s $(($(stat -c "%s" "$1") - $n)) "$1"
add a comment |Â
up vote
0
down vote
A bash version, implementing the python algorithm, but less efficient as it needs many processes:
#!/bin/bash
n=1
while test "$(tail -n $n "$1")" == ""; do
((n++))
done
((n--))
truncate -s $(($(stat -c "%s" "$1") - $n)) "$1"
add a comment |Â
up vote
0
down vote
up vote
0
down vote
A bash version, implementing the python algorithm, but less efficient as it needs many processes:
#!/bin/bash
n=1
while test "$(tail -n $n "$1")" == ""; do
((n++))
done
((n--))
truncate -s $(($(stat -c "%s" "$1") - $n)) "$1"
A bash version, implementing the python algorithm, but less efficient as it needs many processes:
#!/bin/bash
n=1
while test "$(tail -n $n "$1")" == ""; do
((n++))
done
((n--))
truncate -s $(($(stat -c "%s" "$1") - $n)) "$1"
answered Jul 9 '13 at 10:27
jfg956
3,10711223
3,10711223
add a comment |Â
add a comment |Â
up vote
0
down vote
This one is quick to type, and, if you know sed, easy to remember:
tac < file | sed '/[^[:blank:]]/,$!d' | tac
It uses the sed script to delete leading blank lines from useful one line scripts for sed, referenced by Alexey, above, and tac (reverse cat).
In a quick test, on an 18MB, 64,000 line file, Alexey's approach was faster, (0.036 vs 0.046 seconds).
add a comment |Â
up vote
0
down vote
This one is quick to type, and, if you know sed, easy to remember:
tac < file | sed '/[^[:blank:]]/,$!d' | tac
It uses the sed script to delete leading blank lines from useful one line scripts for sed, referenced by Alexey, above, and tac (reverse cat).
In a quick test, on an 18MB, 64,000 line file, Alexey's approach was faster, (0.036 vs 0.046 seconds).
add a comment |Â
up vote
0
down vote
up vote
0
down vote
This one is quick to type, and, if you know sed, easy to remember:
tac < file | sed '/[^[:blank:]]/,$!d' | tac
It uses the sed script to delete leading blank lines from useful one line scripts for sed, referenced by Alexey, above, and tac (reverse cat).
In a quick test, on an 18MB, 64,000 line file, Alexey's approach was faster, (0.036 vs 0.046 seconds).
This one is quick to type, and, if you know sed, easy to remember:
tac < file | sed '/[^[:blank:]]/,$!d' | tac
It uses the sed script to delete leading blank lines from useful one line scripts for sed, referenced by Alexey, above, and tac (reverse cat).
In a quick test, on an 18MB, 64,000 line file, Alexey's approach was faster, (0.036 vs 0.046 seconds).
edited Aug 31 at 10:22
answered Aug 31 at 10:02
freeB
11
11
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f81685%2fhow-to-remove-multiple-newlines-at-eof%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password