How to delete all lines that contain only duplicates characters?
Clash Royale CLAN TAG#URR8PPP
up vote
-7
down vote
favorite
For example:
22111155
44788
56667775
aannmmmm88
66h1122
PPDd88
The output should be:
44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated
Delete the line that all its characters are repeated at least two times in the same line.
If there was any character not repeated in the same line so print that line only.
text-processing
 |Â
show 1 more comment
up vote
-7
down vote
favorite
For example:
22111155
44788
56667775
aannmmmm88
66h1122
PPDd88
The output should be:
44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated
Delete the line that all its characters are repeated at least two times in the same line.
If there was any character not repeated in the same line so print that line only.
text-processing
3
Is this homework
â eyoung100
Sep 28 at 5:50
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
1
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15
 |Â
show 1 more comment
up vote
-7
down vote
favorite
up vote
-7
down vote
favorite
For example:
22111155
44788
56667775
aannmmmm88
66h1122
PPDd88
The output should be:
44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated
Delete the line that all its characters are repeated at least two times in the same line.
If there was any character not repeated in the same line so print that line only.
text-processing
For example:
22111155
44788
56667775
aannmmmm88
66h1122
PPDd88
The output should be:
44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated
Delete the line that all its characters are repeated at least two times in the same line.
If there was any character not repeated in the same line so print that line only.
text-processing
text-processing
edited Sep 28 at 9:15
ñÃÂsýù÷
16k92563
16k92563
asked Sep 28 at 5:46
Ahmed
474
474
3
Is this homework
â eyoung100
Sep 28 at 5:50
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
1
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15
 |Â
show 1 more comment
3
Is this homework
â eyoung100
Sep 28 at 5:50
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
1
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15
3
3
Is this homework
â eyoung100
Sep 28 at 5:50
Is this homework
â eyoung100
Sep 28 at 5:50
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
1
1
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15
 |Â
show 1 more comment
5 Answers
5
active
oldest
votes
up vote
3
down vote
With perl
:
perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'
With sed
:
sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'
We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
add a comment |Â
up vote
2
down vote
perl one-liner: removes all pairs of characters, print the line if there are characters left over.
perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file
As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, changeif
tounless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
add a comment |Â
up vote
1
down vote
There is a command that extract only non-repeated words in a list:
$ printf '%sn' one one two | uniq -u
two
You could divide every character in a word on each line and use uniq:
$ echo "1122e4455" | grep -o . | sort | uniq -u
e
All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.
add a comment |Â
up vote
1
down vote
Adaption of the solution to your recent problem:
awk '
split ("", N) # delete N array
L = 0 # reset boolean L used for print decision
for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
break # and quit the for loop
L # print if non-duplicate chars exist
' file
44788
66h1122
PPDd88
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
You didn't specify having more than one field in your input file. Change$1
to$0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
add a comment |Â
up vote
1
down vote
sed -e '
/n/!h;s/^/n/;
/^n$/d
/^n(.).*1/!g;b;
:b;s/^(n(.).*)2/1/;tb
s/n./n/;s/^/n/;D
'
Explanation:
- Place a marker
n
at the beginning of the pattern ,which travels to the right during the process. - We setup an infinite loop and provide for two exits inside the loop.
- One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
- Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
- The alternative is that the first element has at least a duplicate somewhere in the string which we go about in
t-loop
totally purging that element's presence from the string.
Another way is usingPerl
as shown:
perl -lne '
my $s = $_;
s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
print $s if /./;
'
Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).
add a comment |Â
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
With perl
:
perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'
With sed
:
sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'
We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
add a comment |Â
up vote
3
down vote
With perl
:
perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'
With sed
:
sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'
We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
add a comment |Â
up vote
3
down vote
up vote
3
down vote
With perl
:
perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'
With sed
:
sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'
We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.
With perl
:
perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'
With sed
:
sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'
We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.
edited Sep 29 at 7:14
answered Sep 28 at 9:24
Stéphane Chazelas
287k53529867
287k53529867
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
add a comment |Â
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
@Isaac good points thanks. Now edited in.
â Stéphane Chazelas
Sep 29 at 7:16
add a comment |Â
up vote
2
down vote
perl one-liner: removes all pairs of characters, print the line if there are characters left over.
perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file
As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, changeif
tounless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
add a comment |Â
up vote
2
down vote
perl one-liner: removes all pairs of characters, print the line if there are characters left over.
perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file
As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, changeif
tounless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
add a comment |Â
up vote
2
down vote
up vote
2
down vote
perl one-liner: removes all pairs of characters, print the line if there are characters left over.
perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file
As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.
perl one-liner: removes all pairs of characters, print the line if there are characters left over.
perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file
As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.
edited Sep 29 at 14:57
answered Sep 28 at 13:51
glenn jackman
48.3k365105
48.3k365105
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, changeif
tounless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
add a comment |Â
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, changeif
tounless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
â Ahmed
Sep 28 at 15:54
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change
if
to unless
â glenn jackman
Sep 28 at 18:10
I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change
if
to unless
â glenn jackman
Sep 28 at 18:10
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
â Ahmed
Sep 29 at 13:55
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
yes, you are quite right.
â glenn jackman
Sep 29 at 14:57
add a comment |Â
up vote
1
down vote
There is a command that extract only non-repeated words in a list:
$ printf '%sn' one one two | uniq -u
two
You could divide every character in a word on each line and use uniq:
$ echo "1122e4455" | grep -o . | sort | uniq -u
e
All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.
add a comment |Â
up vote
1
down vote
There is a command that extract only non-repeated words in a list:
$ printf '%sn' one one two | uniq -u
two
You could divide every character in a word on each line and use uniq:
$ echo "1122e4455" | grep -o . | sort | uniq -u
e
All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.
add a comment |Â
up vote
1
down vote
up vote
1
down vote
There is a command that extract only non-repeated words in a list:
$ printf '%sn' one one two | uniq -u
two
You could divide every character in a word on each line and use uniq:
$ echo "1122e4455" | grep -o . | sort | uniq -u
e
All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.
There is a command that extract only non-repeated words in a list:
$ printf '%sn' one one two | uniq -u
two
You could divide every character in a word on each line and use uniq:
$ echo "1122e4455" | grep -o . | sort | uniq -u
e
All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.
edited Sep 28 at 7:15
answered Sep 28 at 6:59
Isaac
7,61511137
7,61511137
add a comment |Â
add a comment |Â
up vote
1
down vote
Adaption of the solution to your recent problem:
awk '
split ("", N) # delete N array
L = 0 # reset boolean L used for print decision
for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
break # and quit the for loop
L # print if non-duplicate chars exist
' file
44788
66h1122
PPDd88
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
You didn't specify having more than one field in your input file. Change$1
to$0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
add a comment |Â
up vote
1
down vote
Adaption of the solution to your recent problem:
awk '
split ("", N) # delete N array
L = 0 # reset boolean L used for print decision
for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
break # and quit the for loop
L # print if non-duplicate chars exist
' file
44788
66h1122
PPDd88
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
You didn't specify having more than one field in your input file. Change$1
to$0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
add a comment |Â
up vote
1
down vote
up vote
1
down vote
Adaption of the solution to your recent problem:
awk '
split ("", N) # delete N array
L = 0 # reset boolean L used for print decision
for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
break # and quit the for loop
L # print if non-duplicate chars exist
' file
44788
66h1122
PPDd88
Adaption of the solution to your recent problem:
awk '
split ("", N) # delete N array
L = 0 # reset boolean L used for print decision
for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
break # and quit the for loop
L # print if non-duplicate chars exist
' file
44788
66h1122
PPDd88
edited Sep 28 at 11:43
answered Sep 28 at 9:01
RudiC
1,8329
1,8329
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
You didn't specify having more than one field in your input file. Change$1
to$0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
add a comment |Â
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
You didn't specify having more than one field in your input file. Change$1
to$0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
â Ahmed
Sep 28 at 10:48
2
2
You didn't specify having more than one field in your input file. Change
$1
to $0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.â RudiC
Sep 28 at 11:45
You didn't specify having more than one field in your input file. Change
$1
to $0
(done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.â RudiC
Sep 28 at 11:45
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
â Ahmed
Sep 28 at 13:37
add a comment |Â
up vote
1
down vote
sed -e '
/n/!h;s/^/n/;
/^n$/d
/^n(.).*1/!g;b;
:b;s/^(n(.).*)2/1/;tb
s/n./n/;s/^/n/;D
'
Explanation:
- Place a marker
n
at the beginning of the pattern ,which travels to the right during the process. - We setup an infinite loop and provide for two exits inside the loop.
- One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
- Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
- The alternative is that the first element has at least a duplicate somewhere in the string which we go about in
t-loop
totally purging that element's presence from the string.
Another way is usingPerl
as shown:
perl -lne '
my $s = $_;
s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
print $s if /./;
'
Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).
add a comment |Â
up vote
1
down vote
sed -e '
/n/!h;s/^/n/;
/^n$/d
/^n(.).*1/!g;b;
:b;s/^(n(.).*)2/1/;tb
s/n./n/;s/^/n/;D
'
Explanation:
- Place a marker
n
at the beginning of the pattern ,which travels to the right during the process. - We setup an infinite loop and provide for two exits inside the loop.
- One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
- Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
- The alternative is that the first element has at least a duplicate somewhere in the string which we go about in
t-loop
totally purging that element's presence from the string.
Another way is usingPerl
as shown:
perl -lne '
my $s = $_;
s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
print $s if /./;
'
Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).
add a comment |Â
up vote
1
down vote
up vote
1
down vote
sed -e '
/n/!h;s/^/n/;
/^n$/d
/^n(.).*1/!g;b;
:b;s/^(n(.).*)2/1/;tb
s/n./n/;s/^/n/;D
'
Explanation:
- Place a marker
n
at the beginning of the pattern ,which travels to the right during the process. - We setup an infinite loop and provide for two exits inside the loop.
- One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
- Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
- The alternative is that the first element has at least a duplicate somewhere in the string which we go about in
t-loop
totally purging that element's presence from the string.
Another way is usingPerl
as shown:
perl -lne '
my $s = $_;
s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
print $s if /./;
'
Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).
sed -e '
/n/!h;s/^/n/;
/^n$/d
/^n(.).*1/!g;b;
:b;s/^(n(.).*)2/1/;tb
s/n./n/;s/^/n/;D
'
Explanation:
- Place a marker
n
at the beginning of the pattern ,which travels to the right during the process. - We setup an infinite loop and provide for two exits inside the loop.
- One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
- Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
- The alternative is that the first element has at least a duplicate somewhere in the string which we go about in
t-loop
totally purging that element's presence from the string.
Another way is usingPerl
as shown:
perl -lne '
my $s = $_;
s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
print $s if /./;
'
Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).
answered Sep 29 at 12:09
Rakesh Sharma
64513
64513
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f471982%2fhow-to-delete-all-lines-that-contain-only-duplicates-characters%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
3
Is this homework
â eyoung100
Sep 28 at 5:50
Is this some school exercise?
â RalfFriedl
Sep 28 at 5:52
interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
â Sundeep
Sep 28 at 6:27
Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
â Ahmed
Sep 28 at 6:33
1
@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
â Sundeep
Sep 28 at 7:15