How to delete all lines that contain only duplicates characters?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
-7
down vote

favorite












For example:



22111155
44788
56667775
aannmmmm88
66h1122
PPDd88


The output should be:



44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated


Delete the line that all its characters are repeated at least two times in the same line.



If there was any character not repeated in the same line so print that line only.










share|improve this question



















  • 3




    Is this homework
    – eyoung100
    Sep 28 at 5:50










  • Is this some school exercise?
    – RalfFriedl
    Sep 28 at 5:52










  • interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
    – Sundeep
    Sep 28 at 6:27










  • Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
    – Ahmed
    Sep 28 at 6:33






  • 1




    @Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
    – Sundeep
    Sep 28 at 7:15














up vote
-7
down vote

favorite












For example:



22111155
44788
56667775
aannmmmm88
66h1122
PPDd88


The output should be:



44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated


Delete the line that all its characters are repeated at least two times in the same line.



If there was any character not repeated in the same line so print that line only.










share|improve this question



















  • 3




    Is this homework
    – eyoung100
    Sep 28 at 5:50










  • Is this some school exercise?
    – RalfFriedl
    Sep 28 at 5:52










  • interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
    – Sundeep
    Sep 28 at 6:27










  • Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
    – Ahmed
    Sep 28 at 6:33






  • 1




    @Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
    – Sundeep
    Sep 28 at 7:15












up vote
-7
down vote

favorite









up vote
-7
down vote

favorite











For example:



22111155
44788
56667775
aannmmmm88
66h1122
PPDd88


The output should be:



44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated


Delete the line that all its characters are repeated at least two times in the same line.



If there was any character not repeated in the same line so print that line only.










share|improve this question















For example:



22111155
44788
56667775
aannmmmm88
66h1122
PPDd88


The output should be:



44788 # 7 is not repeated
66h1122 # letter h is not repeated
PPDd88 # letters D or d are not repeated


Delete the line that all its characters are repeated at least two times in the same line.



If there was any character not repeated in the same line so print that line only.







text-processing






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Sep 28 at 9:15









αғsнιη

16k92563




16k92563










asked Sep 28 at 5:46









Ahmed

474




474







  • 3




    Is this homework
    – eyoung100
    Sep 28 at 5:50










  • Is this some school exercise?
    – RalfFriedl
    Sep 28 at 5:52










  • interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
    – Sundeep
    Sep 28 at 6:27










  • Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
    – Ahmed
    Sep 28 at 6:33






  • 1




    @Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
    – Sundeep
    Sep 28 at 7:15












  • 3




    Is this homework
    – eyoung100
    Sep 28 at 5:50










  • Is this some school exercise?
    – RalfFriedl
    Sep 28 at 5:52










  • interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
    – Sundeep
    Sep 28 at 6:27










  • Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
    – Ahmed
    Sep 28 at 6:33






  • 1




    @Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
    – Sundeep
    Sep 28 at 7:15







3




3




Is this homework
– eyoung100
Sep 28 at 5:50




Is this homework
– eyoung100
Sep 28 at 5:50












Is this some school exercise?
– RalfFriedl
Sep 28 at 5:52




Is this some school exercise?
– RalfFriedl
Sep 28 at 5:52












interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
– Sundeep
Sep 28 at 6:27




interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this
– Sundeep
Sep 28 at 6:27












Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
– Ahmed
Sep 28 at 6:33




Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks
– Ahmed
Sep 28 at 6:33




1




1




@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
– Sundeep
Sep 28 at 7:15




@Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work
– Sundeep
Sep 28 at 7:15










5 Answers
5






active

oldest

votes

















up vote
3
down vote













With perl:



perl -ne 'my %count;
$count$_++ for /./g;
print if grep $_ == 1 values %count'


With sed:



sed '
/./!d;h;s/$/
/
:1
s/(.)(.*)1(.*n)/231/
s/(.)1*(.*n.*1)/2/
t1
/^n/d;g'


We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.






share|improve this answer






















  • @Isaac good points thanks. Now edited in.
    – Stéphane Chazelas
    Sep 29 at 7:16

















up vote
2
down vote













perl one-liner: removes all pairs of characters, print the line if there are characters left over.



perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file



As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.






share|improve this answer






















  • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
    – Ahmed
    Sep 28 at 15:54










  • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
    – glenn jackman
    Sep 28 at 18:10










  • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
    – Ahmed
    Sep 29 at 13:55










  • yes, you are quite right.
    – glenn jackman
    Sep 29 at 14:57


















up vote
1
down vote













There is a command that extract only non-repeated words in a list:



$ printf '%sn' one one two | uniq -u
two


You could divide every character in a word on each line and use uniq:



$ echo "1122e4455" | grep -o . | sort | uniq -u
e


All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.






share|improve this answer





























    up vote
    1
    down vote













    Adaption of the solution to your recent problem:



    awk '
    split ("", N) # delete N array
    L = 0 # reset boolean L used for print decision
    for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
    for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
    break # and quit the for loop


    L # print if non-duplicate chars exist
    ' file
    44788
    66h1122
    PPDd88





    share|improve this answer






















    • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
      – Ahmed
      Sep 28 at 10:48






    • 2




      You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
      – RudiC
      Sep 28 at 11:45










    • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
      – Ahmed
      Sep 28 at 13:37

















    up vote
    1
    down vote













    sed -e '
    /n/!h;s/^/n/;
    /^n$/d
    /^n(.).*1/!g;b;
    :b;s/^(n(.).*)2/1/;tb
    s/n./n/;s/^/n/;D
    '


    Explanation:



    • Place a marker n at the beginning of the pattern ,which travels to the right during the process.

    • We setup an infinite loop and provide for two exits inside the loop.

    • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.

    • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.

    • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

    Another way is usingPerl as shown:



    perl -lne '
    my $s = $_;
    s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
    print $s if /./;
    '


    Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).






    share|improve this answer




















      Your Answer







      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: false,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













       

      draft saved


      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f471982%2fhow-to-delete-all-lines-that-contain-only-duplicates-characters%23new-answer', 'question_page');

      );

      Post as a guest






























      5 Answers
      5






      active

      oldest

      votes








      5 Answers
      5






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      3
      down vote













      With perl:



      perl -ne 'my %count;
      $count$_++ for /./g;
      print if grep $_ == 1 values %count'


      With sed:



      sed '
      /./!d;h;s/$/
      /
      :1
      s/(.)(.*)1(.*n)/231/
      s/(.)1*(.*n.*1)/2/
      t1
      /^n/d;g'


      We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.






      share|improve this answer






















      • @Isaac good points thanks. Now edited in.
        – Stéphane Chazelas
        Sep 29 at 7:16














      up vote
      3
      down vote













      With perl:



      perl -ne 'my %count;
      $count$_++ for /./g;
      print if grep $_ == 1 values %count'


      With sed:



      sed '
      /./!d;h;s/$/
      /
      :1
      s/(.)(.*)1(.*n)/231/
      s/(.)1*(.*n.*1)/2/
      t1
      /^n/d;g'


      We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.






      share|improve this answer






















      • @Isaac good points thanks. Now edited in.
        – Stéphane Chazelas
        Sep 29 at 7:16












      up vote
      3
      down vote










      up vote
      3
      down vote









      With perl:



      perl -ne 'my %count;
      $count$_++ for /./g;
      print if grep $_ == 1 values %count'


      With sed:



      sed '
      /./!d;h;s/$/
      /
      :1
      s/(.)(.*)1(.*n)/231/
      s/(.)1*(.*n.*1)/2/
      t1
      /^n/d;g'


      We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.






      share|improve this answer














      With perl:



      perl -ne 'my %count;
      $count$_++ for /./g;
      print if grep $_ == 1 values %count'


      With sed:



      sed '
      /./!d;h;s/$/
      /
      :1
      s/(.)(.*)1(.*n)/231/
      s/(.)1*(.*n.*1)/2/
      t1
      /^n/d;g'


      We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Sep 29 at 7:14

























      answered Sep 28 at 9:24









      Stéphane Chazelas

      287k53529867




      287k53529867











      • @Isaac good points thanks. Now edited in.
        – Stéphane Chazelas
        Sep 29 at 7:16
















      • @Isaac good points thanks. Now edited in.
        – Stéphane Chazelas
        Sep 29 at 7:16















      @Isaac good points thanks. Now edited in.
      – Stéphane Chazelas
      Sep 29 at 7:16




      @Isaac good points thanks. Now edited in.
      – Stéphane Chazelas
      Sep 29 at 7:16












      up vote
      2
      down vote













      perl one-liner: removes all pairs of characters, print the line if there are characters left over.



      perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file



      As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.






      share|improve this answer






















      • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
        – Ahmed
        Sep 28 at 15:54










      • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
        – glenn jackman
        Sep 28 at 18:10










      • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
        – Ahmed
        Sep 29 at 13:55










      • yes, you are quite right.
        – glenn jackman
        Sep 29 at 14:57















      up vote
      2
      down vote













      perl one-liner: removes all pairs of characters, print the line if there are characters left over.



      perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file



      As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.






      share|improve this answer






















      • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
        – Ahmed
        Sep 28 at 15:54










      • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
        – glenn jackman
        Sep 28 at 18:10










      • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
        – Ahmed
        Sep 29 at 13:55










      • yes, you are quite right.
        – glenn jackman
        Sep 29 at 14:57













      up vote
      2
      down vote










      up vote
      2
      down vote









      perl one-liner: removes all pairs of characters, print the line if there are characters left over.



      perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file



      As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.






      share|improve this answer














      perl one-liner: removes all pairs of characters, print the line if there are characters left over.



      perl -lne '($copy = $_) =~ s/(.)1//g; print if $copy' file



      As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Sep 29 at 14:57

























      answered Sep 28 at 13:51









      glenn jackman

      48.3k365105




      48.3k365105











      • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
        – Ahmed
        Sep 28 at 15:54










      • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
        – glenn jackman
        Sep 28 at 18:10










      • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
        – Ahmed
        Sep 29 at 13:55










      • yes, you are quite right.
        – glenn jackman
        Sep 29 at 14:57

















      • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
        – Ahmed
        Sep 28 at 15:54










      • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
        – glenn jackman
        Sep 28 at 18:10










      • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
        – Ahmed
        Sep 29 at 13:55










      • yes, you are quite right.
        – glenn jackman
        Sep 29 at 14:57
















      If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
      – Ahmed
      Sep 28 at 15:54




      If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be?
      – Ahmed
      Sep 28 at 15:54












      I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
      – glenn jackman
      Sep 28 at 18:10




      I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless
      – glenn jackman
      Sep 28 at 18:10












      Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
      – Ahmed
      Sep 29 at 13:55




      Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted.
      – Ahmed
      Sep 29 at 13:55












      yes, you are quite right.
      – glenn jackman
      Sep 29 at 14:57





      yes, you are quite right.
      – glenn jackman
      Sep 29 at 14:57











      up vote
      1
      down vote













      There is a command that extract only non-repeated words in a list:



      $ printf '%sn' one one two | uniq -u
      two


      You could divide every character in a word on each line and use uniq:



      $ echo "1122e4455" | grep -o . | sort | uniq -u
      e


      All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.






      share|improve this answer


























        up vote
        1
        down vote













        There is a command that extract only non-repeated words in a list:



        $ printf '%sn' one one two | uniq -u
        two


        You could divide every character in a word on each line and use uniq:



        $ echo "1122e4455" | grep -o . | sort | uniq -u
        e


        All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.






        share|improve this answer
























          up vote
          1
          down vote










          up vote
          1
          down vote









          There is a command that extract only non-repeated words in a list:



          $ printf '%sn' one one two | uniq -u
          two


          You could divide every character in a word on each line and use uniq:



          $ echo "1122e4455" | grep -o . | sort | uniq -u
          e


          All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.






          share|improve this answer














          There is a command that extract only non-repeated words in a list:



          $ printf '%sn' one one two | uniq -u
          two


          You could divide every character in a word on each line and use uniq:



          $ echo "1122e4455" | grep -o . | sort | uniq -u
          e


          All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Sep 28 at 7:15

























          answered Sep 28 at 6:59









          Isaac

          7,61511137




          7,61511137




















              up vote
              1
              down vote













              Adaption of the solution to your recent problem:



              awk '
              split ("", N) # delete N array
              L = 0 # reset boolean L used for print decision
              for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
              for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
              break # and quit the for loop


              L # print if non-duplicate chars exist
              ' file
              44788
              66h1122
              PPDd88





              share|improve this answer






















              • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
                – Ahmed
                Sep 28 at 10:48






              • 2




                You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
                – RudiC
                Sep 28 at 11:45










              • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
                – Ahmed
                Sep 28 at 13:37














              up vote
              1
              down vote













              Adaption of the solution to your recent problem:



              awk '
              split ("", N) # delete N array
              L = 0 # reset boolean L used for print decision
              for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
              for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
              break # and quit the for loop


              L # print if non-duplicate chars exist
              ' file
              44788
              66h1122
              PPDd88





              share|improve this answer






















              • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
                – Ahmed
                Sep 28 at 10:48






              • 2




                You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
                – RudiC
                Sep 28 at 11:45










              • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
                – Ahmed
                Sep 28 at 13:37












              up vote
              1
              down vote










              up vote
              1
              down vote









              Adaption of the solution to your recent problem:



              awk '
              split ("", N) # delete N array
              L = 0 # reset boolean L used for print decision
              for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
              for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
              break # and quit the for loop


              L # print if non-duplicate chars exist
              ' file
              44788
              66h1122
              PPDd88





              share|improve this answer














              Adaption of the solution to your recent problem:



              awk '
              split ("", N) # delete N array
              L = 0 # reset boolean L used for print decision
              for (i=1; i<=length; i++) N[substr($0, i, 1)]++ # calculate count of characters
              for (n in N) if (N[n] < 2) L = 1 # for non-duplicate chars: set print decision
              break # and quit the for loop


              L # print if non-duplicate chars exist
              ' file
              44788
              66h1122
              PPDd88






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Sep 28 at 11:43

























              answered Sep 28 at 9:01









              RudiC

              1,8329




              1,8329











              • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
                – Ahmed
                Sep 28 at 10:48






              • 2




                You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
                – RudiC
                Sep 28 at 11:45










              • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
                – Ahmed
                Sep 28 at 13:37
















              • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
                – Ahmed
                Sep 28 at 10:48






              • 2




                You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
                – RudiC
                Sep 28 at 11:45










              • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
                – Ahmed
                Sep 28 at 13:37















              RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
              – Ahmed
              Sep 28 at 10:48




              RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line.
              – Ahmed
              Sep 28 at 10:48




              2




              2




              You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
              – RudiC
              Sep 28 at 11:45




              You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own.
              – RudiC
              Sep 28 at 11:45












              RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
              – Ahmed
              Sep 28 at 13:37




              RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one
              – Ahmed
              Sep 28 at 13:37










              up vote
              1
              down vote













              sed -e '
              /n/!h;s/^/n/;
              /^n$/d
              /^n(.).*1/!g;b;
              :b;s/^(n(.).*)2/1/;tb
              s/n./n/;s/^/n/;D
              '


              Explanation:



              • Place a marker n at the beginning of the pattern ,which travels to the right during the process.

              • We setup an infinite loop and provide for two exits inside the loop.

              • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.

              • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.

              • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

              Another way is usingPerl as shown:



              perl -lne '
              my $s = $_;
              s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
              print $s if /./;
              '


              Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).






              share|improve this answer
























                up vote
                1
                down vote













                sed -e '
                /n/!h;s/^/n/;
                /^n$/d
                /^n(.).*1/!g;b;
                :b;s/^(n(.).*)2/1/;tb
                s/n./n/;s/^/n/;D
                '


                Explanation:



                • Place a marker n at the beginning of the pattern ,which travels to the right during the process.

                • We setup an infinite loop and provide for two exits inside the loop.

                • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.

                • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.

                • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

                Another way is usingPerl as shown:



                perl -lne '
                my $s = $_;
                s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
                print $s if /./;
                '


                Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).






                share|improve this answer






















                  up vote
                  1
                  down vote










                  up vote
                  1
                  down vote









                  sed -e '
                  /n/!h;s/^/n/;
                  /^n$/d
                  /^n(.).*1/!g;b;
                  :b;s/^(n(.).*)2/1/;tb
                  s/n./n/;s/^/n/;D
                  '


                  Explanation:



                  • Place a marker n at the beginning of the pattern ,which travels to the right during the process.

                  • We setup an infinite loop and provide for two exits inside the loop.

                  • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.

                  • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.

                  • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

                  Another way is usingPerl as shown:



                  perl -lne '
                  my $s = $_;
                  s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
                  print $s if /./;
                  '


                  Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).






                  share|improve this answer












                  sed -e '
                  /n/!h;s/^/n/;
                  /^n$/d
                  /^n(.).*1/!g;b;
                  :b;s/^(n(.).*)2/1/;tb
                  s/n./n/;s/^/n/;D
                  '


                  Explanation:



                  • Place a marker n at the beginning of the pattern ,which travels to the right during the process.

                  • We setup an infinite loop and provide for two exits inside the loop.

                  • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.

                  • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.

                  • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

                  Another way is usingPerl as shown:



                  perl -lne '
                  my $s = $_;
                  s/Q$1E//g while $s =~ /(.)(?=.*?1)/g;
                  print $s if /./;
                  '


                  Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Sep 29 at 12:09









                  Rakesh Sharma

                  64513




                  64513



























                       

                      draft saved


                      draft discarded















































                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f471982%2fhow-to-delete-all-lines-that-contain-only-duplicates-characters%23new-answer', 'question_page');

                      );

                      Post as a guest













































































                      Popular posts from this blog

                      How to check contact read email or not when send email to Individual?

                      Displaying single band from multi-band raster using QGIS

                      How many registers does an x86_64 CPU actually have?