removing “na” from a column and collapsing the row

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












0















I would like to remove rows that have an "na"



file



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003


output



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003


I was trying to do this in R but i couldn't collapse the rows, just remove the na










share|improve this question



















  • 4





    With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

    – Stefan Hamcke
    Dec 29 '18 at 21:22












  • If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

    – steeldriver
    Dec 29 '18 at 21:51
















0















I would like to remove rows that have an "na"



file



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003


output



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003


I was trying to do this in R but i couldn't collapse the rows, just remove the na










share|improve this question



















  • 4





    With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

    – Stefan Hamcke
    Dec 29 '18 at 21:22












  • If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

    – steeldriver
    Dec 29 '18 at 21:51














0












0








0








I would like to remove rows that have an "na"



file



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003


output



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003


I was trying to do this in R but i couldn't collapse the rows, just remove the na










share|improve this question
















I would like to remove rows that have an "na"



file



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
na
na
0.000
0.000
na
0.002
0.002
0.003


output



0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003


I was trying to do this in R but i couldn't collapse the rows, just remove the na







awk sed grep






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Dec 29 '18 at 23:42









Rui F Ribeiro

39.4k1479131




39.4k1479131










asked Dec 29 '18 at 21:21









Age87Age87

1477




1477







  • 4





    With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

    – Stefan Hamcke
    Dec 29 '18 at 21:22












  • If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

    – steeldriver
    Dec 29 '18 at 21:51













  • 4





    With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

    – Stefan Hamcke
    Dec 29 '18 at 21:22












  • If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

    – steeldriver
    Dec 29 '18 at 21:51








4




4





With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

– Stefan Hamcke
Dec 29 '18 at 21:22






With grep -v na. Removes all lines containing the string na.The -v is the short form for --invert-match.

– Stefan Hamcke
Dec 29 '18 at 21:22














If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

– steeldriver
Dec 29 '18 at 21:51






If you decide you do want to do it in R, it would be something like data = read.table("data.txt", na.strings = "na") then data = na.omit(data) I think

– steeldriver
Dec 29 '18 at 21:51











3 Answers
3






active

oldest

votes


















1














With awk:



awk '!/na/' file > new_file


The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:



awk '!/na/ print' file > new_file


To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.



awk '!/na/ && !/^$/ print' file > new_file


With sed:



sed '/na/d' file > new_file


The "/d tells sed to delete the line containing the matched pattern "/na/"



To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed



sed -i '/na/d' file


Using nothing but the bash shell with bash builtin:



while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file


With Python:



with open('file', 'r') as rf:
with open('out', 'w') as of:
for line in rf:
if not "na" in line:
of.write(line)


As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.



$ cat file
0.000
0.000
0.055
0.036
0.003
0.002
0.000
0.002
0.002
0.002
0.000
0.000
0.000
0.002
0.002
0.003





share|improve this answer
































    0














    As simple as:



    grep -v na file > new_file





    share|improve this answer























    • You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

      – Kusalananda
      Jan 7 at 22:34


















    0














    I tried with below 3 methods

    1.using awk

    command:awk '$0 !~/na/print $0' filename

    2. using sed

    sed -n '/na/!p' filename

    3. using python

    #!/usr/bin/python
    import re
    k=open('filename','r')
    a=re.compile(r'[0-9]')
    for i in k:
    if re.search(a,i):
    print i,


    output



    0.000
    0.000
    0.055
    0.036
    0.003
    0.002
    0.000
    0.002
    0.002
    0.002
    0.000
    0.000
    0.000
    0.002
    0.002
    0.003





    share|improve this answer






















      Your Answer








      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491519%2fremoving-na-from-a-column-and-collapsing-the-row%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      3 Answers
      3






      active

      oldest

      votes








      3 Answers
      3






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      1














      With awk:



      awk '!/na/' file > new_file


      The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:



      awk '!/na/ print' file > new_file


      To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.



      awk '!/na/ && !/^$/ print' file > new_file


      With sed:



      sed '/na/d' file > new_file


      The "/d tells sed to delete the line containing the matched pattern "/na/"



      To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed



      sed -i '/na/d' file


      Using nothing but the bash shell with bash builtin:



      while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file


      With Python:



      with open('file', 'r') as rf:
      with open('out', 'w') as of:
      for line in rf:
      if not "na" in line:
      of.write(line)


      As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.



      $ cat file
      0.000
      0.000
      0.055
      0.036
      0.003
      0.002
      0.000
      0.002
      0.002
      0.002
      0.000
      0.000
      0.000
      0.002
      0.002
      0.003





      share|improve this answer





























        1














        With awk:



        awk '!/na/' file > new_file


        The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:



        awk '!/na/ print' file > new_file


        To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.



        awk '!/na/ && !/^$/ print' file > new_file


        With sed:



        sed '/na/d' file > new_file


        The "/d tells sed to delete the line containing the matched pattern "/na/"



        To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed



        sed -i '/na/d' file


        Using nothing but the bash shell with bash builtin:



        while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file


        With Python:



        with open('file', 'r') as rf:
        with open('out', 'w') as of:
        for line in rf:
        if not "na" in line:
        of.write(line)


        As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.



        $ cat file
        0.000
        0.000
        0.055
        0.036
        0.003
        0.002
        0.000
        0.002
        0.002
        0.002
        0.000
        0.000
        0.000
        0.002
        0.002
        0.003





        share|improve this answer



























          1












          1








          1







          With awk:



          awk '!/na/' file > new_file


          The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:



          awk '!/na/ print' file > new_file


          To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.



          awk '!/na/ && !/^$/ print' file > new_file


          With sed:



          sed '/na/d' file > new_file


          The "/d tells sed to delete the line containing the matched pattern "/na/"



          To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed



          sed -i '/na/d' file


          Using nothing but the bash shell with bash builtin:



          while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file


          With Python:



          with open('file', 'r') as rf:
          with open('out', 'w') as of:
          for line in rf:
          if not "na" in line:
          of.write(line)


          As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.



          $ cat file
          0.000
          0.000
          0.055
          0.036
          0.003
          0.002
          0.000
          0.002
          0.002
          0.002
          0.000
          0.000
          0.000
          0.002
          0.002
          0.003





          share|improve this answer















          With awk:



          awk '!/na/' file > new_file


          The "!" means do not match. The "/na/" specifies the pattern "na". Together they mean do not match lines containing the pattern na. The default behavior of awk is to print, but you can also specifically tell it to print (call the print function) like this:



          awk '!/na/ print' file > new_file


          To not print the empty lines in the file, use this regular expression. !/^$/ This tells awk not to match on an empty line. It will match by starting from the beginning of the line designated by a "^" to the end of the line designated with a "$" with no content in between. The "&&" tells it to use both expressions when evaluating each line.



          awk '!/na/ && !/^$/ print' file > new_file


          With sed:



          sed '/na/d' file > new_file


          The "/d tells sed to delete the line containing the matched pattern "/na/"



          To directly modify the source file, without needing to write to a new file, use the -i (in place) option with sed



          sed -i '/na/d' file


          Using nothing but the bash shell with bash builtin:



          while read -r line; do [[ ! $line =~ na ]] && echo "$line"; done < file > o; mv o file


          With Python:



          with open('file', 'r') as rf:
          with open('out', 'w') as of:
          for line in rf:
          if not "na" in line:
          of.write(line)


          As with all of the other examples, this will not leave a blank line, given the sample input provided in the original question.



          $ cat file
          0.000
          0.000
          0.055
          0.036
          0.003
          0.002
          0.000
          0.002
          0.002
          0.002
          0.000
          0.000
          0.000
          0.002
          0.002
          0.003






          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Dec 30 '18 at 11:32

























          answered Dec 30 '18 at 10:41









          Chris SmithChris Smith

          564




          564























              0














              As simple as:



              grep -v na file > new_file





              share|improve this answer























              • You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

                – Kusalananda
                Jan 7 at 22:34















              0














              As simple as:



              grep -v na file > new_file





              share|improve this answer























              • You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

                – Kusalananda
                Jan 7 at 22:34













              0












              0








              0







              As simple as:



              grep -v na file > new_file





              share|improve this answer













              As simple as:



              grep -v na file > new_file






              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Dec 30 '18 at 1:47









              Emilio GalarragaEmilio Galarraga

              51129




              51129












              • You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

                – Kusalananda
                Jan 7 at 22:34

















              • You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

                – Kusalananda
                Jan 7 at 22:34
















              You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

              – Kusalananda
              Jan 7 at 22:34





              You'd better use -x with grep here to only match across complete lines. I would probably use grep -x -F -v na to be absolutely correct.

              – Kusalananda
              Jan 7 at 22:34











              0














              I tried with below 3 methods

              1.using awk

              command:awk '$0 !~/na/print $0' filename

              2. using sed

              sed -n '/na/!p' filename

              3. using python

              #!/usr/bin/python
              import re
              k=open('filename','r')
              a=re.compile(r'[0-9]')
              for i in k:
              if re.search(a,i):
              print i,


              output



              0.000
              0.000
              0.055
              0.036
              0.003
              0.002
              0.000
              0.002
              0.002
              0.002
              0.000
              0.000
              0.000
              0.002
              0.002
              0.003





              share|improve this answer



























                0














                I tried with below 3 methods

                1.using awk

                command:awk '$0 !~/na/print $0' filename

                2. using sed

                sed -n '/na/!p' filename

                3. using python

                #!/usr/bin/python
                import re
                k=open('filename','r')
                a=re.compile(r'[0-9]')
                for i in k:
                if re.search(a,i):
                print i,


                output



                0.000
                0.000
                0.055
                0.036
                0.003
                0.002
                0.000
                0.002
                0.002
                0.002
                0.000
                0.000
                0.000
                0.002
                0.002
                0.003





                share|improve this answer

























                  0












                  0








                  0







                  I tried with below 3 methods

                  1.using awk

                  command:awk '$0 !~/na/print $0' filename

                  2. using sed

                  sed -n '/na/!p' filename

                  3. using python

                  #!/usr/bin/python
                  import re
                  k=open('filename','r')
                  a=re.compile(r'[0-9]')
                  for i in k:
                  if re.search(a,i):
                  print i,


                  output



                  0.000
                  0.000
                  0.055
                  0.036
                  0.003
                  0.002
                  0.000
                  0.002
                  0.002
                  0.002
                  0.000
                  0.000
                  0.000
                  0.002
                  0.002
                  0.003





                  share|improve this answer













                  I tried with below 3 methods

                  1.using awk

                  command:awk '$0 !~/na/print $0' filename

                  2. using sed

                  sed -n '/na/!p' filename

                  3. using python

                  #!/usr/bin/python
                  import re
                  k=open('filename','r')
                  a=re.compile(r'[0-9]')
                  for i in k:
                  if re.search(a,i):
                  print i,


                  output



                  0.000
                  0.000
                  0.055
                  0.036
                  0.003
                  0.002
                  0.000
                  0.002
                  0.002
                  0.002
                  0.000
                  0.000
                  0.000
                  0.002
                  0.002
                  0.003






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Dec 31 '18 at 5:27









                  Praveen Kumar BSPraveen Kumar BS

                  1,250138




                  1,250138



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491519%2fremoving-na-from-a-column-and-collapsing-the-row%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown






                      Popular posts from this blog

                      Peggy Mitchell

                      Palaiologos

                      The Forum (Inglewood, California)