How to pipe an output of a list as the input of grep in Linux?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite












I have a table_1



A D G
B E H
C F I


I print the column 1 of the above table using awk 'print $1' table_1 to get list_1



A
B
C


I want to use list_1 above to grep -f from table_2



A n m
B m n
C n m
D m n
E n m


to get table_3



A n m
B m n
C n m


but I want to do it using a one-liner, not needing to save an intermediate file of list_1.



How can I structure the command?







share|improve this question


















  • 2




    grep -f <(awk 'print $1' table_1) table_2?
    – Cyrus
    Feb 1 at 6:31










  • Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
    – Philippos
    Feb 1 at 6:46














up vote
0
down vote

favorite












I have a table_1



A D G
B E H
C F I


I print the column 1 of the above table using awk 'print $1' table_1 to get list_1



A
B
C


I want to use list_1 above to grep -f from table_2



A n m
B m n
C n m
D m n
E n m


to get table_3



A n m
B m n
C n m


but I want to do it using a one-liner, not needing to save an intermediate file of list_1.



How can I structure the command?







share|improve this question


















  • 2




    grep -f <(awk 'print $1' table_1) table_2?
    – Cyrus
    Feb 1 at 6:31










  • Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
    – Philippos
    Feb 1 at 6:46












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have a table_1



A D G
B E H
C F I


I print the column 1 of the above table using awk 'print $1' table_1 to get list_1



A
B
C


I want to use list_1 above to grep -f from table_2



A n m
B m n
C n m
D m n
E n m


to get table_3



A n m
B m n
C n m


but I want to do it using a one-liner, not needing to save an intermediate file of list_1.



How can I structure the command?







share|improve this question














I have a table_1



A D G
B E H
C F I


I print the column 1 of the above table using awk 'print $1' table_1 to get list_1



A
B
C


I want to use list_1 above to grep -f from table_2



A n m
B m n
C n m
D m n
E n m


to get table_3



A n m
B m n
C n m


but I want to do it using a one-liner, not needing to save an intermediate file of list_1.



How can I structure the command?









share|improve this question













share|improve this question




share|improve this question








edited Feb 1 at 9:13









αғsнιη

15k82462




15k82462










asked Feb 1 at 5:45









Johnny Tam

1208




1208







  • 2




    grep -f <(awk 'print $1' table_1) table_2?
    – Cyrus
    Feb 1 at 6:31










  • Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
    – Philippos
    Feb 1 at 6:46












  • 2




    grep -f <(awk 'print $1' table_1) table_2?
    – Cyrus
    Feb 1 at 6:31










  • Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
    – Philippos
    Feb 1 at 6:46







2




2




grep -f <(awk 'print $1' table_1) table_2?
– Cyrus
Feb 1 at 6:31




grep -f <(awk 'print $1' table_1) table_2?
– Cyrus
Feb 1 at 6:31












Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
– Philippos
Feb 1 at 6:46




Is it save to assume that A or any search term can't be part of any other column of table_2? Maybe you should add a ^ before and a whitespace after your search terms.
– Philippos
Feb 1 at 6:46










4 Answers
4






active

oldest

votes

















up vote
3
down vote













A trivial (but not bash-specific) variation on Cyrus’s comment:



awk 'print $1' table_1 | grep -f - table_2


which uses the widespread convention that a filename of -
means “read from the standard input”.






share|improve this answer




















  • but this is not so robust... grep will search any where in the line and would even match part of column...
    – Sundeep
    Feb 1 at 8:22






  • 1




    Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
    – G-Man
    Feb 1 at 9:37

















up vote
2
down vote













Use awk:



awk 'NR==FNRa[$1];next ($1 in a)' table_1 table_2





share|improve this answer





























    up vote
    1
    down vote













    Here’s another way to use join. 
    If the files are sorted, and table_2 really has only three columns,
    you can use



    join -o "2.1 2.2 2.3" table_1 table_2 > table_3


    As Kusalananda says,
    join is a program that exists specifically to combine two files
    by matching values from one column of the first file
    against values from one column of the second file. 
    By default, if uses the first column of each file (you can override this). 
    By default, it combines the matching lines, like this:



    $ join table_1 table_2
    A D G n m
    B E H m n
    C F I n m


    The -o "2.1 2.2 2.3" says “output the first field of the second file,
    and then the second field of the second file,
    and then the first third of the second file
    (but nothing from the first file1)”. 
    Unfortunately, there don’t seem to be any shortcuts, shorthands,
    accelerators, or wildcards here;
    there’s no way to say “output the entire line from the second file”. 
    If the second file has many fields,
    then the -o format will have to be very long.



    This differs from the grep-based solutions (yours and mine)
    in that it specifically matches the values from column 1 of table_1
    against the values from column 1 of table_2,
    and will not give you lines from table_2
    that have an A in column 2 or column 3, or an AZ in column 1. 
    join, like grep, recognizes a -i (--ignore-case) option,
    so, if the first line of table_2 is a n m (with a lower-case a),
    you will get that line in your table_3 output
    if (and only if) you specify -i.

    ________
    1 Except for the first field of the first file —
    which is the same as first field of the second file.






    share|improve this answer





























      up vote
      0
      down vote













      Assuming that both tables are sorted on the first column, and using a shell like bash or ksh93 that understands process substitution with <(...):



      $ join <( awk ' print $1 ' table_1 ) table_2
      A n m
      B m n
      C n m


      If the tables are not sorted, then we need to sort them:



      $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 )
      A n m
      B m n
      C n m


      The join utility performs a relational INNER JOIN operation on the first column (by default) of the two files that you give it.



      Redirect the output to a new file to create table_3:



      $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 ) >table_3





      share|improve this answer




















        Your Answer







        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "106"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        convertImagesToLinks: false,
        noModals: false,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );








         

        draft saved


        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f421143%2fhow-to-pipe-an-output-of-a-list-as-the-input-of-grep-in-linux%23new-answer', 'question_page');

        );

        Post as a guest






























        4 Answers
        4






        active

        oldest

        votes








        4 Answers
        4






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes








        up vote
        3
        down vote













        A trivial (but not bash-specific) variation on Cyrus’s comment:



        awk 'print $1' table_1 | grep -f - table_2


        which uses the widespread convention that a filename of -
        means “read from the standard input”.






        share|improve this answer




















        • but this is not so robust... grep will search any where in the line and would even match part of column...
          – Sundeep
          Feb 1 at 8:22






        • 1




          Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
          – G-Man
          Feb 1 at 9:37














        up vote
        3
        down vote













        A trivial (but not bash-specific) variation on Cyrus’s comment:



        awk 'print $1' table_1 | grep -f - table_2


        which uses the widespread convention that a filename of -
        means “read from the standard input”.






        share|improve this answer




















        • but this is not so robust... grep will search any where in the line and would even match part of column...
          – Sundeep
          Feb 1 at 8:22






        • 1




          Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
          – G-Man
          Feb 1 at 9:37












        up vote
        3
        down vote










        up vote
        3
        down vote









        A trivial (but not bash-specific) variation on Cyrus’s comment:



        awk 'print $1' table_1 | grep -f - table_2


        which uses the widespread convention that a filename of -
        means “read from the standard input”.






        share|improve this answer












        A trivial (but not bash-specific) variation on Cyrus’s comment:



        awk 'print $1' table_1 | grep -f - table_2


        which uses the widespread convention that a filename of -
        means “read from the standard input”.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Feb 1 at 7:34









        G-Man

        11.5k82657




        11.5k82657











        • but this is not so robust... grep will search any where in the line and would even match part of column...
          – Sundeep
          Feb 1 at 8:22






        • 1




          Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
          – G-Man
          Feb 1 at 9:37
















        • but this is not so robust... grep will search any where in the line and would even match part of column...
          – Sundeep
          Feb 1 at 8:22






        • 1




          Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
          – G-Man
          Feb 1 at 9:37















        but this is not so robust... grep will search any where in the line and would even match part of column...
        – Sundeep
        Feb 1 at 8:22




        but this is not so robust... grep will search any where in the line and would even match part of column...
        – Sundeep
        Feb 1 at 8:22




        1




        1




        Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
        – G-Man
        Feb 1 at 9:37




        Where does the question specify which behavior is desired? The question says that the OP’s current workflow is to create the temporary list_1 file and use it (implicitly, in grep -f list_1 table_2), and asks how to streamline that operation into a “one-liner” with no temporary file.  It doesn’t say that the OP wants to use the values from column 1 of table_1 to match against the values from column 1 of table_2.
        – G-Man
        Feb 1 at 9:37












        up vote
        2
        down vote













        Use awk:



        awk 'NR==FNRa[$1];next ($1 in a)' table_1 table_2





        share|improve this answer


























          up vote
          2
          down vote













          Use awk:



          awk 'NR==FNRa[$1];next ($1 in a)' table_1 table_2





          share|improve this answer
























            up vote
            2
            down vote










            up vote
            2
            down vote









            Use awk:



            awk 'NR==FNRa[$1];next ($1 in a)' table_1 table_2





            share|improve this answer














            Use awk:



            awk 'NR==FNRa[$1];next ($1 in a)' table_1 table_2






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Feb 1 at 8:50

























            answered Feb 1 at 7:00









            αғsнιη

            15k82462




            15k82462




















                up vote
                1
                down vote













                Here’s another way to use join. 
                If the files are sorted, and table_2 really has only three columns,
                you can use



                join -o "2.1 2.2 2.3" table_1 table_2 > table_3


                As Kusalananda says,
                join is a program that exists specifically to combine two files
                by matching values from one column of the first file
                against values from one column of the second file. 
                By default, if uses the first column of each file (you can override this). 
                By default, it combines the matching lines, like this:



                $ join table_1 table_2
                A D G n m
                B E H m n
                C F I n m


                The -o "2.1 2.2 2.3" says “output the first field of the second file,
                and then the second field of the second file,
                and then the first third of the second file
                (but nothing from the first file1)”. 
                Unfortunately, there don’t seem to be any shortcuts, shorthands,
                accelerators, or wildcards here;
                there’s no way to say “output the entire line from the second file”. 
                If the second file has many fields,
                then the -o format will have to be very long.



                This differs from the grep-based solutions (yours and mine)
                in that it specifically matches the values from column 1 of table_1
                against the values from column 1 of table_2,
                and will not give you lines from table_2
                that have an A in column 2 or column 3, or an AZ in column 1. 
                join, like grep, recognizes a -i (--ignore-case) option,
                so, if the first line of table_2 is a n m (with a lower-case a),
                you will get that line in your table_3 output
                if (and only if) you specify -i.

                ________
                1 Except for the first field of the first file —
                which is the same as first field of the second file.






                share|improve this answer


























                  up vote
                  1
                  down vote













                  Here’s another way to use join. 
                  If the files are sorted, and table_2 really has only three columns,
                  you can use



                  join -o "2.1 2.2 2.3" table_1 table_2 > table_3


                  As Kusalananda says,
                  join is a program that exists specifically to combine two files
                  by matching values from one column of the first file
                  against values from one column of the second file. 
                  By default, if uses the first column of each file (you can override this). 
                  By default, it combines the matching lines, like this:



                  $ join table_1 table_2
                  A D G n m
                  B E H m n
                  C F I n m


                  The -o "2.1 2.2 2.3" says “output the first field of the second file,
                  and then the second field of the second file,
                  and then the first third of the second file
                  (but nothing from the first file1)”. 
                  Unfortunately, there don’t seem to be any shortcuts, shorthands,
                  accelerators, or wildcards here;
                  there’s no way to say “output the entire line from the second file”. 
                  If the second file has many fields,
                  then the -o format will have to be very long.



                  This differs from the grep-based solutions (yours and mine)
                  in that it specifically matches the values from column 1 of table_1
                  against the values from column 1 of table_2,
                  and will not give you lines from table_2
                  that have an A in column 2 or column 3, or an AZ in column 1. 
                  join, like grep, recognizes a -i (--ignore-case) option,
                  so, if the first line of table_2 is a n m (with a lower-case a),
                  you will get that line in your table_3 output
                  if (and only if) you specify -i.

                  ________
                  1 Except for the first field of the first file —
                  which is the same as first field of the second file.






                  share|improve this answer
























                    up vote
                    1
                    down vote










                    up vote
                    1
                    down vote









                    Here’s another way to use join. 
                    If the files are sorted, and table_2 really has only three columns,
                    you can use



                    join -o "2.1 2.2 2.3" table_1 table_2 > table_3


                    As Kusalananda says,
                    join is a program that exists specifically to combine two files
                    by matching values from one column of the first file
                    against values from one column of the second file. 
                    By default, if uses the first column of each file (you can override this). 
                    By default, it combines the matching lines, like this:



                    $ join table_1 table_2
                    A D G n m
                    B E H m n
                    C F I n m


                    The -o "2.1 2.2 2.3" says “output the first field of the second file,
                    and then the second field of the second file,
                    and then the first third of the second file
                    (but nothing from the first file1)”. 
                    Unfortunately, there don’t seem to be any shortcuts, shorthands,
                    accelerators, or wildcards here;
                    there’s no way to say “output the entire line from the second file”. 
                    If the second file has many fields,
                    then the -o format will have to be very long.



                    This differs from the grep-based solutions (yours and mine)
                    in that it specifically matches the values from column 1 of table_1
                    against the values from column 1 of table_2,
                    and will not give you lines from table_2
                    that have an A in column 2 or column 3, or an AZ in column 1. 
                    join, like grep, recognizes a -i (--ignore-case) option,
                    so, if the first line of table_2 is a n m (with a lower-case a),
                    you will get that line in your table_3 output
                    if (and only if) you specify -i.

                    ________
                    1 Except for the first field of the first file —
                    which is the same as first field of the second file.






                    share|improve this answer














                    Here’s another way to use join. 
                    If the files are sorted, and table_2 really has only three columns,
                    you can use



                    join -o "2.1 2.2 2.3" table_1 table_2 > table_3


                    As Kusalananda says,
                    join is a program that exists specifically to combine two files
                    by matching values from one column of the first file
                    against values from one column of the second file. 
                    By default, if uses the first column of each file (you can override this). 
                    By default, it combines the matching lines, like this:



                    $ join table_1 table_2
                    A D G n m
                    B E H m n
                    C F I n m


                    The -o "2.1 2.2 2.3" says “output the first field of the second file,
                    and then the second field of the second file,
                    and then the first third of the second file
                    (but nothing from the first file1)”. 
                    Unfortunately, there don’t seem to be any shortcuts, shorthands,
                    accelerators, or wildcards here;
                    there’s no way to say “output the entire line from the second file”. 
                    If the second file has many fields,
                    then the -o format will have to be very long.



                    This differs from the grep-based solutions (yours and mine)
                    in that it specifically matches the values from column 1 of table_1
                    against the values from column 1 of table_2,
                    and will not give you lines from table_2
                    that have an A in column 2 or column 3, or an AZ in column 1. 
                    join, like grep, recognizes a -i (--ignore-case) option,
                    so, if the first line of table_2 is a n m (with a lower-case a),
                    you will get that line in your table_3 output
                    if (and only if) you specify -i.

                    ________
                    1 Except for the first field of the first file —
                    which is the same as first field of the second file.







                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Feb 1 at 17:28

























                    answered Feb 1 at 9:37









                    G-Man

                    11.5k82657




                    11.5k82657




















                        up vote
                        0
                        down vote













                        Assuming that both tables are sorted on the first column, and using a shell like bash or ksh93 that understands process substitution with <(...):



                        $ join <( awk ' print $1 ' table_1 ) table_2
                        A n m
                        B m n
                        C n m


                        If the tables are not sorted, then we need to sort them:



                        $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 )
                        A n m
                        B m n
                        C n m


                        The join utility performs a relational INNER JOIN operation on the first column (by default) of the two files that you give it.



                        Redirect the output to a new file to create table_3:



                        $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 ) >table_3





                        share|improve this answer
























                          up vote
                          0
                          down vote













                          Assuming that both tables are sorted on the first column, and using a shell like bash or ksh93 that understands process substitution with <(...):



                          $ join <( awk ' print $1 ' table_1 ) table_2
                          A n m
                          B m n
                          C n m


                          If the tables are not sorted, then we need to sort them:



                          $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 )
                          A n m
                          B m n
                          C n m


                          The join utility performs a relational INNER JOIN operation on the first column (by default) of the two files that you give it.



                          Redirect the output to a new file to create table_3:



                          $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 ) >table_3





                          share|improve this answer






















                            up vote
                            0
                            down vote










                            up vote
                            0
                            down vote









                            Assuming that both tables are sorted on the first column, and using a shell like bash or ksh93 that understands process substitution with <(...):



                            $ join <( awk ' print $1 ' table_1 ) table_2
                            A n m
                            B m n
                            C n m


                            If the tables are not sorted, then we need to sort them:



                            $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 )
                            A n m
                            B m n
                            C n m


                            The join utility performs a relational INNER JOIN operation on the first column (by default) of the two files that you give it.



                            Redirect the output to a new file to create table_3:



                            $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 ) >table_3





                            share|improve this answer












                            Assuming that both tables are sorted on the first column, and using a shell like bash or ksh93 that understands process substitution with <(...):



                            $ join <( awk ' print $1 ' table_1 ) table_2
                            A n m
                            B m n
                            C n m


                            If the tables are not sorted, then we need to sort them:



                            $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 )
                            A n m
                            B m n
                            C n m


                            The join utility performs a relational INNER JOIN operation on the first column (by default) of the two files that you give it.



                            Redirect the output to a new file to create table_3:



                            $ join <( awk ' print $1 ' table_1 | sort ) <( sort table_2 ) >table_3






                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Feb 1 at 9:02









                            Kusalananda

                            103k13202318




                            103k13202318






















                                 

                                draft saved


                                draft discarded


























                                 


                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f421143%2fhow-to-pipe-an-output-of-a-list-as-the-input-of-grep-in-linux%23new-answer', 'question_page');

                                );

                                Post as a guest













































































                                Popular posts from this blog

                                How to check contact read email or not when send email to Individual?

                                Bahrain

                                Postfix configuration issue with fips on centos 7; mailgun relay