remove duplicate and replace it with empty space

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












0















I have a table as of below



fruits shopname 
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3


I want to group all rows based on column 1 and replace the duplicates with empty space.



It will look like below.



fruits shopname 
Apple x1
x3
banana x2
x3
orange x1
x2


I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.










share|improve this question
























  • Hello Anthony. This looks like coursework, so what have you tried so far?

    – roaima
    Feb 11 at 14:32












  • You can't group by column 1 - it contains only unique values 1 to 6.

    – roaima
    Feb 11 at 14:33







  • 1





    According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

    – Bodo
    Feb 11 at 14:34
















0















I have a table as of below



fruits shopname 
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3


I want to group all rows based on column 1 and replace the duplicates with empty space.



It will look like below.



fruits shopname 
Apple x1
x3
banana x2
x3
orange x1
x2


I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.










share|improve this question
























  • Hello Anthony. This looks like coursework, so what have you tried so far?

    – roaima
    Feb 11 at 14:32












  • You can't group by column 1 - it contains only unique values 1 to 6.

    – roaima
    Feb 11 at 14:33







  • 1





    According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

    – Bodo
    Feb 11 at 14:34














0












0








0








I have a table as of below



fruits shopname 
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3


I want to group all rows based on column 1 and replace the duplicates with empty space.



It will look like below.



fruits shopname 
Apple x1
x3
banana x2
x3
orange x1
x2


I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.










share|improve this question
















I have a table as of below



fruits shopname 
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3


I want to group all rows based on column 1 and replace the duplicates with empty space.



It will look like below.



fruits shopname 
Apple x1
x3
banana x2
x3
orange x1
x2


I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.







awk sed uniq






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Feb 11 at 15:52









glenn jackman

52.1k572112




52.1k572112










asked Feb 11 at 14:25









AnthonyAnthony

1




1












  • Hello Anthony. This looks like coursework, so what have you tried so far?

    – roaima
    Feb 11 at 14:32












  • You can't group by column 1 - it contains only unique values 1 to 6.

    – roaima
    Feb 11 at 14:33







  • 1





    According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

    – Bodo
    Feb 11 at 14:34


















  • Hello Anthony. This looks like coursework, so what have you tried so far?

    – roaima
    Feb 11 at 14:32












  • You can't group by column 1 - it contains only unique values 1 to 6.

    – roaima
    Feb 11 at 14:33







  • 1





    According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

    – Bodo
    Feb 11 at 14:34

















Hello Anthony. This looks like coursework, so what have you tried so far?

– roaima
Feb 11 at 14:32






Hello Anthony. This looks like coursework, so what have you tried so far?

– roaima
Feb 11 at 14:32














You can't group by column 1 - it contains only unique values 1 to 6.

– roaima
Feb 11 at 14:33






You can't group by column 1 - it contains only unique values 1 to 6.

– roaima
Feb 11 at 14:33





1




1





According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

– Bodo
Feb 11 at 14:34






According to your text column fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.

– Bodo
Feb 11 at 14:34











4 Answers
4






active

oldest

votes


















0














You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.



Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.



I'd use perl for this, but the syntax for perl can be somewhat overwhelming.



perl -lane '
if ($. == 1) print; next
push @$shops$F[0], $F[1];
}END sortEND 











0














You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.



Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.



I'd use perl for this, but the syntax for perl can be somewhat overwhelming.



perl -lane '
if ($. == 1) print; next
push @$shops$F[0], $F[1];
ENDimprove this answer


































    You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.



    Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.



    I'd use perl for this, but the syntax for perl can be somewhat overwhelming.



    perl -lane '
    if ($. == 1) print; next
    push @$shops$F[0], $F[1];
    END{
    for $fruit (sort lc $a cmp lc $b keys %shops)
    $label = $fruit;
    for $shop (@$shops$fruit)
    printf "%st%sn", $label, $shop;
    $label = "";


    ' file






    share|improve this answer














    share|improve this answer



    share|improve this answer








    answered Feb 11 at 16:03


























    community wiki





    glenn jackman
























        0














        I have used below method to achieve the same



        for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '


        output



        fruits shopname 
        Apple x1
        x3
        banana x2
        x3
        orange x1
        x2





        share|improve this answer



























          0














          I have used below method to achieve the same



          for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '


          output



          fruits shopname 
          Apple x1
          x3
          banana x2
          x3
          orange x1
          x2





          share|improve this answer

























            0












            0








            0







            I have used below method to achieve the same



            for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '


            output



            fruits shopname 
            Apple x1
            x3
            banana x2
            x3
            orange x1
            x2





            share|improve this answer













            I have used below method to achieve the same



            for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '


            output



            fruits shopname 
            Apple x1
            x3
            banana x2
            x3
            orange x1
            x2






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Feb 11 at 16:44









            Praveen Kumar BSPraveen Kumar BS

            1,5281310




            1,5281310





















                0














                Try:



                sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
                Apple x1
                x3
                banana x2
                x3
                orange x1
                x2


                I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.



                sort -t $'t' -uk1,1 <(tail -n+2 infile)
                Apple x1
                banana x2
                orange x1





                share|improve this answer





























                  0














                  Try:



                  sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
                  Apple x1
                  x3
                  banana x2
                  x3
                  orange x1
                  x2


                  I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.



                  sort -t $'t' -uk1,1 <(tail -n+2 infile)
                  Apple x1
                  banana x2
                  orange x1





                  share|improve this answer



























                    0












                    0








                    0







                    Try:



                    sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
                    Apple x1
                    x3
                    banana x2
                    x3
                    orange x1
                    x2


                    I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.



                    sort -t $'t' -uk1,1 <(tail -n+2 infile)
                    Apple x1
                    banana x2
                    orange x1





                    share|improve this answer















                    Try:



                    sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
                    Apple x1
                    x3
                    banana x2
                    x3
                    orange x1
                    x2


                    I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.



                    sort -t $'t' -uk1,1 <(tail -n+2 infile)
                    Apple x1
                    banana x2
                    orange x1






                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Feb 11 at 17:00

























                    answered Feb 11 at 16:05









                    αғsнιηαғsнιη

                    17k102966




                    17k102966





















                        0














                        Another version using sed, but the first that creates an input file.



                        Make sure you disable your bash history expansion with set +H before running this



                        Code: (copy & paste into your shell)



                        # replace comma with tab to enable copy&paste from stackexchange,
                        # sort the table, write the file
                        cat <<EOF | tr ";" "t" |sort > fruits.txt
                        Apple;x1
                        orange;x1
                        banana;x2
                        Apple;x3
                        orange;x2
                        banana;x3
                        EOF

                        echo "BEFORE:"
                        cat fruits.txt

                        for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done

                        echo "RESULT:"
                        cat fruits.txt


                        Output:



                        BEFORE:
                        Apple x1
                        Apple x3
                        banana x2
                        banana x3
                        orange x1
                        orange x2
                        RESULT:
                        Apple x1
                        x3
                        banana x2
                        x3
                        orange x1
                        x2





                        share|improve this answer



























                          0














                          Another version using sed, but the first that creates an input file.



                          Make sure you disable your bash history expansion with set +H before running this



                          Code: (copy & paste into your shell)



                          # replace comma with tab to enable copy&paste from stackexchange,
                          # sort the table, write the file
                          cat <<EOF | tr ";" "t" |sort > fruits.txt
                          Apple;x1
                          orange;x1
                          banana;x2
                          Apple;x3
                          orange;x2
                          banana;x3
                          EOF

                          echo "BEFORE:"
                          cat fruits.txt

                          for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done

                          echo "RESULT:"
                          cat fruits.txt


                          Output:



                          BEFORE:
                          Apple x1
                          Apple x3
                          banana x2
                          banana x3
                          orange x1
                          orange x2
                          RESULT:
                          Apple x1
                          x3
                          banana x2
                          x3
                          orange x1
                          x2





                          share|improve this answer

























                            0












                            0








                            0







                            Another version using sed, but the first that creates an input file.



                            Make sure you disable your bash history expansion with set +H before running this



                            Code: (copy & paste into your shell)



                            # replace comma with tab to enable copy&paste from stackexchange,
                            # sort the table, write the file
                            cat <<EOF | tr ";" "t" |sort > fruits.txt
                            Apple;x1
                            orange;x1
                            banana;x2
                            Apple;x3
                            orange;x2
                            banana;x3
                            EOF

                            echo "BEFORE:"
                            cat fruits.txt

                            for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done

                            echo "RESULT:"
                            cat fruits.txt


                            Output:



                            BEFORE:
                            Apple x1
                            Apple x3
                            banana x2
                            banana x3
                            orange x1
                            orange x2
                            RESULT:
                            Apple x1
                            x3
                            banana x2
                            x3
                            orange x1
                            x2





                            share|improve this answer













                            Another version using sed, but the first that creates an input file.



                            Make sure you disable your bash history expansion with set +H before running this



                            Code: (copy & paste into your shell)



                            # replace comma with tab to enable copy&paste from stackexchange,
                            # sort the table, write the file
                            cat <<EOF | tr ";" "t" |sort > fruits.txt
                            Apple;x1
                            orange;x1
                            banana;x2
                            Apple;x3
                            orange;x2
                            banana;x3
                            EOF

                            echo "BEFORE:"
                            cat fruits.txt

                            for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done

                            echo "RESULT:"
                            cat fruits.txt


                            Output:



                            BEFORE:
                            Apple x1
                            Apple x3
                            banana x2
                            banana x3
                            orange x1
                            orange x2
                            RESULT:
                            Apple x1
                            x3
                            banana x2
                            x3
                            orange x1
                            x2






                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Feb 11 at 18:56









                            FreddyFreddy

                            1,2649




                            1,2649



























                                draft saved

                                draft discarded
















































                                Thanks for contributing an answer to Unix & Linux Stack Exchange!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid


                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.

                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f499951%2fremove-duplicate-and-replace-it-with-empty-space%23new-answer', 'question_page');

                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown






                                Popular posts from this blog

                                Peggy Mitchell

                                Palaiologos

                                The Forum (Inglewood, California)