remove duplicate and replace it with empty space

Clash Royale CLAN TAG#URR8PPP
I have a table as of below
fruits shopname
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3
I want to group all rows based on column 1 and replace the duplicates with empty space.
It will look like below.
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.
awk sed uniq
add a comment |
I have a table as of below
fruits shopname
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3
I want to group all rows based on column 1 and replace the duplicates with empty space.
It will look like below.
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.
awk sed uniq
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
1
According to your text columnfruitsseems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.
– Bodo
Feb 11 at 14:34
add a comment |
I have a table as of below
fruits shopname
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3
I want to group all rows based on column 1 and replace the duplicates with empty space.
It will look like below.
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.
awk sed uniq
I have a table as of below
fruits shopname
Apple x1
orange x1
banana x2
Apple x3
orange x2
banana x3
I want to group all rows based on column 1 and replace the duplicates with empty space.
It will look like below.
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
I know we can remove duplicates with uniq command.
but here I want to group them and replace duplicates with empty space.
awk sed uniq
awk sed uniq
edited Feb 11 at 15:52
glenn jackman
52.1k572112
52.1k572112
asked Feb 11 at 14:25
AnthonyAnthony
1
1
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
1
According to your text columnfruitsseems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.
– Bodo
Feb 11 at 14:34
add a comment |
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
1
According to your text columnfruitsseems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.
– Bodo
Feb 11 at 14:34
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
1
1
According to your text column
fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.– Bodo
Feb 11 at 14:34
According to your text column
fruits seems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.– Bodo
Feb 11 at 14:34
add a comment |
4 Answers
4
active
oldest
votes
You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.
Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.
I'd use perl for this, but the syntax for perl can be somewhat overwhelming.
perl -lane '
if ($. == 1) print; next
push @$shops$F[0], $F[1];
}END sortEND
You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.
Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.
I'd use perl for this, but the syntax for perl can be somewhat overwhelming.
perl -lane '
if ($. == 1) print; next
push @$shops$F[0], $F[1];
ENDimprove this answer
add a comment ENDimprove this answer
You need to read the file line by line, and build up a data structure that associates the fruit to a list of shop numbers. You can do this with awk's multidimensional arrays, or with GNU awk's array of arrays.
Then, after you have read the file, you will loop over the fruits, and for each fruit, print out a line for each shop.
I'd use perl for this, but the syntax for perl can be somewhat overwhelming.
perl -lane '
if ($. == 1) print; next
push @$shops$F[0], $F[1];
END{
for $fruit (sort lc $a cmp lc $b keys %shops)
$label = $fruit;
for $shop (@$shops$fruit)
printf "%st%sn", $label, $shop;
$label = "";
' file
answered Feb 11 at 16:03
community wiki
glenn jackman
add a comment |
add a comment |
I have used below method to achieve the same
for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '
output
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
add a comment |
I have used below method to achieve the same
for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '
output
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
add a comment |
I have used below method to achieve the same
for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '
output
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
I have used below method to achieve the same
for i in `awk 'print $1' y.txt| sort| uniq| tr "n" " "`; do awk -v i="$i" '$1 == i print $2' y.txt| sed "1s/.*/$it&/g"| sed '/^x/s/.*/t&/g';done| sed '1i fruits shopname '
output
fruits shopname
Apple x1
x3
banana x2
x3
orange x1
x2
answered Feb 11 at 16:44
Praveen Kumar BSPraveen Kumar BS
1,5281310
1,5281310
add a comment |
add a comment |
Try:
sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
Apple x1
x3
banana x2
x3
orange x1
x2
I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.
sort -t $'t' -uk1,1 <(tail -n+2 infile)
Apple x1
banana x2
orange x1
add a comment |
Try:
sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
Apple x1
x3
banana x2
x3
orange x1
x2
I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.
sort -t $'t' -uk1,1 <(tail -n+2 infile)
Apple x1
banana x2
orange x1
add a comment |
Try:
sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
Apple x1
x3
banana x2
x3
orange x1
x2
I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.
sort -t $'t' -uk1,1 <(tail -n+2 infile)
Apple x1
banana x2
orange x1
Try:
sort -t $'t' <(tail -n+2 infile) |awk 'seen[$1]++ $1="" 1' OFS='t'
Apple x1
x3
banana x2
x3
orange x1
x2
I don't understand why do you need empty fruits name, which you can simply query the data you need, everything other than result you can consider it's empty.
sort -t $'t' -uk1,1 <(tail -n+2 infile)
Apple x1
banana x2
orange x1
edited Feb 11 at 17:00
answered Feb 11 at 16:05
αғsнιηαғsнιη
17k102966
17k102966
add a comment |
add a comment |
Another version using sed, but the first that creates an input file.
Make sure you disable your bash history expansion with set +H before running this
Code: (copy & paste into your shell)
# replace comma with tab to enable copy&paste from stackexchange,
# sort the table, write the file
cat <<EOF | tr ";" "t" |sort > fruits.txt
Apple;x1
orange;x1
banana;x2
Apple;x3
orange;x2
banana;x3
EOF
echo "BEFORE:"
cat fruits.txt
for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done
echo "RESULT:"
cat fruits.txt
Output:
BEFORE:
Apple x1
Apple x3
banana x2
banana x3
orange x1
orange x2
RESULT:
Apple x1
x3
banana x2
x3
orange x1
x2
add a comment |
Another version using sed, but the first that creates an input file.
Make sure you disable your bash history expansion with set +H before running this
Code: (copy & paste into your shell)
# replace comma with tab to enable copy&paste from stackexchange,
# sort the table, write the file
cat <<EOF | tr ";" "t" |sort > fruits.txt
Apple;x1
orange;x1
banana;x2
Apple;x3
orange;x2
banana;x3
EOF
echo "BEFORE:"
cat fruits.txt
for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done
echo "RESULT:"
cat fruits.txt
Output:
BEFORE:
Apple x1
Apple x3
banana x2
banana x3
orange x1
orange x2
RESULT:
Apple x1
x3
banana x2
x3
orange x1
x2
add a comment |
Another version using sed, but the first that creates an input file.
Make sure you disable your bash history expansion with set +H before running this
Code: (copy & paste into your shell)
# replace comma with tab to enable copy&paste from stackexchange,
# sort the table, write the file
cat <<EOF | tr ";" "t" |sort > fruits.txt
Apple;x1
orange;x1
banana;x2
Apple;x3
orange;x2
banana;x3
EOF
echo "BEFORE:"
cat fruits.txt
for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done
echo "RESULT:"
cat fruits.txt
Output:
BEFORE:
Apple x1
Apple x3
banana x2
banana x3
orange x1
orange x2
RESULT:
Apple x1
x3
banana x2
x3
orange x1
x2
Another version using sed, but the first that creates an input file.
Make sure you disable your bash history expansion with set +H before running this
Code: (copy & paste into your shell)
# replace comma with tab to enable copy&paste from stackexchange,
# sort the table, write the file
cat <<EOF | tr ";" "t" |sort > fruits.txt
Apple;x1
orange;x1
banana;x2
Apple;x3
orange;x2
banana;x3
EOF
echo "BEFORE:"
cat fruits.txt
for fruit in $(cut -f1 fruits.txt|sort -u); do sed -i "/$fruit/!b;n;s/^w+//" fruits.txt; done
echo "RESULT:"
cat fruits.txt
Output:
BEFORE:
Apple x1
Apple x3
banana x2
banana x3
orange x1
orange x2
RESULT:
Apple x1
x3
banana x2
x3
orange x1
x2
answered Feb 11 at 18:56
FreddyFreddy
1,2649
1,2649
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f499951%2fremove-duplicate-and-replace-it-with-empty-space%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Hello Anthony. This looks like coursework, so what have you tried so far?
– roaima
Feb 11 at 14:32
You can't group by column 1 - it contains only unique values 1 to 6.
– roaima
Feb 11 at 14:33
1
According to your text column
fruitsseems to be the real column 1. Adding line numbers is confusing. I suggest to remove the line number column.– Bodo
Feb 11 at 14:34