Switch columns in CSV using awk? [duplicate]
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
This question already has an answer here:
#!/bin/bash - no such file or directory
8 answers
I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.
The original file looks like this:
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is
awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv
But I am getting the 3rd column (which used to be the first) in a new line:
col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196
Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.
text-processing awk csv columns
marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
add a comment |Â
up vote
1
down vote
favorite
This question already has an answer here:
#!/bin/bash - no such file or directory
8 answers
I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.
The original file looks like this:
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is
awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv
But I am getting the 3rd column (which used to be the first) in a new line:
col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196
Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.
text-processing awk csv columns
marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
Please paste a sample fromold.csv
â Sparhawk
Aug 7 at 23:44
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
â Nasir Riley
Aug 8 at 0:04
4
It might be helpful to include the output ofcat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)
â steeldriver
Aug 8 at 0:29
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
This question already has an answer here:
#!/bin/bash - no such file or directory
8 answers
I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.
The original file looks like this:
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is
awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv
But I am getting the 3rd column (which used to be the first) in a new line:
col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196
Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.
text-processing awk csv columns
This question already has an answer here:
#!/bin/bash - no such file or directory
8 answers
I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.
The original file looks like this:
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is
awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv
But I am getting the 3rd column (which used to be the first) in a new line:
col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196
Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.
This question already has an answer here:
#!/bin/bash - no such file or directory
8 answers
text-processing awk csv columns
text-processing awk csv columns
edited Aug 8 at 1:27
slmâ¦
238k65491662
238k65491662
asked Aug 7 at 23:36
ahmed_m
84
84
marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35
This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.
1
Please paste a sample fromold.csv
â Sparhawk
Aug 7 at 23:44
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
â Nasir Riley
Aug 8 at 0:04
4
It might be helpful to include the output ofcat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)
â steeldriver
Aug 8 at 0:29
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04
add a comment |Â
1
Please paste a sample fromold.csv
â Sparhawk
Aug 7 at 23:44
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
â Nasir Riley
Aug 8 at 0:04
4
It might be helpful to include the output ofcat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)
â steeldriver
Aug 8 at 0:29
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04
1
1
Please paste a sample from
old.csv
â Sparhawk
Aug 7 at 23:44
Please paste a sample from
old.csv
â Sparhawk
Aug 7 at 23:44
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,
awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?â Nasir Riley
Aug 8 at 0:04
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,
awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?â Nasir Riley
Aug 8 at 0:04
4
4
It might be helpful to include the output of
cat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)â steeldriver
Aug 8 at 0:29
It might be helpful to include the output of
cat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)â steeldriver
Aug 8 at 0:29
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
It seems like your input file has some additional data in it, such as, DOS style newlines (rn
), whereas typically on Unix systems, the files only have n
.
For example:
$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
We can use hexdump
to see the actual ASCII of this file:
$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031
Notice the 0a
in the HEX output, this is a newline (n
). If I use basically your awk
with this file it works as expected:
$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196
If we convert the old.csv
file to one that's typically form a Windows/DOS system using a CLI tool unix2dos
the modified file, old_dos.csv
looks like this:
$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034
Now we see 0d
& 0a
which is a rn
. Using awk
on this file acts oddly:
$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
It seems like your input file has some additional data in it, such as, DOS style newlines (rn
), whereas typically on Unix systems, the files only have n
.
For example:
$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
We can use hexdump
to see the actual ASCII of this file:
$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031
Notice the 0a
in the HEX output, this is a newline (n
). If I use basically your awk
with this file it works as expected:
$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196
If we convert the old.csv
file to one that's typically form a Windows/DOS system using a CLI tool unix2dos
the modified file, old_dos.csv
looks like this:
$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034
Now we see 0d
& 0a
which is a rn
. Using awk
on this file acts oddly:
$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
add a comment |Â
up vote
2
down vote
accepted
It seems like your input file has some additional data in it, such as, DOS style newlines (rn
), whereas typically on Unix systems, the files only have n
.
For example:
$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
We can use hexdump
to see the actual ASCII of this file:
$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031
Notice the 0a
in the HEX output, this is a newline (n
). If I use basically your awk
with this file it works as expected:
$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196
If we convert the old.csv
file to one that's typically form a Windows/DOS system using a CLI tool unix2dos
the modified file, old_dos.csv
looks like this:
$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034
Now we see 0d
& 0a
which is a rn
. Using awk
on this file acts oddly:
$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
add a comment |Â
up vote
2
down vote
accepted
up vote
2
down vote
accepted
It seems like your input file has some additional data in it, such as, DOS style newlines (rn
), whereas typically on Unix systems, the files only have n
.
For example:
$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
We can use hexdump
to see the actual ASCII of this file:
$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031
Notice the 0a
in the HEX output, this is a newline (n
). If I use basically your awk
with this file it works as expected:
$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196
If we convert the old.csv
file to one that's typically form a Windows/DOS system using a CLI tool unix2dos
the modified file, old_dos.csv
looks like this:
$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034
Now we see 0d
& 0a
which is a rn
. Using awk
on this file acts oddly:
$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
It seems like your input file has some additional data in it, such as, DOS style newlines (rn
), whereas typically on Unix systems, the files only have n
.
For example:
$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36
We can use hexdump
to see the actual ASCII of this file:
$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031
Notice the 0a
in the HEX output, this is a newline (n
). If I use basically your awk
with this file it works as expected:
$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196
If we convert the old.csv
file to one that's typically form a Windows/DOS system using a CLI tool unix2dos
the modified file, old_dos.csv
looks like this:
$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034
Now we see 0d
& 0a
which is a rn
. Using awk
on this file acts oddly:
$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
answered Aug 8 at 1:23
slmâ¦
238k65491662
238k65491662
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
add a comment |Â
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
You are correct. I found ^M at end of each line. Removed those and it's working fine now.
â ahmed_m
Aug 8 at 3:05
add a comment |Â
1
Please paste a sample from
old.csv
â Sparhawk
Aug 7 at 23:44
Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have,
awk -F , 'print $2,$3,$1' OFS=, old.csv
, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?â Nasir Riley
Aug 8 at 0:04
4
It might be helpful to include the output of
cat -A old.csv
so we can see any non-printing characters (including DOS-style line endings for example)â steeldriver
Aug 8 at 0:29
@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
â ahmed_m
Aug 8 at 3:04