Switch columns in CSV using awk? [duplicate]

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite













This question already has an answer here:



  • #!/bin/bash - no such file or directory

    8 answers



I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.



The original file looks like this:



col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is



awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv


But I am getting the 3rd column (which used to be the first) in a new line:



col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196


Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.










share|improve this question















marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.










  • 1




    Please paste a sample from old.csv
    – Sparhawk
    Aug 7 at 23:44










  • Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
    – Nasir Riley
    Aug 8 at 0:04






  • 4




    It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
    – steeldriver
    Aug 8 at 0:29










  • @steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
    – ahmed_m
    Aug 8 at 3:04














up vote
1
down vote

favorite













This question already has an answer here:



  • #!/bin/bash - no such file or directory

    8 answers



I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.



The original file looks like this:



col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is



awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv


But I am getting the 3rd column (which used to be the first) in a new line:



col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196


Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.










share|improve this question















marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.










  • 1




    Please paste a sample from old.csv
    – Sparhawk
    Aug 7 at 23:44










  • Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
    – Nasir Riley
    Aug 8 at 0:04






  • 4




    It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
    – steeldriver
    Aug 8 at 0:29










  • @steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
    – ahmed_m
    Aug 8 at 3:04












up vote
1
down vote

favorite









up vote
1
down vote

favorite












This question already has an answer here:



  • #!/bin/bash - no such file or directory

    8 answers



I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.



The original file looks like this:



col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is



awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv


But I am getting the 3rd column (which used to be the first) in a new line:



col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196


Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.










share|improve this question
















This question already has an answer here:



  • #!/bin/bash - no such file or directory

    8 answers



I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.



The original file looks like this:



col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is



awk -F, 'print $2,$3,$1' OFS=, old.csv > new.csv


But I am getting the 3rd column (which used to be the first) in a new line:



col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196


Why is awk sending the 3rd column data to a new line?
I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.





This question already has an answer here:



  • #!/bin/bash - no such file or directory

    8 answers







text-processing awk csv columns






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Aug 8 at 1:27









slm♦

238k65491662




238k65491662










asked Aug 7 at 23:36









ahmed_m

84




84




marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.






marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 at 8:35


This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.









  • 1




    Please paste a sample from old.csv
    – Sparhawk
    Aug 7 at 23:44










  • Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
    – Nasir Riley
    Aug 8 at 0:04






  • 4




    It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
    – steeldriver
    Aug 8 at 0:29










  • @steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
    – ahmed_m
    Aug 8 at 3:04












  • 1




    Please paste a sample from old.csv
    – Sparhawk
    Aug 7 at 23:44










  • Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
    – Nasir Riley
    Aug 8 at 0:04






  • 4




    It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
    – steeldriver
    Aug 8 at 0:29










  • @steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
    – ahmed_m
    Aug 8 at 3:04







1




1




Please paste a sample from old.csv
– Sparhawk
Aug 7 at 23:44




Please paste a sample from old.csv
– Sparhawk
Aug 7 at 23:44












Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
– Nasir Riley
Aug 8 at 0:04




Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , 'print $2,$3,$1' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently?
– Nasir Riley
Aug 8 at 0:04




4




4




It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
– steeldriver
Aug 8 at 0:29




It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example)
– steeldriver
Aug 8 at 0:29












@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
– ahmed_m
Aug 8 at 3:04




@steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now.
– ahmed_m
Aug 8 at 3:04










1 Answer
1






active

oldest

votes

















up vote
2
down vote



accepted










It seems like your input file has some additional data in it, such as, DOS style newlines (rn), whereas typically on Unix systems, the files only have n.



For example:



$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


We can use hexdump to see the actual ASCII of this file:



$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031


Notice the 0a in the HEX output, this is a newline (n). If I use basically your awk with this file it works as expected:



$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196


If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:



$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034


Now we see 0d & 0a which is a rn. Using awk on this file acts oddly:



$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36





share|improve this answer




















  • You are correct. I found ^M at end of each line. Removed those and it's working fine now.
    – ahmed_m
    Aug 8 at 3:05

















1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
2
down vote



accepted










It seems like your input file has some additional data in it, such as, DOS style newlines (rn), whereas typically on Unix systems, the files only have n.



For example:



$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


We can use hexdump to see the actual ASCII of this file:



$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031


Notice the 0a in the HEX output, this is a newline (n). If I use basically your awk with this file it works as expected:



$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196


If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:



$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034


Now we see 0d & 0a which is a rn. Using awk on this file acts oddly:



$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36





share|improve this answer




















  • You are correct. I found ^M at end of each line. Removed those and it's working fine now.
    – ahmed_m
    Aug 8 at 3:05














up vote
2
down vote



accepted










It seems like your input file has some additional data in it, such as, DOS style newlines (rn), whereas typically on Unix systems, the files only have n.



For example:



$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


We can use hexdump to see the actual ASCII of this file:



$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031


Notice the 0a in the HEX output, this is a newline (n). If I use basically your awk with this file it works as expected:



$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196


If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:



$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034


Now we see 0d & 0a which is a rn. Using awk on this file acts oddly:



$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36





share|improve this answer




















  • You are correct. I found ^M at end of each line. Removed those and it's working fine now.
    – ahmed_m
    Aug 8 at 3:05












up vote
2
down vote



accepted







up vote
2
down vote



accepted






It seems like your input file has some additional data in it, such as, DOS style newlines (rn), whereas typically on Unix systems, the files only have n.



For example:



$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


We can use hexdump to see the actual ASCII of this file:



$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031


Notice the 0a in the HEX output, this is a newline (n). If I use basically your awk with this file it works as expected:



$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196


If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:



$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034


Now we see 0d & 0a which is a rn. Using awk on this file acts oddly:



$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36





share|improve this answer












It seems like your input file has some additional data in it, such as, DOS style newlines (rn), whereas typically on Unix systems, the files only have n.



For example:



$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36


We can use hexdump to see the actual ASCII of this file:



$ hexdump -C old.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0a 32 |col1,col2,col3.2|
00000010 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0a 31 |,2015-01-04,23.1|
00000020 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c 33 36 |96,2015-01-20,36|
00000030 0a |.|
00000031


Notice the 0a in the HEX output, this is a newline (n). If I use basically your awk with this file it works as expected:



$ awk -F, 'print $2,$3,$1' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196


If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:



$ hexdump -C old_dos.csv
00000000 63 6f 6c 31 2c 63 6f 6c 32 2c 63 6f 6c 33 0d 0a |col1,col2,col3..|
00000010 32 2c 32 30 31 35 2d 30 31 2d 30 34 2c 32 33 0d |2,2015-01-04,23.|
00000020 0a 31 39 36 2c 32 30 31 35 2d 30 31 2d 32 30 2c |.196,2015-01-20,|
00000030 33 36 0d 0a |36..|
00000034


Now we see 0d & 0a which is a rn. Using awk on this file acts oddly:



$ awk -F, 'print $2,$3,$1' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36






share|improve this answer












share|improve this answer



share|improve this answer










answered Aug 8 at 1:23









slm♦

238k65491662




238k65491662











  • You are correct. I found ^M at end of each line. Removed those and it's working fine now.
    – ahmed_m
    Aug 8 at 3:05
















  • You are correct. I found ^M at end of each line. Removed those and it's working fine now.
    – ahmed_m
    Aug 8 at 3:05















You are correct. I found ^M at end of each line. Removed those and it's working fine now.
– ahmed_m
Aug 8 at 3:05




You are correct. I found ^M at end of each line. Removed those and it's working fine now.
– ahmed_m
Aug 8 at 3:05


Popular posts from this blog

How to check contact read email or not when send email to Individual?

Bahrain

Postfix configuration issue with fips on centos 7; mailgun relay