Can sed replace new line characters?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












30














Is there an issue with sed and new line character?

I have a file test.txt with the following contents



aaaaa 
bbbbb
ccccc
ddddd


The following does not work:
sed -r -i 's/n/,/g' test.txt



I know that I can use tr for this but my question is why it seems not possible with sed.



If this is a side effect of processing the file line by line I would be interested in why this happens. I think grep removes new lines. Does sed do the same?










share|improve this question























  • In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
    – omoser
    Feb 26 '15 at 9:51










  • tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
    – Stéphane Chazelas
    Jan 10 '17 at 14:09















30














Is there an issue with sed and new line character?

I have a file test.txt with the following contents



aaaaa 
bbbbb
ccccc
ddddd


The following does not work:
sed -r -i 's/n/,/g' test.txt



I know that I can use tr for this but my question is why it seems not possible with sed.



If this is a side effect of processing the file line by line I would be interested in why this happens. I think grep removes new lines. Does sed do the same?










share|improve this question























  • In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
    – omoser
    Feb 26 '15 at 9:51










  • tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
    – Stéphane Chazelas
    Jan 10 '17 at 14:09













30












30








30


6





Is there an issue with sed and new line character?

I have a file test.txt with the following contents



aaaaa 
bbbbb
ccccc
ddddd


The following does not work:
sed -r -i 's/n/,/g' test.txt



I know that I can use tr for this but my question is why it seems not possible with sed.



If this is a side effect of processing the file line by line I would be interested in why this happens. I think grep removes new lines. Does sed do the same?










share|improve this question















Is there an issue with sed and new line character?

I have a file test.txt with the following contents



aaaaa 
bbbbb
ccccc
ddddd


The following does not work:
sed -r -i 's/n/,/g' test.txt



I know that I can use tr for this but my question is why it seems not possible with sed.



If this is a side effect of processing the file line by line I would be interested in why this happens. I think grep removes new lines. Does sed do the same?







sed






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 15 '17 at 22:51









Braiam

23.1k1975137




23.1k1975137










asked Feb 12 '14 at 20:08









Jim

2,861123458




2,861123458











  • In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
    – omoser
    Feb 26 '15 at 9:51










  • tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
    – Stéphane Chazelas
    Jan 10 '17 at 14:09
















  • In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
    – omoser
    Feb 26 '15 at 9:51










  • tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
    – Stéphane Chazelas
    Jan 10 '17 at 14:09















In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
– omoser
Feb 26 '15 at 9:51




In this case sed might be not the best tool to use (eg. "tr"). There are tools that are more intuitive, easier to read/maintain, performing better (especially on big data) etc. ... Don't use your hammer to put the screws in (even if it works). You can find a comparison on: http://slash4.de/blog/python/sed-replace-newline-or-python-awk-tr-perl-xargs.html
– omoser
Feb 26 '15 at 9:51












tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
– Stéphane Chazelas
Jan 10 '17 at 14:09




tr would add a trailing , and would output an unterminated line. Best is to use paste instead: paste -sd , test.txt
– Stéphane Chazelas
Jan 10 '17 at 14:09










7 Answers
7






active

oldest

votes


















38














With GNU sed and provided POSIXLY_CORRECT is not in the environment (for single-line input):



sed -i ':a;N;$!ba;s/n/,/g' test.txt


From https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :



  1. create a label via :a

  2. append the current and next line to the pattern space via N

  3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).

  4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).





share|improve this answer






















  • This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
    – Jim
    Feb 12 '14 at 20:27







  • 1




    @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
    – Anthon
    Feb 12 '14 at 20:30











  • This results in ba: Event not found
    – krb686
    May 21 '15 at 14:07










  • @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
    – Anthon
    May 21 '15 at 14:39










  • @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
    – krb686
    May 21 '15 at 17:58



















6














From Oracle's web site:




The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.




Basically this means that because sed is reading line by line the newline character is not matched.



The solution from https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:



sed ':a;N;$!ba;s/n/ /g'


An explanation into how that works is provided on that page.






share|improve this answer


















  • 1




    That will get you spaces between the lines, not commas
    – Anthon
    Feb 12 '14 at 20:33










  • I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
    – user208145
    Sep 25 '15 at 1:51










  • Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
    – Stéphane Chazelas
    Feb 12 '16 at 15:16


















5














This works:



sed -z 's/n/,/g' 


-z is included since 4.2.2






share|improve this answer


















  • 1




    This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
    – don_crissti
    Aug 8 '17 at 17:06


















3





+50









sed always removes the trailing newline just before populating pattern space, and then appends one before writing out the results of its script. A newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - newlines in sed's pattern space always reflect a change, and never occur in the input stream. newlines are the only delimiter a sedder can count on with unknown input.



If you want to replace all newlines with commas and your file is not very large, then you can do:



sed 'H;1h;$!d;x;y/n/,/'


That appends every input line to hold space - except the first, which instead overwrites hold space - following a newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all newline characters are y///translated to commas.



For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.






share|improve this answer




























    0














    Alternatively, you can use a slightly simpler syntax:



    sed ':a;N;s/n/,/g;ba'


    ...just changing sequence order.






    share|improve this answer


















    • 3




      But runs the s command for each input line on a pattern space that is increasingly big.
      – Stéphane Chazelas
      Feb 12 '16 at 15:08


















    0














    There's some very nice sed magic here. And some good points raised about pattern space overflow. I love to use sed even when it's not the simplest way, because it's so compact and powerful. However it has it's limitations, and for large amounts of data the pattern space would have to be mahoosive.



    GNU says this:



    For those who want to write portable sed scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The posix standard specifies that conforming sed implementations shall support at least 8192 byte line lengths. GNU sed has no built-in limit on line length; as long as it can malloc() more (virtual) memory, you can feed or construct lines as long as you like.


    However, recursion is used to handle subpatterns and indefinite repetition. This means that the available stack space may limit the size of the buffer that can be processed by certain patterns.


    I don't have much to add, but I would like to point you towards my go-to guide for sed. It's excellent.
    http://www.grymoire.com/Unix/Sed.html



    and here is my solution:



    for i in $(cat test.txt); do echo -n $i','; done; echo '' >> somewhere


    well it works






    share|improve this answer
















    • 3




      You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
      – Stéphane Chazelas
      Feb 12 '16 at 15:18


















    -1














    Let's say you want to replace newlines by n. I wanted to do that, so here's what I did:



    (echo foo; echo bar; echo baz) | sed -r '$!s/$/\n/' | tr -d 'n' 
    # Output: foonbarnbaz


    Here's what it does: for all lines except the last, append n. Then, delete newlines with tr.






    share|improve this answer




















      Your Answer








      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f114943%2fcan-sed-replace-new-line-characters%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      7 Answers
      7






      active

      oldest

      votes








      7 Answers
      7






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      38














      With GNU sed and provided POSIXLY_CORRECT is not in the environment (for single-line input):



      sed -i ':a;N;$!ba;s/n/,/g' test.txt


      From https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :



      1. create a label via :a

      2. append the current and next line to the pattern space via N

      3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).

      4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).





      share|improve this answer






















      • This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
        – Jim
        Feb 12 '14 at 20:27







      • 1




        @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
        – Anthon
        Feb 12 '14 at 20:30











      • This results in ba: Event not found
        – krb686
        May 21 '15 at 14:07










      • @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
        – Anthon
        May 21 '15 at 14:39










      • @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
        – krb686
        May 21 '15 at 17:58
















      38














      With GNU sed and provided POSIXLY_CORRECT is not in the environment (for single-line input):



      sed -i ':a;N;$!ba;s/n/,/g' test.txt


      From https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :



      1. create a label via :a

      2. append the current and next line to the pattern space via N

      3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).

      4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).





      share|improve this answer






















      • This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
        – Jim
        Feb 12 '14 at 20:27







      • 1




        @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
        – Anthon
        Feb 12 '14 at 20:30











      • This results in ba: Event not found
        – krb686
        May 21 '15 at 14:07










      • @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
        – Anthon
        May 21 '15 at 14:39










      • @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
        – krb686
        May 21 '15 at 17:58














      38












      38








      38






      With GNU sed and provided POSIXLY_CORRECT is not in the environment (for single-line input):



      sed -i ':a;N;$!ba;s/n/,/g' test.txt


      From https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :



      1. create a label via :a

      2. append the current and next line to the pattern space via N

      3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).

      4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).





      share|improve this answer














      With GNU sed and provided POSIXLY_CORRECT is not in the environment (for single-line input):



      sed -i ':a;N;$!ba;s/n/,/g' test.txt


      From https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :



      1. create a label via :a

      2. append the current and next line to the pattern space via N

      3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).

      4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).






      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited May 23 '17 at 11:33









      Community

      1




      1










      answered Feb 12 '14 at 20:26









      Anthon

      60.2k17102163




      60.2k17102163











      • This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
        – Jim
        Feb 12 '14 at 20:27







      • 1




        @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
        – Anthon
        Feb 12 '14 at 20:30











      • This results in ba: Event not found
        – krb686
        May 21 '15 at 14:07










      • @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
        – Anthon
        May 21 '15 at 14:39










      • @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
        – krb686
        May 21 '15 at 17:58

















      • This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
        – Jim
        Feb 12 '14 at 20:27







      • 1




        @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
        – Anthon
        Feb 12 '14 at 20:30











      • This results in ba: Event not found
        – krb686
        May 21 '15 at 14:07










      • @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
        – Anthon
        May 21 '15 at 14:39










      • @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
        – krb686
        May 21 '15 at 17:58
















      This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
      – Jim
      Feb 12 '14 at 20:27





      This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a ,
      – Jim
      Feb 12 '14 at 20:27





      1




      1




      @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
      – Anthon
      Feb 12 '14 at 20:30





      @jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer.
      – Anthon
      Feb 12 '14 at 20:30













      This results in ba: Event not found
      – krb686
      May 21 '15 at 14:07




      This results in ba: Event not found
      – krb686
      May 21 '15 at 14:07












      @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
      – Anthon
      May 21 '15 at 14:39




      @krb686 What is the "This" you are referring to? Did you run the above sed command with those exact options? On what test.txt file? With which version of sed (try sed --version)?
      – Anthon
      May 21 '15 at 14:39












      @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
      – krb686
      May 21 '15 at 17:58





      @Anthon Sorry, I think I meant to say "the". I read another SO post that informed me that csh requires me to escape the !. Interestingly, that still did not work for me and I ended up having to double escape the ! in my .csh script. So I don't really have a problem at the moment, but you do you know why that might be? What worked for me was sed :a;N;$\!ba;s/n/ /g'
      – krb686
      May 21 '15 at 17:58














      6














      From Oracle's web site:




      The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.




      Basically this means that because sed is reading line by line the newline character is not matched.



      The solution from https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:



      sed ':a;N;$!ba;s/n/ /g'


      An explanation into how that works is provided on that page.






      share|improve this answer


















      • 1




        That will get you spaces between the lines, not commas
        – Anthon
        Feb 12 '14 at 20:33










      • I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
        – user208145
        Sep 25 '15 at 1:51










      • Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
        – Stéphane Chazelas
        Feb 12 '16 at 15:16















      6














      From Oracle's web site:




      The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.




      Basically this means that because sed is reading line by line the newline character is not matched.



      The solution from https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:



      sed ':a;N;$!ba;s/n/ /g'


      An explanation into how that works is provided on that page.






      share|improve this answer


















      • 1




        That will get you spaces between the lines, not commas
        – Anthon
        Feb 12 '14 at 20:33










      • I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
        – user208145
        Sep 25 '15 at 1:51










      • Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
        – Stéphane Chazelas
        Feb 12 '16 at 15:16













      6












      6








      6






      From Oracle's web site:




      The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.




      Basically this means that because sed is reading line by line the newline character is not matched.



      The solution from https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:



      sed ':a;N;$!ba;s/n/ /g'


      An explanation into how that works is provided on that page.






      share|improve this answer














      From Oracle's web site:




      The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.




      Basically this means that because sed is reading line by line the newline character is not matched.



      The solution from https://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:



      sed ':a;N;$!ba;s/n/ /g'


      An explanation into how that works is provided on that page.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited May 23 '17 at 12:39









      Community

      1




      1










      answered Feb 12 '14 at 20:30









      user204992

      611




      611







      • 1




        That will get you spaces between the lines, not commas
        – Anthon
        Feb 12 '14 at 20:33










      • I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
        – user208145
        Sep 25 '15 at 1:51










      • Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
        – Stéphane Chazelas
        Feb 12 '16 at 15:16












      • 1




        That will get you spaces between the lines, not commas
        – Anthon
        Feb 12 '14 at 20:33










      • I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
        – user208145
        Sep 25 '15 at 1:51










      • Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
        – Stéphane Chazelas
        Feb 12 '16 at 15:16







      1




      1




      That will get you spaces between the lines, not commas
      – Anthon
      Feb 12 '14 at 20:33




      That will get you spaces between the lines, not commas
      – Anthon
      Feb 12 '14 at 20:33












      I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
      – user208145
      Sep 25 '15 at 1:51




      I used a modified form of this to parse VPN logs and put the user "authenticated" and time stamp information on the same line. Cheers!
      – user208145
      Sep 25 '15 at 1:51












      Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
      – Stéphane Chazelas
      Feb 12 '16 at 15:16




      Note that that syntax is GNU specific, and even with GNU sed, if POSIXLY_CORRECT is in the environment and the input has only one line, there will be no output.
      – Stéphane Chazelas
      Feb 12 '16 at 15:16











      5














      This works:



      sed -z 's/n/,/g' 


      -z is included since 4.2.2






      share|improve this answer


















      • 1




        This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
        – don_crissti
        Aug 8 '17 at 17:06















      5














      This works:



      sed -z 's/n/,/g' 


      -z is included since 4.2.2






      share|improve this answer


















      • 1




        This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
        – don_crissti
        Aug 8 '17 at 17:06













      5












      5








      5






      This works:



      sed -z 's/n/,/g' 


      -z is included since 4.2.2






      share|improve this answer














      This works:



      sed -z 's/n/,/g' 


      -z is included since 4.2.2







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited May 2 at 16:24









      StackzOfZtuff

      15916




      15916










      answered Aug 8 '17 at 16:59









      Hielke Walinga

      18015




      18015







      • 1




        This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
        – don_crissti
        Aug 8 '17 at 17:06












      • 1




        This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
        – don_crissti
        Aug 8 '17 at 17:06







      1




      1




      This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
      – don_crissti
      Aug 8 '17 at 17:06




      This will also replace the trailing newline which might not be what OP wants... compare the result with mikeserv's solution.
      – don_crissti
      Aug 8 '17 at 17:06











      3





      +50









      sed always removes the trailing newline just before populating pattern space, and then appends one before writing out the results of its script. A newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - newlines in sed's pattern space always reflect a change, and never occur in the input stream. newlines are the only delimiter a sedder can count on with unknown input.



      If you want to replace all newlines with commas and your file is not very large, then you can do:



      sed 'H;1h;$!d;x;y/n/,/'


      That appends every input line to hold space - except the first, which instead overwrites hold space - following a newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all newline characters are y///translated to commas.



      For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.






      share|improve this answer

























        3





        +50









        sed always removes the trailing newline just before populating pattern space, and then appends one before writing out the results of its script. A newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - newlines in sed's pattern space always reflect a change, and never occur in the input stream. newlines are the only delimiter a sedder can count on with unknown input.



        If you want to replace all newlines with commas and your file is not very large, then you can do:



        sed 'H;1h;$!d;x;y/n/,/'


        That appends every input line to hold space - except the first, which instead overwrites hold space - following a newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all newline characters are y///translated to commas.



        For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.






        share|improve this answer























          3





          +50







          3





          +50



          3




          +50




          sed always removes the trailing newline just before populating pattern space, and then appends one before writing out the results of its script. A newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - newlines in sed's pattern space always reflect a change, and never occur in the input stream. newlines are the only delimiter a sedder can count on with unknown input.



          If you want to replace all newlines with commas and your file is not very large, then you can do:



          sed 'H;1h;$!d;x;y/n/,/'


          That appends every input line to hold space - except the first, which instead overwrites hold space - following a newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all newline characters are y///translated to commas.



          For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.






          share|improve this answer












          sed always removes the trailing newline just before populating pattern space, and then appends one before writing out the results of its script. A newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - newlines in sed's pattern space always reflect a change, and never occur in the input stream. newlines are the only delimiter a sedder can count on with unknown input.



          If you want to replace all newlines with commas and your file is not very large, then you can do:



          sed 'H;1h;$!d;x;y/n/,/'


          That appends every input line to hold space - except the first, which instead overwrites hold space - following a newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all newline characters are y///translated to commas.



          For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 24 '14 at 0:14









          mikeserv

          45.3k567153




          45.3k567153





















              0














              Alternatively, you can use a slightly simpler syntax:



              sed ':a;N;s/n/,/g;ba'


              ...just changing sequence order.






              share|improve this answer


















              • 3




                But runs the s command for each input line on a pattern space that is increasingly big.
                – Stéphane Chazelas
                Feb 12 '16 at 15:08















              0














              Alternatively, you can use a slightly simpler syntax:



              sed ':a;N;s/n/,/g;ba'


              ...just changing sequence order.






              share|improve this answer


















              • 3




                But runs the s command for each input line on a pattern space that is increasingly big.
                – Stéphane Chazelas
                Feb 12 '16 at 15:08













              0












              0








              0






              Alternatively, you can use a slightly simpler syntax:



              sed ':a;N;s/n/,/g;ba'


              ...just changing sequence order.






              share|improve this answer














              Alternatively, you can use a slightly simpler syntax:



              sed ':a;N;s/n/,/g;ba'


              ...just changing sequence order.







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Nov 22 '14 at 13:24









              terdon

              128k31246423




              128k31246423










              answered Nov 22 '14 at 12:57









              Rodec

              11




              11







              • 3




                But runs the s command for each input line on a pattern space that is increasingly big.
                – Stéphane Chazelas
                Feb 12 '16 at 15:08












              • 3




                But runs the s command for each input line on a pattern space that is increasingly big.
                – Stéphane Chazelas
                Feb 12 '16 at 15:08







              3




              3




              But runs the s command for each input line on a pattern space that is increasingly big.
              – Stéphane Chazelas
              Feb 12 '16 at 15:08




              But runs the s command for each input line on a pattern space that is increasingly big.
              – Stéphane Chazelas
              Feb 12 '16 at 15:08











              0














              There's some very nice sed magic here. And some good points raised about pattern space overflow. I love to use sed even when it's not the simplest way, because it's so compact and powerful. However it has it's limitations, and for large amounts of data the pattern space would have to be mahoosive.



              GNU says this:



              For those who want to write portable sed scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The posix standard specifies that conforming sed implementations shall support at least 8192 byte line lengths. GNU sed has no built-in limit on line length; as long as it can malloc() more (virtual) memory, you can feed or construct lines as long as you like.


              However, recursion is used to handle subpatterns and indefinite repetition. This means that the available stack space may limit the size of the buffer that can be processed by certain patterns.


              I don't have much to add, but I would like to point you towards my go-to guide for sed. It's excellent.
              http://www.grymoire.com/Unix/Sed.html



              and here is my solution:



              for i in $(cat test.txt); do echo -n $i','; done; echo '' >> somewhere


              well it works






              share|improve this answer
















              • 3




                You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
                – Stéphane Chazelas
                Feb 12 '16 at 15:18















              0














              There's some very nice sed magic here. And some good points raised about pattern space overflow. I love to use sed even when it's not the simplest way, because it's so compact and powerful. However it has it's limitations, and for large amounts of data the pattern space would have to be mahoosive.



              GNU says this:



              For those who want to write portable sed scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The posix standard specifies that conforming sed implementations shall support at least 8192 byte line lengths. GNU sed has no built-in limit on line length; as long as it can malloc() more (virtual) memory, you can feed or construct lines as long as you like.


              However, recursion is used to handle subpatterns and indefinite repetition. This means that the available stack space may limit the size of the buffer that can be processed by certain patterns.


              I don't have much to add, but I would like to point you towards my go-to guide for sed. It's excellent.
              http://www.grymoire.com/Unix/Sed.html



              and here is my solution:



              for i in $(cat test.txt); do echo -n $i','; done; echo '' >> somewhere


              well it works






              share|improve this answer
















              • 3




                You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
                – Stéphane Chazelas
                Feb 12 '16 at 15:18













              0












              0








              0






              There's some very nice sed magic here. And some good points raised about pattern space overflow. I love to use sed even when it's not the simplest way, because it's so compact and powerful. However it has it's limitations, and for large amounts of data the pattern space would have to be mahoosive.



              GNU says this:



              For those who want to write portable sed scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The posix standard specifies that conforming sed implementations shall support at least 8192 byte line lengths. GNU sed has no built-in limit on line length; as long as it can malloc() more (virtual) memory, you can feed or construct lines as long as you like.


              However, recursion is used to handle subpatterns and indefinite repetition. This means that the available stack space may limit the size of the buffer that can be processed by certain patterns.


              I don't have much to add, but I would like to point you towards my go-to guide for sed. It's excellent.
              http://www.grymoire.com/Unix/Sed.html



              and here is my solution:



              for i in $(cat test.txt); do echo -n $i','; done; echo '' >> somewhere


              well it works






              share|improve this answer












              There's some very nice sed magic here. And some good points raised about pattern space overflow. I love to use sed even when it's not the simplest way, because it's so compact and powerful. However it has it's limitations, and for large amounts of data the pattern space would have to be mahoosive.



              GNU says this:



              For those who want to write portable sed scripts, be aware that some implementations have been known to limit line lengths (for the pattern and hold spaces) to be no more than 4000 bytes. The posix standard specifies that conforming sed implementations shall support at least 8192 byte line lengths. GNU sed has no built-in limit on line length; as long as it can malloc() more (virtual) memory, you can feed or construct lines as long as you like.


              However, recursion is used to handle subpatterns and indefinite repetition. This means that the available stack space may limit the size of the buffer that can be processed by certain patterns.


              I don't have much to add, but I would like to point you towards my go-to guide for sed. It's excellent.
              http://www.grymoire.com/Unix/Sed.html



              and here is my solution:



              for i in $(cat test.txt); do echo -n $i','; done; echo '' >> somewhere


              well it works







              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Feb 12 '16 at 14:44









              xeuari

              1




              1







              • 3




                You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
                – Stéphane Chazelas
                Feb 12 '16 at 15:18












              • 3




                You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
                – Stéphane Chazelas
                Feb 12 '16 at 15:18







              3




              3




              You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
              – Stéphane Chazelas
              Feb 12 '16 at 15:18




              You might want to read Why is using a shell loop to process text considered bad practice? and Security implications of forgetting to quote a variable in bash/POSIX shells and maybe Why is printf better than echo?
              – Stéphane Chazelas
              Feb 12 '16 at 15:18











              -1














              Let's say you want to replace newlines by n. I wanted to do that, so here's what I did:



              (echo foo; echo bar; echo baz) | sed -r '$!s/$/\n/' | tr -d 'n' 
              # Output: foonbarnbaz


              Here's what it does: for all lines except the last, append n. Then, delete newlines with tr.






              share|improve this answer

























                -1














                Let's say you want to replace newlines by n. I wanted to do that, so here's what I did:



                (echo foo; echo bar; echo baz) | sed -r '$!s/$/\n/' | tr -d 'n' 
                # Output: foonbarnbaz


                Here's what it does: for all lines except the last, append n. Then, delete newlines with tr.






                share|improve this answer























                  -1












                  -1








                  -1






                  Let's say you want to replace newlines by n. I wanted to do that, so here's what I did:



                  (echo foo; echo bar; echo baz) | sed -r '$!s/$/\n/' | tr -d 'n' 
                  # Output: foonbarnbaz


                  Here's what it does: for all lines except the last, append n. Then, delete newlines with tr.






                  share|improve this answer












                  Let's say you want to replace newlines by n. I wanted to do that, so here's what I did:



                  (echo foo; echo bar; echo baz) | sed -r '$!s/$/\n/' | tr -d 'n' 
                  # Output: foonbarnbaz


                  Here's what it does: for all lines except the last, append n. Then, delete newlines with tr.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Feb 12 '16 at 10:32









                  Camilo Martin

                  38639




                  38639



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.





                      Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                      Please pay close attention to the following guidance:


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f114943%2fcan-sed-replace-new-line-characters%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown






                      Popular posts from this blog

                      Peggy Mitchell

                      Palaiologos

                      The Forum (Inglewood, California)