Command substitution: splitting on newline but not space

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
27
down vote

favorite
5












I know I can solve this problem several ways, but I'm wondering if there is a way to do it using only bash built-ins, and if not, what is the most efficient way to do it.



I have a file with contents like



AAA
B C DDD
FOO BAR


by which I only mean it has several lines and each line may or may not have spaces. I want to run a command like



cmd AAA "B C DDD" "FOO BAR"


If I use cmd $(< file) I get



cmd AAA B C DDD FOO BAR


and if I use cmd "$(< file)" I get



cmd "AAA B C DDD FOO BAR"


How do I get each line treated a exactly one parameter?










share|improve this question























  • question with +922 total votes: How do I split a string on a delimiter in Bash?
    – Trevor Boyd Smith
    Jun 17 '16 at 14:55














up vote
27
down vote

favorite
5












I know I can solve this problem several ways, but I'm wondering if there is a way to do it using only bash built-ins, and if not, what is the most efficient way to do it.



I have a file with contents like



AAA
B C DDD
FOO BAR


by which I only mean it has several lines and each line may or may not have spaces. I want to run a command like



cmd AAA "B C DDD" "FOO BAR"


If I use cmd $(< file) I get



cmd AAA B C DDD FOO BAR


and if I use cmd "$(< file)" I get



cmd "AAA B C DDD FOO BAR"


How do I get each line treated a exactly one parameter?










share|improve this question























  • question with +922 total votes: How do I split a string on a delimiter in Bash?
    – Trevor Boyd Smith
    Jun 17 '16 at 14:55












up vote
27
down vote

favorite
5









up vote
27
down vote

favorite
5






5





I know I can solve this problem several ways, but I'm wondering if there is a way to do it using only bash built-ins, and if not, what is the most efficient way to do it.



I have a file with contents like



AAA
B C DDD
FOO BAR


by which I only mean it has several lines and each line may or may not have spaces. I want to run a command like



cmd AAA "B C DDD" "FOO BAR"


If I use cmd $(< file) I get



cmd AAA B C DDD FOO BAR


and if I use cmd "$(< file)" I get



cmd "AAA B C DDD FOO BAR"


How do I get each line treated a exactly one parameter?










share|improve this question















I know I can solve this problem several ways, but I'm wondering if there is a way to do it using only bash built-ins, and if not, what is the most efficient way to do it.



I have a file with contents like



AAA
B C DDD
FOO BAR


by which I only mean it has several lines and each line may or may not have spaces. I want to run a command like



cmd AAA "B C DDD" "FOO BAR"


If I use cmd $(< file) I get



cmd AAA B C DDD FOO BAR


and if I use cmd "$(< file)" I get



cmd "AAA B C DDD FOO BAR"


How do I get each line treated a exactly one parameter?







bash command-substitution






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Oct 10 '14 at 10:49









peterph

22.8k24357




22.8k24357










asked May 27 '12 at 21:22









Old Pro

5742514




5742514











  • question with +922 total votes: How do I split a string on a delimiter in Bash?
    – Trevor Boyd Smith
    Jun 17 '16 at 14:55
















  • question with +922 total votes: How do I split a string on a delimiter in Bash?
    – Trevor Boyd Smith
    Jun 17 '16 at 14:55















question with +922 total votes: How do I split a string on a delimiter in Bash?
– Trevor Boyd Smith
Jun 17 '16 at 14:55




question with +922 total votes: How do I split a string on a delimiter in Bash?
– Trevor Boyd Smith
Jun 17 '16 at 14:55










5 Answers
5






active

oldest

votes

















up vote
25
down vote













Portably:



set -f # turn off globbing
IFS='
' # split at newlines only
cmd $(cat <file)
unset IFS
set +f


Or using a subshell to make the IFS and option changes local:



( set -f; IFS='
'; exec cmd $(cat <file) )


The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.



There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.



In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "$a[@]" expands to the elements of the array, one per word.



set -f; IFS=$'n'
a=($(cat <file))
set +f; unset IFS
cmd "$a[@]"





share|improve this answer



























    up vote
    10
    down vote













    You can do this with a temporary array.



    Setup:



    $ cat input
    AAA
    A B C
    DE F
    $ cat t.sh
    #! /bin/bash
    echo "$1"
    echo "$2"
    echo "$3"


    Fill the array:



    $ IFS=$'n'; set -f; foo=($(<input))


    Use the array:



    $ for a in "$foo[@]" ; do echo "--" "$a" "--" ; done
    -- AAA --
    -- A B C --
    -- DE F --

    $ ./t.sh "$foo[@]"
    AAA
    A B C
    DE F


    Can't figure out a way of doing that without that temporary variable - unless the IFS change isn't important for cmd, in which case:



    $ IFS=$'n'; set -f; cmd $(<input) 


    should do it.






    share|improve this answer






















    • IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
      – Old Pro
      May 27 '12 at 23:27







    • 6




      @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
      – Gilles
      May 28 '12 at 0:17

















    up vote
    8
    down vote



    accepted










    Looks like the canonical way to do this in bash is something like



    unset args
    while IFS= read -r line; do
    args+=("$line")
    done < file

    cmd "$args[@]"


    or, if your version of bash has mapfile:



    mapfile -t args < filename
    cmd "$args[@]"


    The only difference I can find between the mapfile and the while-read loop versus the one-liner



    (set -f; IFS=$'n'; cmd $(<file))


    is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I'd prefer anyway, so double bonus on it being compact.



    I would use IFS=$'n' cmd $(<file) but it doesn't work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.



    Though it doesn't work in my case, I've now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:



    find / -name '*.config' -print0 | xargs -0 md5


    feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution



    tr "n" "00" <file | xargs -0 cmd


    although this, too, ignores empty lines, though it does capture lines that have only whitespace.






    share|improve this answer






















    • Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
      – user79743
      Jul 24 '15 at 21:51

















    up vote
    3
    down vote













    You could use the bash built-in mapfile to read the file into an array



    mapfile -t foo < filename
    cmd "$foo[@]"


    or, untested, xargs might do it



    xargs cmd < filename





    share|improve this answer




















    • From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
      – Old Pro
      May 27 '12 at 22:36










    • You would need xargs -d or xargs -L
      – James Youngman
      May 27 '12 at 22:48










    • @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
      – Old Pro
      May 27 '12 at 23:43






    • 1




      @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
      – glenn jackman
      May 28 '12 at 1:32










    • mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
      – Peter.O
      May 28 '12 at 5:02


















    up vote
    0
    down vote













    old=$IFS
    IFS=' '
    array=`cat Submissions` #input the text in this variable
    for ... #use parts of variable in the for loop
    ...
    done
    IFS=$old


    Best way I could find. Just Works.





    share








    New contributor




    MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.

















      Your Answer







      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: false,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













       

      draft saved


      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f39473%2fcommand-substitution-splitting-on-newline-but-not-space%23new-answer', 'question_page');

      );

      Post as a guest






























      5 Answers
      5






      active

      oldest

      votes








      5 Answers
      5






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      25
      down vote













      Portably:



      set -f # turn off globbing
      IFS='
      ' # split at newlines only
      cmd $(cat <file)
      unset IFS
      set +f


      Or using a subshell to make the IFS and option changes local:



      ( set -f; IFS='
      '; exec cmd $(cat <file) )


      The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.



      There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.



      In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "$a[@]" expands to the elements of the array, one per word.



      set -f; IFS=$'n'
      a=($(cat <file))
      set +f; unset IFS
      cmd "$a[@]"





      share|improve this answer
























        up vote
        25
        down vote













        Portably:



        set -f # turn off globbing
        IFS='
        ' # split at newlines only
        cmd $(cat <file)
        unset IFS
        set +f


        Or using a subshell to make the IFS and option changes local:



        ( set -f; IFS='
        '; exec cmd $(cat <file) )


        The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.



        There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.



        In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "$a[@]" expands to the elements of the array, one per word.



        set -f; IFS=$'n'
        a=($(cat <file))
        set +f; unset IFS
        cmd "$a[@]"





        share|improve this answer






















          up vote
          25
          down vote










          up vote
          25
          down vote









          Portably:



          set -f # turn off globbing
          IFS='
          ' # split at newlines only
          cmd $(cat <file)
          unset IFS
          set +f


          Or using a subshell to make the IFS and option changes local:



          ( set -f; IFS='
          '; exec cmd $(cat <file) )


          The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.



          There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.



          In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "$a[@]" expands to the elements of the array, one per word.



          set -f; IFS=$'n'
          a=($(cat <file))
          set +f; unset IFS
          cmd "$a[@]"





          share|improve this answer












          Portably:



          set -f # turn off globbing
          IFS='
          ' # split at newlines only
          cmd $(cat <file)
          unset IFS
          set +f


          Or using a subshell to make the IFS and option changes local:



          ( set -f; IFS='
          '; exec cmd $(cat <file) )


          The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.



          There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.



          In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "$a[@]" expands to the elements of the array, one per word.



          set -f; IFS=$'n'
          a=($(cat <file))
          set +f; unset IFS
          cmd "$a[@]"






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered May 28 '12 at 0:15









          Gilles

          516k12210261555




          516k12210261555






















              up vote
              10
              down vote













              You can do this with a temporary array.



              Setup:



              $ cat input
              AAA
              A B C
              DE F
              $ cat t.sh
              #! /bin/bash
              echo "$1"
              echo "$2"
              echo "$3"


              Fill the array:



              $ IFS=$'n'; set -f; foo=($(<input))


              Use the array:



              $ for a in "$foo[@]" ; do echo "--" "$a" "--" ; done
              -- AAA --
              -- A B C --
              -- DE F --

              $ ./t.sh "$foo[@]"
              AAA
              A B C
              DE F


              Can't figure out a way of doing that without that temporary variable - unless the IFS change isn't important for cmd, in which case:



              $ IFS=$'n'; set -f; cmd $(<input) 


              should do it.






              share|improve this answer






















              • IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
                – Old Pro
                May 27 '12 at 23:27







              • 6




                @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
                – Gilles
                May 28 '12 at 0:17














              up vote
              10
              down vote













              You can do this with a temporary array.



              Setup:



              $ cat input
              AAA
              A B C
              DE F
              $ cat t.sh
              #! /bin/bash
              echo "$1"
              echo "$2"
              echo "$3"


              Fill the array:



              $ IFS=$'n'; set -f; foo=($(<input))


              Use the array:



              $ for a in "$foo[@]" ; do echo "--" "$a" "--" ; done
              -- AAA --
              -- A B C --
              -- DE F --

              $ ./t.sh "$foo[@]"
              AAA
              A B C
              DE F


              Can't figure out a way of doing that without that temporary variable - unless the IFS change isn't important for cmd, in which case:



              $ IFS=$'n'; set -f; cmd $(<input) 


              should do it.






              share|improve this answer






















              • IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
                – Old Pro
                May 27 '12 at 23:27







              • 6




                @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
                – Gilles
                May 28 '12 at 0:17












              up vote
              10
              down vote










              up vote
              10
              down vote









              You can do this with a temporary array.



              Setup:



              $ cat input
              AAA
              A B C
              DE F
              $ cat t.sh
              #! /bin/bash
              echo "$1"
              echo "$2"
              echo "$3"


              Fill the array:



              $ IFS=$'n'; set -f; foo=($(<input))


              Use the array:



              $ for a in "$foo[@]" ; do echo "--" "$a" "--" ; done
              -- AAA --
              -- A B C --
              -- DE F --

              $ ./t.sh "$foo[@]"
              AAA
              A B C
              DE F


              Can't figure out a way of doing that without that temporary variable - unless the IFS change isn't important for cmd, in which case:



              $ IFS=$'n'; set -f; cmd $(<input) 


              should do it.






              share|improve this answer














              You can do this with a temporary array.



              Setup:



              $ cat input
              AAA
              A B C
              DE F
              $ cat t.sh
              #! /bin/bash
              echo "$1"
              echo "$2"
              echo "$3"


              Fill the array:



              $ IFS=$'n'; set -f; foo=($(<input))


              Use the array:



              $ for a in "$foo[@]" ; do echo "--" "$a" "--" ; done
              -- AAA --
              -- A B C --
              -- DE F --

              $ ./t.sh "$foo[@]"
              AAA
              A B C
              DE F


              Can't figure out a way of doing that without that temporary variable - unless the IFS change isn't important for cmd, in which case:



              $ IFS=$'n'; set -f; cmd $(<input) 


              should do it.







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited May 28 '12 at 0:16









              Gilles

              516k12210261555




              516k12210261555










              answered May 27 '12 at 21:35









              Mat

              38.3k7117125




              38.3k7117125











              • IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
                – Old Pro
                May 27 '12 at 23:27







              • 6




                @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
                – Gilles
                May 28 '12 at 0:17
















              • IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
                – Old Pro
                May 27 '12 at 23:27







              • 6




                @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
                – Gilles
                May 28 '12 at 0:17















              IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
              – Old Pro
              May 27 '12 at 23:27





              IFS always gets me confused. IFS=$'n' cmd $(<input) doesn't work. IFS=$'n'; cmd $(<input); unset IFS does work. Why? I guess I'll use (IFS=$'n'; cmd $(<input))
              – Old Pro
              May 27 '12 at 23:27





              6




              6




              @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
              – Gilles
              May 28 '12 at 0:17




              @OldPro IFS=$'n' cmd $(<input) doesn't work because it only sets IFS in the environment of cmd. $(<input) is expanded to form the command, before the assignment to IFS is performed.
              – Gilles
              May 28 '12 at 0:17










              up vote
              8
              down vote



              accepted










              Looks like the canonical way to do this in bash is something like



              unset args
              while IFS= read -r line; do
              args+=("$line")
              done < file

              cmd "$args[@]"


              or, if your version of bash has mapfile:



              mapfile -t args < filename
              cmd "$args[@]"


              The only difference I can find between the mapfile and the while-read loop versus the one-liner



              (set -f; IFS=$'n'; cmd $(<file))


              is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I'd prefer anyway, so double bonus on it being compact.



              I would use IFS=$'n' cmd $(<file) but it doesn't work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.



              Though it doesn't work in my case, I've now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:



              find / -name '*.config' -print0 | xargs -0 md5


              feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution



              tr "n" "00" <file | xargs -0 cmd


              although this, too, ignores empty lines, though it does capture lines that have only whitespace.






              share|improve this answer






















              • Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
                – user79743
                Jul 24 '15 at 21:51














              up vote
              8
              down vote



              accepted










              Looks like the canonical way to do this in bash is something like



              unset args
              while IFS= read -r line; do
              args+=("$line")
              done < file

              cmd "$args[@]"


              or, if your version of bash has mapfile:



              mapfile -t args < filename
              cmd "$args[@]"


              The only difference I can find between the mapfile and the while-read loop versus the one-liner



              (set -f; IFS=$'n'; cmd $(<file))


              is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I'd prefer anyway, so double bonus on it being compact.



              I would use IFS=$'n' cmd $(<file) but it doesn't work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.



              Though it doesn't work in my case, I've now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:



              find / -name '*.config' -print0 | xargs -0 md5


              feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution



              tr "n" "00" <file | xargs -0 cmd


              although this, too, ignores empty lines, though it does capture lines that have only whitespace.






              share|improve this answer






















              • Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
                – user79743
                Jul 24 '15 at 21:51












              up vote
              8
              down vote



              accepted







              up vote
              8
              down vote



              accepted






              Looks like the canonical way to do this in bash is something like



              unset args
              while IFS= read -r line; do
              args+=("$line")
              done < file

              cmd "$args[@]"


              or, if your version of bash has mapfile:



              mapfile -t args < filename
              cmd "$args[@]"


              The only difference I can find between the mapfile and the while-read loop versus the one-liner



              (set -f; IFS=$'n'; cmd $(<file))


              is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I'd prefer anyway, so double bonus on it being compact.



              I would use IFS=$'n' cmd $(<file) but it doesn't work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.



              Though it doesn't work in my case, I've now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:



              find / -name '*.config' -print0 | xargs -0 md5


              feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution



              tr "n" "00" <file | xargs -0 cmd


              although this, too, ignores empty lines, though it does capture lines that have only whitespace.






              share|improve this answer














              Looks like the canonical way to do this in bash is something like



              unset args
              while IFS= read -r line; do
              args+=("$line")
              done < file

              cmd "$args[@]"


              or, if your version of bash has mapfile:



              mapfile -t args < filename
              cmd "$args[@]"


              The only difference I can find between the mapfile and the while-read loop versus the one-liner



              (set -f; IFS=$'n'; cmd $(<file))


              is that the former will convert a blank line to an empty argument, while the one-liner will ignore a blank line. In this case the one-liner behavior is what I'd prefer anyway, so double bonus on it being compact.



              I would use IFS=$'n' cmd $(<file) but it doesn't work, because $(<file) is interpreted to form the command line before IFS=$'n' takes effect.



              Though it doesn't work in my case, I've now learned that a lot of tools support terminating lines with null (00) instead of newline (n) which does make a lot of this easier when dealing with, say, file names, which are common sources of these situations:



              find / -name '*.config' -print0 | xargs -0 md5


              feeds a list of fully-qualified file names as arguments to md5 without any globbing or interpolating or whatever. That leads to the non-built-in solution



              tr "n" "00" <file | xargs -0 cmd


              although this, too, ignores empty lines, though it does capture lines that have only whitespace.







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited May 28 '12 at 7:54

























              answered May 28 '12 at 0:32









              Old Pro

              5742514




              5742514











              • Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
                – user79743
                Jul 24 '15 at 21:51
















              • Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
                – user79743
                Jul 24 '15 at 21:51















              Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
              – user79743
              Jul 24 '15 at 21:51




              Using cmd $(<file) values without quoting (using the ability of bash to split words) is always a risky bet. If any line is * it will be expanded by the shell to a list of files.
              – user79743
              Jul 24 '15 at 21:51










              up vote
              3
              down vote













              You could use the bash built-in mapfile to read the file into an array



              mapfile -t foo < filename
              cmd "$foo[@]"


              or, untested, xargs might do it



              xargs cmd < filename





              share|improve this answer




















              • From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
                – Old Pro
                May 27 '12 at 22:36










              • You would need xargs -d or xargs -L
                – James Youngman
                May 27 '12 at 22:48










              • @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
                – Old Pro
                May 27 '12 at 23:43






              • 1




                @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
                – glenn jackman
                May 28 '12 at 1:32










              • mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
                – Peter.O
                May 28 '12 at 5:02















              up vote
              3
              down vote













              You could use the bash built-in mapfile to read the file into an array



              mapfile -t foo < filename
              cmd "$foo[@]"


              or, untested, xargs might do it



              xargs cmd < filename





              share|improve this answer




















              • From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
                – Old Pro
                May 27 '12 at 22:36










              • You would need xargs -d or xargs -L
                – James Youngman
                May 27 '12 at 22:48










              • @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
                – Old Pro
                May 27 '12 at 23:43






              • 1




                @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
                – glenn jackman
                May 28 '12 at 1:32










              • mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
                – Peter.O
                May 28 '12 at 5:02













              up vote
              3
              down vote










              up vote
              3
              down vote









              You could use the bash built-in mapfile to read the file into an array



              mapfile -t foo < filename
              cmd "$foo[@]"


              or, untested, xargs might do it



              xargs cmd < filename





              share|improve this answer












              You could use the bash built-in mapfile to read the file into an array



              mapfile -t foo < filename
              cmd "$foo[@]"


              or, untested, xargs might do it



              xargs cmd < filename






              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered May 27 '12 at 22:03









              glenn jackman

              49.1k469106




              49.1k469106











              • From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
                – Old Pro
                May 27 '12 at 22:36










              • You would need xargs -d or xargs -L
                – James Youngman
                May 27 '12 at 22:48










              • @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
                – Old Pro
                May 27 '12 at 23:43






              • 1




                @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
                – glenn jackman
                May 28 '12 at 1:32










              • mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
                – Peter.O
                May 28 '12 at 5:02

















              • From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
                – Old Pro
                May 27 '12 at 22:36










              • You would need xargs -d or xargs -L
                – James Youngman
                May 27 '12 at 22:48










              • @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
                – Old Pro
                May 27 '12 at 23:43






              • 1




                @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
                – glenn jackman
                May 28 '12 at 1:32










              • mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
                – Peter.O
                May 28 '12 at 5:02
















              From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
              – Old Pro
              May 27 '12 at 22:36




              From the mapfile documentation: "mapfile isn't a common or portable shell feature". And indeed is it not supported on my system. xargs doesn't help, either.
              – Old Pro
              May 27 '12 at 22:36












              You would need xargs -d or xargs -L
              – James Youngman
              May 27 '12 at 22:48




              You would need xargs -d or xargs -L
              – James Youngman
              May 27 '12 at 22:48












              @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
              – Old Pro
              May 27 '12 at 23:43




              @James, no, I don't have a -d option and xargs -L 1 runs the command once per line but still splits args on whitespace.
              – Old Pro
              May 27 '12 at 23:43




              1




              1




              @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
              – glenn jackman
              May 28 '12 at 1:32




              @OldPro, well you did ask for "a way to do it using only bash built-ins" instead of "a common or portable shell feature". If your version of bash is too old, can you update it?
              – glenn jackman
              May 28 '12 at 1:32












              mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
              – Peter.O
              May 28 '12 at 5:02





              mapfile is very handy for me, as it grabs blank lines as array items, which the IFS method does not do. IFS treats contiguous newlines as a single delimiter... Thanks for presenting it, as I wasn't aware of the command (though, based on the OP's input data and the expected command line, it seems he actually wants to ignore blank lines).
              – Peter.O
              May 28 '12 at 5:02











              up vote
              0
              down vote













              old=$IFS
              IFS=' '
              array=`cat Submissions` #input the text in this variable
              for ... #use parts of variable in the for loop
              ...
              done
              IFS=$old


              Best way I could find. Just Works.





              share








              New contributor




              MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
              Check out our Code of Conduct.





















                up vote
                0
                down vote













                old=$IFS
                IFS=' '
                array=`cat Submissions` #input the text in this variable
                for ... #use parts of variable in the for loop
                ...
                done
                IFS=$old


                Best way I could find. Just Works.





                share








                New contributor




                MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                Check out our Code of Conduct.



















                  up vote
                  0
                  down vote










                  up vote
                  0
                  down vote









                  old=$IFS
                  IFS=' '
                  array=`cat Submissions` #input the text in this variable
                  for ... #use parts of variable in the for loop
                  ...
                  done
                  IFS=$old


                  Best way I could find. Just Works.





                  share








                  New contributor




                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  old=$IFS
                  IFS=' '
                  array=`cat Submissions` #input the text in this variable
                  for ... #use parts of variable in the for loop
                  ...
                  done
                  IFS=$old


                  Best way I could find. Just Works.






                  share








                  New contributor




                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.








                  share


                  share






                  New contributor




                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.









                  answered 8 mins ago









                  MEGADON

                  12




                  12




                  New contributor




                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.





                  New contributor





                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.






                  MEGADON is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
                  Check out our Code of Conduct.



























                       

                      draft saved


                      draft discarded















































                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f39473%2fcommand-substitution-splitting-on-newline-but-not-space%23new-answer', 'question_page');

                      );

                      Post as a guest













































































                      Popular posts from this blog

                      How to check contact read email or not when send email to Individual?

                      Bahrain

                      Postfix configuration issue with fips on centos 7; mailgun relay