mjhjmtu

First off I apologize for the title gore, I am struggling for a good way to summarize this issue. Anyway, I have been piping various things to functions using xargs and I am running into a weird problem.

I am currently running the command: echo "hello world" | xargs curl http://localhost:8080/function/func_wordcount -d" which takes the stdout of echo "hello world" and then pipes it to func_wordcount using the -d option. The -d option is for sending raw data and my func_wordcount takes the raw data input and prints the number of words and the number of letters.

For example, when I write echo "hello" | xargs curl http://localhost:8080/function/func_wordcount -d" the output is: 1, 5 meaning that there was one word which contained 5 letters.

However, when I try to include many words I get an error. When I write echo "hello world" | xargs curl http://localhost:8080/function/func_wordcount -d" I get the output 1, 5, then a newline with the error: curl: (6) could not resolve host: world. So I am pretty sure that it is splitting hello world into two words when I convert the stdout to raw data using the -d option.

Also, just to show that the function works without piping and converting, when I run my function with just curl http://localhost:8080/function/func_wordcount -d "hello world" I get 2, 11 showing that there are two words and 11 characters.

My question is how to work around this splitting issue. The part that I find confusing is why it is parsing just the first half of the input and completing that, and then throwing an error on the second part instead of just sending one chunk of data. I have only been able to send input that is not delimited at all by spaces so the functions uses become very limited. Any help is appreciated.

That's what xargs is about. It takes a list of words (blank or newline separated, also understanding some form of quoting) on its stdin and passes them as arguments to the command(s) it runs.

Here, if you wanted to pass hello word as one argument to curl, you'd need:

echo '"hello world"' | xargs curl...

Where xargs understands those "..." as a quoting mechanism to prevent the space between hello and world from being treated as an argument separator. xargs also understands and '...' (in different ways from POSIX shells).

With GNU xargs, you can also specify the delimiter. For instance with:

echo hello world | xargs -d 'n' curl...

Only newline would be understood as delimiter (not blanks) and the quoting mechanisms are disabled.

That lets you pass the content of every line (as opposed to every word) as argument to the command.

While -d is rarely supported outside of GNU xargs, -0, another of its extensions (to delimit on NULs and disable quote processing) is more widely supported. So you can also do:

echo hello world | tr 'n' '' | xargs -0 curl...

If there are several lines of input, xargs will pass all of them as separate arguments to curl. If you want to call one curl for every line, you can add a -n 1 option.

You could also use:

echo hello world | xargs -I@@ curl... @@

That calls one curl for every line, but note that leading blanks are stripped and xargs still does some quote processing, so should be avoided for arbitrary data.

From the GNU xargs man page:

xargs reads items from the standard input, delimited by blanks (which
can be protected with double or single quotes or a backslash) or newlines, and executes the command
one or more times with any initial-arguments followed by items read from standard input.

i.e. the default behaviour is that any whitespace in the input acts as a separator, so hello world indeed results in two arguments.

If you want xargs to keep full lines intact, use `xargs -d 'n' (in GNU xargs, I can't remember the others).

However, that would still result in a multi-line input giving your curl multiple arguments. If you want to avoid that, and run the curl command once for each input line, use xargs -d 'n' -n 1.

If your input only ever contains one line, you might as well use command substitution instead of the pipe:

curl http://... -d "$(echo "hello world")"