mjhjmtu

What I'm really trying to do is run X number of jobs, with X amount in parallel for testing an API race condition.

I've come up with this

echo 1..10 | xargs -n1 | parallel -m 'echo ""';

which prints

7 8 9
10
4 5 6
1 2 3

but what I really want to see is (note order doesn't actually matter).

1
2
3
4
5
6
7
8
9
10

and those would be processed in parallel 4 at a time (or whatever number of cpus/cores, I have, e.g. --jobs 4). For a total of 10 separate executions.

I tried this

echo 1..10 | xargs -n1 | parallel --semaphore --block 3 -m 'echo -n " ";

but it only ever seems to print once. bonus points if your solution doesn't need xargs which seems like a hack around the idea that the default record separator is a newline, but I haven't been able to get a space to work like I want either.

10 is a reasonably small number, but lets say it's much larger, 1000

echo 1..1000 | xargs -n1 | parallel -j1000

prints

parallel: Warning: Only enough file handles to run 60 jobs in parallel.
parallel: Warning: Running 'parallel -j0 -N 60 --pipe parallel -j0' or
parallel: Warning: raising 'ulimit -n' or 'nofile' in /etc/security/limits.conf
parallel: Warning: or /proc/sys/fs/file-max may help.

I don't actually want 1000 processes, I want 4 processes at a time, each process should process 1 record, thus by the time I'm done it will have executed 1000 times.

I want 4 processes at a time, each process should process 1 record

parallel -j4 -k --no-notice 'echo ""' ::: 1..10

• -j4 - number of jobslots. Run up to 4 jobs in parallel




• -k - keep sequence of output same as the order of input. Normally the output of a job will be printed as soon as the job completes




• ::: - arguments

The output:

1
2
3
4
5
6
7
8
9
10

seq 10 | parallel -j4 echo 

Or if you have 4 cores:

seq 10 | parallel echo