Schezuk Schezuk - 1 year ago 116
Linux Question

How to get results from parallel sub-shells in Bash?

I am running about 100 sub-shells and try to echo all results of them. Code:

#!/usr/bin/env bash
function some_function(){} #spider
# ... ...
for i in {1..100}
array[$i]=some_function $i &
echo ${array[@]}

I know these evaluations happen in sub-shells so nothing will I get.

But I cannot use

echo $(some_function $i) &

in the loop instead, because of racing, which made everything unreadable.

No idea why

(set -a; array[$i]=some_function $i; set +a)&


if true; then set -a; array[$i]=some_function $i; set +a; fi &

didn't work either.

I did my best to avoid using any temp file or fifo or fd elsewhere.

What is the proper way to do it?

Edit: I wonder if GNU-Parallel will work, though it is not in msys2.

Answer Source

A subshell cannot set the value of a variable (or array element) of its parent. Your only option is to write the output to a file.

for i in {1..100}; do
    some_function "$i" > "tmp_$i.txt" &


for i in {1..100}; do

(I thought there might be a way to use coprocesses, but it appears you cannot have more than one coprocess at a time.)

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download