alex alex - 1 year ago 61
Bash Question

Bash wait does not work

Part of my code:

for i in "${r_arr[@]}"
${parallel_dir}parallel -j${cap} --semaphore --semaphorename a2_bootstrap_semaphore_${rnd} a2_exec ${i} ${temp_zmat_folder_path}${temp_zmat_folder}/ ${outs_folder}

elapsed_time=$(($SECONDS - $start_time))

rm -rf "${temp_zmat_folder_path}${temp_zmat_folder}"
echo "Cleanup done."
echo "`date +%d-%m-%y` `date +%T`: All processing finished in ${elapsed_time} seconds!"

As you can see I'm starting a number of tasks using GNU Parallel. The problem is that cleanup (
) is executed before last of the parallel tasks is even started. How to prevent such behaviour?

Answer Source

As per their example, you should use this command to wait for all the started jobs to complete

parallel --semaphore --wait

Example (From man page): The command sem is an alias for parallel --semaphore.

for i in *.log ; do
    echo $i
    sem -j10 gzip $i ";" echo done
  sem --wait