MasterHD MasterHD - 10 months ago 88
Linux Question

Torque nested/successive qsub call

I have a jobscript

which runs on a single CPU and compiles source code to create an executable. I then have a 2nd job script
which I call using 32 CPU's to run that newly created executable with MPI. They both work perfectly when I manually call them in succession, but I would like to automate the process by having the first script call the 2nd jobscript just before it ends. Is there a way to properly nest qsub calls or have them be called in succession?

Currently my attempt is to have the first script call the 2nd script right before it ends, but when I try that I get a strange error message from the 2nd (nested) qsub:

qsub: Bad UID for job execution MSG=ruserok failed validating masterhd/masterhd from s59-16.local

I think the 2nd script is being called properly, but maybe the permissions are not the same as when I called the original one. Obviously my user name
is allowed to run the jobscripts because it works fine when I call the jobscript manually. Is there a way to accomplish what I am trying to do?

Here is a more detailed example of the procedure. First I call the first jobscript and specify a variable with

qsub -v outpath='/home/dest_folder/' compile.pbs

variable just specifies where to copy the new executable, and then the 2nd jobscript changes to that output directory and attempts to run


#PBS -N compile
#PBS -l walltime=0:05:00
#PBS -j oe
#PBS -o ocompile.txt

#Perform compiling stuff:
module load gcc-openmpi-1.2.7
rm *.o
make -f Makefile
#Copy the executable to the destination:
cp visct ${outpath}/visct
#Change to the output path before calling the next jobscript:
cd ${outpath}
qsub jobscript


#PBS -N run_exe
#PBS -l nodes=32
#PBS -l walltime=96:00:00
#PBS -j oe
#PBS -o results.txt

module load gcc-openmpi-1.2.7
time mpiexec visct

gzS gzS

You could make a submitting script that qsubs both jobs, but makes the second execute only if, and after, the first was completed without errors:

JOB1CMD="qsub -v outpath='/home/dest_folder/' compile.pbs -t"  # -t for terse output
JOB1OUT=$(eval $JOB1CMD)
JOB1ID=${JOB1OUT%%.*}  # parse to get job id, change accordingly

JOB2CMD="qsub jobscript.pbs -W depend=afterok:$JOB1ID"
eval $JOB2CMD