Limiting the number of simultaneous instances of a program executed within a Perl script (to >1)
I'm using a resource-intensive program in a Perl script [specifically
rcloneI have yet to figure out how I want to call
rclonerclonercloneSome details:
- The script itself is essentially passed a file or directory path containing (possibly numerous) files by another program (this program written in Python--call this program for reference).
<A> - only returns a value to a script and thus knows nothing about the script or
<A>, other than that it accepts input.rclone - cannot be altered (i.e. changing
<A>is beyond my ken)<A> - fires at varying intervals [i.e. sometimes it will execute the script many times rapidly in succession (creating multiple instances), other times it might only fire once every few hours, minutes, etc.]
<A> - Assume that can't be altered directly either (i.e. again, beyond my ken).
rclone - If absolutely necessary, the number of instances of the script can be limited instead of (though I'd prefer it only be
rclone, as the processing done by the script is rather light and needs no limitation).rclone - Modules are fine to use.
- I would like to avoid using Unix-like operating system commands like and
pgrep(unless absolutely necessary).ps
Currently, I'm using a rather poorly written bash script in place of the Perl script. The bash script implements a rudimentary (poorly designed) "check/sleep loop" using
pgrep -wcsleepwhileif
I'll assume for a moment that your script is the only thing running rclone. If you wanted only 1 copy running, you would just use a lockfile.
For N instances (for small N), I would just have N lockfiles - have the program try each lock in turn, in a loop; pause if all the locks are already held and retry 1s later, in a loop. Once it has a lock, run rclone then release the lock when it is done.
A more sound approach would be to use SysV semaphores but, unless you want a large N, really care about response times or are worried about fairness between callers, it is not likely to be worth the time learning them.
If your script is not the only program calling rclone, then would need to intercept all calls - instead of putting this code in your program, could replace rclone by wrapper that implements the parallelism constraint as above and then calls the real program.