Michael M. Michael M. - 3 months ago 9
PowerShell Question

Performance advantage - multiple PowerShells vs Background-Jobs

I made a programm which is starting multiple Background-Jobs at the same time. Currently the process is taking several hours (sometimes 3 hours, sometimes up to 6 hours depending on the files and size).

Is there any advantage in rewriting the code so it doesn't start all the processes in the same Shell using Background-Jobs but instead starts a PowerShell for each individual Process? Or isn't there any big difference between both of them?

$handler_button1_Click=
{
if ($checkBox1.Checked) {
Try{
$job1 = start-jobhere {& C:\Users\mosermich\Desktop\Extractor\Ressources\adp_staging.bat} -Name "ADP-DATA"
$listBox1.Items.Add("ADP-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox2.Checked) {
Try{
$job2 = start-jobhere {& C:\Users\mosermich\Desktop\Extractor\Ressources\kdp_staging.bat} -Name "KDP-DATA"
$listBox1.Items.Add("KDP-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox3.Checked) {
Try{
$job3 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\mdp_staging.bat} -Name "MDP-DATA"
$listBox1.Items.Add("MDP-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox4.Checked) {
Try{
$job4 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zdlb_staging.bat} -Name "ZDL-B-DATA"
$listBox1.Items.Add("ZDLB-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox5.Checked) {
Try{
$job5 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zdls_staging.bat} -Name "ZDL-S-DATA"
$listBox1.Items.Add("ZDLS-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox6.Checked) {
Try{
$job6 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zub_staging.bat} -Name "ZUBOFI-DATA"
$listBox1.Items.Add("ZUBOFI-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox7.Checked) {
Try{
$job7 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\adp_staging_error.bat} -Name "ADP-ERR"
$listBox1.Items.Add("ADP-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox8.Checked) {
Try{
$job8 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\kdp_staging_error.bat} -Name "KDP-ERR"
$listBox1.Items.Add("KDP-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox9.Checked) {
Try{
$job9 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\mdp_staging_error.bat} -Name "MDP-ERR"
$listBox1.Items.Add("MDP-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox10.Checked) {
Try{
$job10 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zdlb_staging_error.bat} -Name "ZDL-B-ERR"
$listBox1.Items.Add("ZDL-B-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox11.Checked) {
Try{
$job11 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zdls_staging_error.bat} -Name "ZDL-S-ERR"
$listBox1.Items.Add("ZDL-S-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($checkBox12.Checked) {
Try{
$job12 = start-jobhere { & C:\Users\mosermich\Desktop\Extractor\Ressources\zub_staging_error.bat} -Name "ZUBOFI-ERR"
$listBox1.Items.Add("ZUBOFI-Error-staging läuft...")
}catch [System.Exception]{}
}

if ($listBox1.Items.Count -eq 0) {
$listBox1.Items.Add("No-Data!")
}

Get-Job | Wait-Job | Where State -eq "Running"


$listBox1.Items.Add("All Jobs have been succesfully finished")

}else{}

Answer

(I guess) you won't see a big difference as spawning background jobs with Start-Job or calling comdlets with the -AsJob actually creates new instances of the PowerShell executable - you can see that with task manager / process explorer etc.

If you create a lot of background jobs simultaneously then it might be the case that there is quite a big of overhead in creating and scheduling those jobs compared to their actual workload. In this case you may want to take a look at PowerShell runspaces which are more lightweight. In particular you can take a look at the excellent PoshRSJob module which makes working with runspaces look like working with the 'regular' PowerShell jobs. It also includes functionality to queue up tasks so that you don't execute 1000 tasks concurrently but only e.g. 8 at any given time.

Comments