PowerShell jobs for multithreading 

PowerShell is a very powerful tool. Sometimes that means you might do something that takes a lot of time. There’s a simple trick to speed things up. PowerShell jobs is a crude form of multithreading (actually more like parallel processing). If you perform something time consuming over and over again it might be useful to use jobs with this.

I had this some time ago. I had a large array which I looped through with a foreach statement. Each iteration could take up to a minute and in some cases I needed to make up to 400 iterations. Since I had a fairly powerful machine at my disposal PowerShell jobs was the perfect and easy to use method of speeding things up.

Below is a slimmed down version of the script I was using. I’ve written it to be easily extensible with additional commands within the foreach loop.

Let’s walk through the script and do some explaining.

Line 1 to 4 set some variables you might want to change. The $ServerList variable holds a list of servers. In this case I’ve retrieved them from the Active Directory. I’ve searched for every server that has the string “server” in the name. I also set a $Maxjobs variable which defines the maximum amount of PowerShell jobs that may run simultaneously. Take care when increasing the amount of jobs you want run. Each job runs in a separate process. Depending on what you do within the job resource usage can increase. So do some testruns while increasing the amount of maximum jobs. The last variable to be changed is $JobTimeout. This defines the maximum amount of time the script waits for the last jobs to be run. Also change this depending on how much code you run within the job.

Line 6 to 10 is the logic needed for the foreach loop with the logic for PoSh jobs directly within the foreach loop. Keep in mind that when you start a job you have to pass variables to the job. In this case we only pass the $Server variable for usage within the job. I also name the job with the $Server variable so we can easily find the jobs later on.

Line 15 to 25 is the code that is actually being executed within the job. I add the $Server variable to a hashtable ($ServerInfo). I then retrieve a list of processes on the server which I also add to the hashtable. Finally I return the $ServerInfo variable which makes sure I can retrieve this variable from the job on a later stage.

Then outside of  the job on lines 28 to 31 (but still inside the foreach loop) I wrote the logic to wait if the maximum amount of jobs have been reached. Next up just outside the foreach loop (line 39) I wait for the last jobs to finish with the maximum timeout that is configured in the top of the script.

Then finally I stop the job if it’s still running. Retrieve the $ServerInfo variable from the job which I then save into the $ServerData array variable. Finally I remove the job and we’re done.

When looking at the output of the $ServerData variable we’re presented with this:

When you open up a single value in the array (which is the hashtable we created earlier). and subsequently open the processlist property you’ll see this:

Finally

I’ve shown you a method to quickly get a crude form a multithreading or more accurately a form of parallel processing of multiple iterations within a foreach loop. There is no need though to use it specifically to speed up a foreach loop. There are bound to more uses. Also I’d like to point out that this might not be the most efficient method of running tasks parallel in PowerShell. PowerShell also support runspaces which is more efficient at running tasks in parallel. I’ll be writing a blogpost about this in the future.

Leave a Reply