How to execute a PowerShell function several times in parallel?

start-parallel powershell
powershell foreach -parallel throttle limit
powershell invoke-command parallel
powershell start-job
powershell run multiple functions simultaneously
powershell callback function example
invoke-async powershell
run multiple powershell scripts in parallel

I'm not sure whether to call this a need for multi-threading, job-based, or async, but basically I have a Powershell script function that takes several parameters and I need to call it several times with different parameters and have these run in parallel.

Currently, I call the function like this:

Execute "param1" "param2" "param3" "param4"

How can I call this multiple times without waiting for each call to Execute return to the caller?

Currently I'm running v2.0 but I can update if necessary

EDIT: here's what I have so far, which doesn't work:

$cmd = {
    param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
    Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}

Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName

I get an error:

cannot convert 'system.object[]' to the type 'system.management.automation.scriptblock' required by parameter 'initializationscript'. specified method is not supported

EDIT2: I've modified my script but I still get the error mentioned above. Here's my mod:

$cmd = {
    param($vmxFilePath,$machineName,$username,$password,$scriptTpath,$scriptFile,$uacDismissScript,$snapshotName)
    Execute $vmxFilePath $machineName $username $password $scriptTpath $scriptFile $uacDismissScript $snapshotName
}

Start-Job -ScriptBlock $cmd -ArgumentList $vmxFilePath, $machineName, $username $password, $scriptTpath, $scriptFile, $uacDismissScript, $snapshotName

No update necessary for this. Define a script block and use Start-Job to run the script block as many times as necessary. Example:

$cmd = {
  param($a, $b)
  Write-Host $a $b
}

$foo = "foo"

1..5 | ForEach-Object {
  Start-Job -ScriptBlock $cmd -ArgumentList $_, $foo
}

The script block takes 2 parameters $a and $b which are passed by the -ArgumentList option. In the example above, the assignments are $_$a and $foo$b. $foo is just an example for a configurable, but static parameter.

Run Get-Job | Remove-Job at some point to remove the finished jobs from the queue (or Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id } if you want to retrieve the output).

How to run powershell Command all at a same time (Parallel , Currently i have bunch of powershell commands i can run those commands one by one using batch file or etc.,. But i need to run all the commands at a same time (Parallel). Kindly help Also a lot of things that work in normal scripts won't work in workflows. Jobs and Still, here's a popular function for using Runspaces:. I've made a very versitile function to do this for you, and unlike the other answers you dont need to rejig your code to get it to work. Just pass your function as a parameter to Async and pipe your input in, each item on the pipeline will run your scriptblock in parallel asynchronously, and emit them as each one is completed.

Here's a quick bogus scriptblock for the purpose of testing:

$Code = {
    param ($init)
    $start = Get-Date
    (1..30) | % { Start-Sleep -Seconds 1; $init +=1 }
    $stop = Get-Date
    Write-Output "Counted from $($init - 30) until $init in $($stop - $start)."
}

This scriptblock can then be passed on to Start-Job, with for example 3 parameters (10, 15, 35)

$jobs = @()
(10,15,35) | % { $jobs += Start-Job -ArgumentList $_ -ScriptBlock $Code }

Wait-Job -Job $jobs | Out-Null
Receive-Job -Job $jobs

This creates 3 jobs, assign them to the $jobs variable, runs them in parallel and then waits for these 3 jobs to finish, and retrieves the results:

Counted from 10 until 40 in 00:00:30.0147167.
Counted from 15 until 45 in 00:00:30.0057163.
Counted from 35 until 65 in 00:00:30.0067163.

This did not take 90 seconds to execute, only 30.

One of the tricky parts is to provide -Argumentlist to Start-Job, and include a param() block inside the ScriptBlock. Otherwise, your values are never seen by the scriptblock.

PowerShell ForEach-Object Parallel Feature, The -ThrottleLimit parameter limits the number of script blocks running in parallel at a given time, and its default value is 5 . This new feature  multiple child scripts (=jobs) each child script runs in it's own PowerShell context / session. the master script adds the jobs to run in parallel. the master script collects the child results.

I'm sorry that everyone missed your issue - I know it is far too late now, but...

This error is caused because you are missing a comma between $username and $password in your list.

You can test it out with this snippet, which I modelled off of the previous answers:

$cmd = {
  param($a, $b, $c, $d)
}
$foo = "foo"
$bar = "bar"
start-job -scriptblock $cmd -ArgumentList "a", $foo, $bar, "gold" #added missing comma for this to work

Running Commands/ScriptBlocks/Functions in Parallel, I was writing a product deployment script in Powershell, and I was having I was having an issue getting two of the processes to run at the same time. and it requires you to understand a bit more about how PowerShell, and  Currently i have bunch of powershell commands i can run those commands one by one using batch file or etc.,. But i need to run all the commands at a same time (Parallel). Kindly help me . Regards, Sandy

You can use an alternative which may be faster than invoking jobs if the function is not a long running one. Max thread is 25 and I'm only invoking this function 10 times, so I expect my total runtime to be 5 seconds. You can wrap Measure-Command around the 'results=' statement to view stats.

Example:

$ScriptBlock = {
    Param ( [int]$RunNumber )
    Start-Sleep -Seconds 5   
    Return $RunNumber
}
$runNumbers = @(1..10)
$MaxThreads = 25
$runspacePool = [RunspaceFactory ]::CreateRunspacePool(1, $MaxThreads)
$runspacePool.Open()
$pipeLines = foreach($num in $runNumbers){
    $pipeline = [powershell]::Create()
    $pipeline.RunspacePool = $runspacePool
    $pipeline.AddScript($ScriptBlock)    | Out-Null   
    $pipeline.AddArgument($num)  | Out-Null
    $pipeline | Add-Member -MemberType NoteProperty -Name 'AsyncResult' -Value $pipeline.BeginInvoke() -PassThru 
}
#obtain results as they come.
$results =  foreach($pipeline in $pipeLines){
    $pipeline.EndInvoke($pipeline.AsyncResult )
}
#cleanup code.
$pipeLines | % { $_.Dispose()}
$pipeLines = $null
if ( $runspacePool ) { $runspacePool.Close()}
#your results
$results

Start-parallel.psm1 1.3.0.0, Runs multiple instances of a command in parallel. .Description This script will allow mutliple instances of a command to run at the same time with different  In broad terms, 1 Runspace = 1 thread of execution, so all we need to "multi-thread" our PowerShell script is a collection of Runspaces that can then in turn execute in parallel. Like the original problem, the job of invoking commands multiple runspaces can be broken down into:

Building Asynchronous PowerShell Functions, The most frequent kind of PowerShell function is a function that executes code There are those times when the code inside of your function may take a Asynchronous functions can be written in a few different ways; workflows, parallel runspaces However, I typically use jobs to run asynchronous code. The whole point is that you can execute this block of PowerShell code simply by typing the name of the function. Example 1: A PowerShell Function to Calculate Baseball Averages Here is a classic example for dissecting, fiddling, changing stuff to see what happens, and thus truly learning how functions are constructed.

PowerShell workflows, Functions do one thing at a time; workflows can do one thing at multiple times—​they can do parallel multitasking. Functions start, run, and finish; a workflow can  We can use multi threading in powershell v2 using start-job command which will execute the script block on multiple servers parallely rather than going one by one. In powershell v3 there is additional option to execute command parallel within foreach loop itself Foreach -parallel ($srv in gc c:\input.txt)

Chapter 23. PowerShell workflows, Functions do one thing at a time; workflows can do one thing at multiple times—​they can do parallel multitasking. Functions start, run, and finish; a workflow can  Invoke-Parallel - Parallel PowerShell This function will take in a script or scriptblock, and run it against specified objects(s) in parallel. It uses runspaces, as there are many situations where jobs or PSRemoting are not appropriate.

Execute multiple Powershell tasks in parallel |VMware Communities, So in a foreach loop if i have 4 VM's, it takes around 1.5 ~ 2 hours. Is there any I run that ps1 file before executing my main script. Sorry i am  There is no built-in mechanism in the core of Powershell to run things in Parallel explicitly. You have several options: 1. Create a workflow - in a workflow you CAN run things in Parallel.

Comments
  • I'm still having a hard time mapping your solution to get my own script to work. I'm going to start simple, and just try to get Start-Job to run my scriptblock with the arguments which my custom function, Execute, requires. Maybe you can expand? Is $foo a parameter in addition to the ones specificed by param($a, $b) ?
  • I see now, thanks. I still get the error though. If you're unsure what this could be I'll just create separate post/question.
  • For testing purposes replace the call of your Execute() function with code that just echoes the parameters. Does that work? If it does, the problem lies with the function you want to call.