Skip to content

Commit

Permalink
Merge pull request #2 from santisq/dev
Browse files Browse the repository at this point in the history
Enhacements
  • Loading branch information
santisq authored Nov 1, 2022
2 parents 06532b0 + 355e47a commit da248e5
Show file tree
Hide file tree
Showing 5 changed files with 91 additions and 55 deletions.
26 changes: 26 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Publish Workflow
on:
release:
types:
- published

env:
DOTNET_CLI_TELEMETRY_OPTOUT: 1
POWERSHELL_TELEMETRY_OPTOUT: 1
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
DOTNET_NOLOGO: true

jobs:
publish:
name: publish
runs-on: windows-latest
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'release'
steps:
- uses: actions/checkout@v3

- name: Publish to Gallery
shell: pwsh
run: >-
Publish-Module $PSScriptRoot -NuGetApiKey $env:PSGALLERY_TOKEN
env:
PSGALLERY_TOKEN: ${{ secrets.PSGALLERY_TOKEN }}
6 changes: 3 additions & 3 deletions PSParallelPipeline.psd1
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
RootModule = 'PSParallelPipeline.psm1'

# Version number of this module.
ModuleVersion = '1.0.0'
ModuleVersion = '1.0.1'

# Supported PSEditions
# CompatiblePSEditions = @()
Expand Down Expand Up @@ -78,7 +78,7 @@ CmdletsToExport = @()
VariablesToExport = @()

# Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export.
AliasesToExport = @('parallel')
AliasesToExport = @('parallel', 'parallelpipeline')

# DSC resources to export from this module
# DscResourcesToExport = @()
Expand All @@ -95,7 +95,7 @@ PrivateData = @{
PSData = @{

# Tags applied to this module. These help with module discovery in online galleries.
Tags = @('parallel', 'concurrency')
Tags = @('parallel', 'concurrency', 'runspace', 'parallel-processing')

# A URL to the license for this module.
LicenseUri = 'https://github.com/santisq/PSParallelPipeline/blob/main/LICENSE'
Expand Down
52 changes: 28 additions & 24 deletions PSParallelPipeline.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@ function Invoke-Parallel {
Enables parallel processing of pipeline input objects.
.DESCRIPTION
PowerShell function that intends to emulate `ForEach-Object -Parallel` functionality in PowerShell Core for those stuck with Windows PowerShell.
PowerShell function that intends to emulate [`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.2#-parallel) for those stuck with Windows PowerShell.
This function shares similar usage and capabilities than the ones available in the built-in cmdlet.
This was inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and Boe Prox's [`PoshRSJob`](https://github.com/proxb/PoshRSJob)
and is merely a simplified take on those.
This project is greatly inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and Boe Prox's [`PoshRSJob`](https://github.com/proxb/PoshRSJob) and is merely a simplified take on those with some few improvements.
TO DO:
- Add `-TimeoutSeconds` parameter.
Expand All @@ -31,17 +30,19 @@ function Invoke-Parallel {
Input objects are blocked until the running script block count falls below the ThrottleLimit.
The default value is `5`.
.PARAMETER ArgumentList
Specifies a hashtable of values to be passed to the Runspaces.
Hashtable Keys become the Variable Name inside the ScriptBlock.
.PARAMETER Variables
Specifies a hash table of variables to have available in the Script Block (Runspaces).
The hash table `Keys` become the Variable Name inside the Script Block.
.PARAMETER Functions
Existing functions in the Local Session to have available in the Script Block (Runspaces).
.PARAMETER ThreadOptions
These options control whether a new thread is created when a command is executed within a Runspace.
See [`PSThreadOptions` Enum](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.psthreadoptions?view=powershellsdk-7.2.0).
This parameter is limited to `ReuseThread` and `UseNewThread`.
Default value is `ReuseThread`.
See [`PSThreadOptions` Enum](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.psthreadoptions?view=powershellsdk-7.2.0) for details.
.PARAMETER Functions
Existing functions in the current scope that we want to have available in the Runspaces.
.EXAMPLE
$message = 'Hello world from {0}'
Expand All @@ -59,9 +60,9 @@ function Invoke-Parallel {
0..10 | Invoke-Parallel {
$message -f [runspace]::DefaultRunspace.InstanceId
Start-Sleep 3
} -ArgumentList @{ message = $message } -ThrottleLimit 3
} -Variables @{ message = $message } -ThrottleLimit 3
Same as Example 1 but with `-ArgumentList` parameter.
Same as Example 1 but with `-Variables` parameter.
.EXAMPLE
$sync = [hashtable]::Synchronized(@{})
Expand All @@ -80,11 +81,11 @@ function Invoke-Parallel {
Get-Process | Invoke-Parallel {
$sync[$_.Name] += @( $_ )
} -ArgumentList @{ sync = $sync }
} -Variables @{ sync = $sync }
$sync
Same as Example 3, but using `-ArgumentList` to pass the reference instance to the runspaces.
Same as Example 3, but using `-Variables` to pass the reference instance to the runspaces.
This method is the recommended when passing reference instances to the runspaces, `$using:` may fail in some situations.
.EXAMPLE
Expand All @@ -101,7 +102,7 @@ function Invoke-Parallel {
#>

[CmdletBinding(PositionalBinding = $false)]
[Alias('parallel')]
[Alias('parallel', 'parallelpipeline')]
param(
[Parameter(Mandatory, ValueFromPipeline)]
[object] $InputObject,
Expand All @@ -113,10 +114,7 @@ function Invoke-Parallel {
[int] $ThrottleLimit = 5,

[Parameter()]
[hashtable] $ArgumentList,

[Parameter()]
[PSThreadOptions] $ThreadOptions = [PSThreadOptions]::ReuseThread,
[hashtable] $Variables,

[Parameter()]
[ArgumentCompleter({
Expand All @@ -128,15 +126,19 @@ function Invoke-Parallel {

(Get-Command -CommandType Filter, Function).Name -like "$wordToComplete*"
})]
[string[]] $Functions
[string[]] $Functions,

[Parameter()]
[ValidateSet('ReuseThread', 'UseNewThread')]
[PSThreadOptions] $ThreadOptions = [PSThreadOptions]::ReuseThread
)

begin {
try {
$iss = [initialsessionstate]::CreateDefault2()

foreach($key in $ArgumentList.PSBase.Keys) {
$iss.Variables.Add([SessionStateVariableEntry]::new($key, $ArgumentList[$key], ''))
foreach($key in $Variables.PSBase.Keys) {
$iss.Variables.Add([SessionStateVariableEntry]::new($key, $Variables[$key], ''))
}

foreach($function in $Functions) {
Expand All @@ -150,9 +152,11 @@ function Invoke-Parallel {
$varText = $usingstatement.Extent.Text
$varPath = $usingstatement.SubExpression.VariablePath.UserPath

if(-not $usingParams.ContainsKey($varText)) {
$key = [Convert]::ToBase64String([Encoding]::Unicode.GetBytes($varText))
$usingParams[$key] = $ExecutionContext.SessionState.PSVariable.Get($varPath).Value
# Credits to mklement0 for catching up a bug here. Thank you!
# https://github.com/mklement0
$key = [Convert]::ToBase64String([Encoding]::Unicode.GetBytes($varText))
if(-not $usingParams.ContainsKey($key)) {
$usingParams.Add($key, $ExecutionContext.SessionState.PSVariable.Get($varPath).Value)
}
}

Expand Down
58 changes: 32 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ Enables parallel processing of pipeline input objects.

```powershell
Invoke-Parallel -InputObject <Object> [-ScriptBlock] <ScriptBlock> [-ThrottleLimit <Int32>]
[-ArgumentList <Hashtable>] [-ThreadOptions <PSThreadOptions>] [-Functions <String[]>] [<CommonParameters>]
[-Variables <Hashtable>] [-Functions <String[]>] [-ThreadOptions <PSThreadOptions>] [<CommonParameters>]
```

## DESCRIPTION

PowerShell function that intends to emulate `ForEach-Object -Parallel` functionality in PowerShell Core for those stuck with Windows PowerShell. This function shares similar usage and capabilities than the ones available in the built-in cmdlet.
PowerShell function that intends to emulate [`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.2#-parallel) for those stuck with Windows PowerShell. This function shares similar usage and capabilities than the ones available in the built-in cmdlet.

This was inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and Boe Prox's [`PoshRSJob`](https://github.com/proxb/PoshRSJob) and is merely a simplified take on those.
This project is greatly inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and Boe Prox's [`PoshRSJob`](https://github.com/proxb/PoshRSJob) and is merely a simplified take on those with some few improvements.

## REQUIREMENTS

Expand All @@ -40,15 +40,15 @@ $message = 'Hello world from {0}'
} -ThrottleLimit 3
```

### EXAMPLE 2: Same as previous example but with `-ArgumentList` parameter
### EXAMPLE 2: Same as previous example but with `-Variables` parameter

```powershell
$message = 'Hello world from {0}'
0..10 | Invoke-Parallel {
$message -f [runspace]::DefaultRunspace.InstanceId
Start-Sleep 3
} -ArgumentList @{ message = $message } -ThrottleLimit 3
} -Variables @{ message = $message } -ThrottleLimit 3
```

### EXAMPLE 3: Adding to a single thread safe instance
Expand All @@ -64,7 +64,7 @@ Get-Process | Invoke-Parallel {
$sync
```

### EXAMPLE 4: Same as previous example but using `-ArgumentList` to pass the reference instance to the Runspaces
### EXAMPLE 4: Same as previous example but using `-Variables` to pass the reference instance to the Runspaces

This method is the recommended when passing reference instances to the runspaces, `$using:` may fail in some situations.

Expand All @@ -73,7 +73,7 @@ $sync = [hashtable]::Synchronized(@{})
Get-Process | Invoke-Parallel {
$sync[$_.Name] += @( $_ )
} -ArgumentList @{ sync = $sync }
} -Variables @{ sync = $sync }
$sync
```
Expand All @@ -91,8 +91,9 @@ function Greet { param($s) "$s hey there!" }
## PARAMETERS

### -InputObject

Specifies the input objects to be processed in the ScriptBlock.
Note: This parameter is intended to be bound from pipeline.
<br>__Note: This parameter is intended to be bound from pipeline.__

```yaml
Type: Object
Expand All @@ -107,8 +108,9 @@ Accept wildcard characters: False
```
### -ScriptBlock
Specifies the operation that is performed on each input object.
This script block is run for every object in the pipeline.
<br>This script block is run for every object in the pipeline.
```yaml
Type: ScriptBlock
Expand All @@ -123,9 +125,10 @@ Accept wildcard characters: False
```
### -ThrottleLimit
Specifies the number of script blocks that are invoked in parallel.
Input objects are blocked until the running script block count falls below the ThrottleLimit.
The default value is `5`.
<br>Input objects are blocked until the running script block count falls below the ThrottleLimit.
<br>The default value is `5`.

```yaml
Type: Int32
Expand All @@ -139,9 +142,10 @@ Accept pipeline input: False
Accept wildcard characters: False
```

### -ArgumentList
Specifies a hashtable of values to be passed to the Runspaces.
Hashtable Keys become the Variable Name inside the ScriptBlock.
### -Variables

Specifies a hash table of variables to have available in the Script Block (Runspaces).
The hash table Keys become the Variable Name inside the Script Block.

```yaml
Type: Hashtable
Expand All @@ -155,35 +159,37 @@ Accept pipeline input: False
Accept wildcard characters: False
```

### -ThreadOptions
These options control whether a new thread is created when a command is executed within a Runspace.
See [`PSThreadOptions` Enum](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.psthreadoptions?view=powershellsdk-7.2.0).
Default value is `ReuseThread`.
### -Functions

Existing functions in the Local Session to have available in the Script Block (Runspaces).

```yaml
Type: PSThreadOptions
Type: String[]
Parameter Sets: (All)
Aliases:
Accepted values: Default, UseNewThread, ReuseThread, UseCurrentThread
Required: False
Position: Named
Default value: ReuseThread
Default value: None
Accept pipeline input: False
Accept wildcard characters: False
```

### -Functions
Existing functions in the current scope that we want to have available in the Runspaces.
### -ThreadOptions

These options control whether a new thread is created when a command is executed within a Runspace.
<br>This parameter is limited to `ReuseThread` and `UseNewThread`. Default value is `ReuseThread`.
<br>See [PSThreadOptions Enum](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.psthreadoptions?view=powershellsdk-7.2.0) for details.

```yaml
Type: String[]
Type: PSThreadOptions
Parameter Sets: (All)
Aliases:
Accepted values: Default, UseNewThread, ReuseThread, UseCurrentThread
Required: False
Position: Named
Default value: None
Default value: ReuseThread
Accept pipeline input: False
Accept wildcard characters: False
```
```
4 changes: 2 additions & 2 deletions tests.ps1 → tests/tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ $message = 'Hello world from {0}'
0..10 | Invoke-Parallel {
$message -f [runspace]::DefaultRunspace.InstanceId
Start-Sleep 3
} -ArgumentList @{ message = $message } -ThrottleLimit 3
} -Variables @{ message = $message } -ThrottleLimit 3

# Example 3
$sync = [hashtable]::Synchronized(@{})
Expand All @@ -29,7 +29,7 @@ $sync = [hashtable]::Synchronized(@{})

Get-Process | Invoke-Parallel {
$sync[$_.Name] += @( $_ )
} -ArgumentList @{ sync = $sync }
} -Variables @{ sync = $sync }

$sync

Expand Down

0 comments on commit da248e5

Please sign in to comment.