Skip to content

Commit

Permalink
Merge branch 'master' into rm-scrutinizer
Browse files Browse the repository at this point in the history
  • Loading branch information
vjik authored Oct 30, 2024
2 parents 91377e9 + f857522 commit a699200
Show file tree
Hide file tree
Showing 45 changed files with 831 additions and 131 deletions.
33 changes: 33 additions & 0 deletions .github/workflows/bechmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
on:
pull_request:
paths-ignore:
- 'docs/**'
- 'README.md'
- 'CHANGELOG.md'
- '.gitignore'
- '.gitattributes'
- 'infection.json.dist'
- 'psalm.xml'
- 'tests/**'

push:
paths-ignore:
- 'docs/**'
- 'README.md'
- 'CHANGELOG.md'
- '.gitignore'
- '.gitattributes'
- 'infection.json.dist'
- 'psalm.xml'
- 'tests/**'

name: bechmark

jobs:
phpbench:
uses: yiisoft/actions/.github/workflows/phpbench.yml@master
with:
os: >-
['ubuntu-latest', 'windows-latest']
php: >-
['8.1']
89 changes: 34 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
<p align="center">
<a href="https://github.com/yiisoft" target="_blank">
<img src="https://avatars0.githubusercontent.com/u/993323" height="100px">
<img src="https://yiisoft.github.io/docs/images/yii_logo.svg" height="100px" alt="Yii">
</a>
<h1 align="center">Yii Queue Extension</h1>
<h1 align="center">Yii Queue</h1>
<br>
</p>

An extension for running tasks asynchronously via queues.

Documentation is at [docs/guide/README.md](docs/guide/README.md).

[![Latest Stable Version](https://poser.pugx.org/yiisoft/queue/v/stable.svg)](https://packagist.org/packages/yiisoft/queue)
[![Total Downloads](https://poser.pugx.org/yiisoft/queue/downloads.svg)](https://packagist.org/packages/yiisoft/queue)
[![Build status](https://github.com/yiisoft/queue/workflows/build/badge.svg)](https://github.com/yiisoft/queue/actions)
Expand All @@ -18,29 +14,26 @@ Documentation is at [docs/guide/README.md](docs/guide/README.md).
[![static analysis](https://github.com/yiisoft/queue/workflows/static%20analysis/badge.svg)](https://github.com/yiisoft/queue/actions?query=workflow%3A%22static+analysis%22)
[![type-coverage](https://shepherd.dev/github/yiisoft/queue/coverage.svg)](https://shepherd.dev/github/yiisoft/queue)

## Installation
An extension for running tasks asynchronously via queues.

The preferred way to install this extension is through [composer](http://getcomposer.org/download/).
## Requirements

Either run
- PHP 8.1 or higher.

```shell
composer require yiisoft/queue
```
## Installation

or add
The package could be installed with [Composer](https://getcomposer.org):

```shell
composer require yiisoft/queue
```
"yiisoft/queue": "~3.0"
```

to the `require` section of your `composer.json` file.

## Ready for yiisoft/config

If you are using [yiisoft/config](https://github.com/yiisoft/config), you'll find out this package has some defaults
in the [`common`](config/di.php) and [`params`](config/params.php) configurations saving your time. Things you should
change to start working with the queue:

- Optionally: define default `\Yiisoft\Queue\Adapter\AdapterInterface` implementation.
- And/or define channel-specific `AdapterInterface` implementations in the `channel-definitions` params key to be used
with the [queue factory](#different-queue-channels).
Expand All @@ -52,7 +45,7 @@ change to start working with the queue:
If you have experience with `yiisoft/yii2-queue`, you will find out that this package is similar.
Though, there are some key differences which are described in the "[migrating from yii2-queue](docs/guide/migrating-from-yii2-queue.md)" article.

## Basic Usage
## General usage

Each queue task consists of two parts:

Expand Down Expand Up @@ -204,7 +197,7 @@ See the documentation for more details about adapter specific console commands a

The component also has the ability to track the status of a job which was pushed into queue.

For more details see [the guide](docs/guide/README.md).
For more details see [the guide](docs/guide/en/README.md).

## Middleware pipelines

Expand All @@ -214,6 +207,7 @@ twice for a queue message. That means you can add extra functionality on message
of the two classes: `PushMiddlewareDispatcher` and `ConsumeMiddlewareDispatcher` respectively.

You can use any of these formats to define a middleware:

- A ready-to-use middleware object: `new FooMiddleware()`. It must implement `MiddlewarePushInterface`,
`MiddlewareConsumeInterface` or `MiddlewareFailureInterface` depending on the place you use it.
- An array in the format of [yiisoft/definitions](https://github.com/yiisoft/definitions).
Expand All @@ -224,6 +218,7 @@ You can use any of these formats to define a middleware:

Middleware will be executed forwards in the same order they are defined. If you define it like the following:
`[$middleware1, $midleware2]`, the execution will look like this:

```mermaid
graph LR
StartPush((Start)) --> PushMiddleware1[$middleware1] --> PushMiddleware2[$middleware2] --> Push(Push to a queue)
Expand All @@ -237,7 +232,8 @@ graph LR
```

### Push pipeline
When you push a message, you can use middlewares to modify both message and queue adapter.

When you push a message, you can use middlewares to modify both message and queue adapter.
With message modification you can add extra data, obfuscate data, collect metrics, etc.
With queue adapter modification you can redirect message to another queue, delay message consuming, and so on.

Expand All @@ -255,17 +251,17 @@ in the `PushRequest` object. You will get a `AdapterNotConfiguredException`, if
You have three places to define push middlewares:

1. `PushMiddlewareDispatcher`. You can pass it either to the constructor, or to the `withMiddlewares()` method, which
creates a completely new dispatcher object with only those middlewares, which are passed as arguments.
If you use [yiisoft/config](yiisoft/config), you can add middleware to the `middlewares-push` key of the
creates a completely new dispatcher object with only those middlewares, which are passed as arguments.
If you use [yiisoft/config](yiisoft/config), you can add middleware to the `middlewares-push` key of the
`yiisoft/queue` array in the `params`.
2. Pass middlewares to either `Queue::withMiddlewares()` or `Queue::withMiddlewaresAdded()` methods. The difference is
that the former will completely replace an existing middleware stack, while the latter will add passed middlewares to
the end of the existing stack. These middlewares will be executed after the common ones, passed directly to the
`PushMiddlewareDispatcher`. It's useful when defining a queue channel. Both methods return a new instance of the `Queue`
2. Pass middlewares to either `Queue::withMiddlewares()` or `Queue::withMiddlewaresAdded()` methods. The difference is
that the former will completely replace an existing middleware stack, while the latter will add passed middlewares to
the end of the existing stack. These middlewares will be executed after the common ones, passed directly to the
`PushMiddlewareDispatcher`. It's useful when defining a queue channel. Both methods return a new instance of the `Queue`
class.
3. Put middlewares into the `Queue::push()` method like this: `$queue->push($message, ...$middlewares)`. These
middlewares have the lowest priority and will be executed after those which are in the `PushMiddlewareDispatcher` and
the ones passed to the `Queue::withMiddlewares()` and `Queue::withMiddlewaresAdded()` and only for the message passed
middlewares have the lowest priority and will be executed after those which are in the `PushMiddlewareDispatcher` and
the ones passed to the `Queue::withMiddlewares()` and `Queue::withMiddlewaresAdded()` and only for the message passed
along with them.

### Consume pipeline
Expand All @@ -275,38 +271,28 @@ You can set a middleware pipeline for a message when it will be consumed from a
### Error handling pipeline

Often when some job is failing, we want to retry its execution a couple more times or redirect it to another queue channel. This can be done in `yiisoft/queue` with Failure middleware pipeline. They are triggered each time message processing via the Consume middleware pipeline is interrupted with any `Throwable`. The key differences from the previous two pipelines:

- You should set up the middleware pipeline separately for each queue channel. That means, the format should be `['channel-name' => [FooMiddleware::class]]` instead of `[FooMiddleware::class]`, like for the other two pipelines. There is also a default key, which will be used for those channels without their own one: `FailureMiddlewareDispatcher::DEFAULT_PIPELINE`.
- The last middleware will throw the exception, which will come with the `FailureHandlingRequest` object. If you don't want the exception to be thrown, your middlewares should `return` a request without calling `$handler->handleFailure()`.

You can declare error handling middleware pipeline in the `FailureMiddlewareDispatcher`, either in the constructor, or in the `withMiddlewares()` method. If you use [yiisoft/config](yiisoft/config), you can add middleware to the `middlewares-fail` key of the `yiisoft/queue` array in the `params`.

See [error handling docs](docs/guide/error-handling.md) for details.

## Extra
## Documentation

### Unit testing
- [Guide](docs/guide/en/README.md)
- [Internals](docs/internals.md)

The package is tested with [PHPUnit](https://phpunit.de/). To run tests:
If you need help or have a question, the [Yii Forum](https://forum.yiiframework.com/c/yii-3-0/63) is a good place for that.
You may also check out other [Yii Community Resources](https://www.yiiframework.com/community).

```shell
./vendor/bin/phpunit
```

### Mutation testing

The package tests are checked with [Infection](https://infection.github.io/) mutation framework. To run it:

```shell
./vendor/bin/infection
```

### Static analysis
## License

The code is statically analyzed with [Psalm](https://psalm.dev/). To run static analysis:
The Yii Queue is free software. It is released under the terms of the BSD License.
Please see [`LICENSE`](./LICENSE.md) for more information.

```shell
./vendor/bin/psalm
```
Maintained by [Yii Software](https://www.yiiframework.com/).

### Support the project

Expand All @@ -319,10 +305,3 @@ The code is statically analyzed with [Psalm](https://psalm.dev/). To run static
[![Telegram](https://img.shields.io/badge/telegram-join-1DA1F2?style=flat&logo=telegram)](https://t.me/yii3en)
[![Facebook](https://img.shields.io/badge/facebook-join-1DA1F2?style=flat&logo=facebook&logoColor=ffffff)](https://www.facebook.com/groups/yiitalk)
[![Slack](https://img.shields.io/badge/slack-join-1DA1F2?style=flat&logo=slack)](https://yiiframework.com/go/slack)

## License

The Yii Queue Extension is free software. It is released under the terms of the BSD License.
Please see [`LICENSE`](./LICENSE.md) for more information.

Maintained by [Yii Software](https://www.yiiframework.com/).
6 changes: 0 additions & 6 deletions UPGRADE.md

This file was deleted.

17 changes: 14 additions & 3 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,22 @@
"license": "BSD-3-Clause",
"support": {
"issues": "https://github.com/yiisoft/queue/issues?state=open",
"source": "https://github.com/yiisoft/queue",
"forum": "https://www.yiiframework.com/forum/",
"wiki": "https://www.yiiframework.com/wiki/",
"irc": "ircs://irc.libera.chat:6697/yii",
"chat": "https://t.me/yii3en",
"source": "https://github.com/yiisoft/queue"
"chat": "https://t.me/yii3en"
},
"funding": [
{
"type": "opencollective",
"url": "https://opencollective.com/yiisoft"
},
{
"type": "github",
"url": "https://github.com/sponsors/yiisoft"
}
],
"minimum-stability": "dev",
"prefer-stable": true,
"require": {
Expand All @@ -36,8 +46,9 @@
},
"require-dev": {
"maglnet/composer-require-checker": "^4.7",
"phpbench/phpbench": "^1.3",
"phpunit/phpunit": "^10.5",
"rector/rector": "^0.19.0",
"rector/rector": "^1.0.0",
"roave/infection-static-analysis-plugin": "^1.34",
"spatie/phpunit-watcher": "^1.23",
"vimeo/psalm": "^5.20",
Expand Down
7 changes: 5 additions & 2 deletions config/di.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
use Yiisoft\Queue\Cli\SimpleLoop;
use Yiisoft\Queue\Command\ListenAllCommand;
use Yiisoft\Queue\Command\RunCommand;
use Yiisoft\Queue\Message\JsonMessageSerializer;
use Yiisoft\Queue\Message\MessageSerializerInterface;
use Yiisoft\Queue\Middleware\Consume\ConsumeMiddlewareDispatcher;
use Yiisoft\Queue\Middleware\Consume\MiddlewareFactoryConsume;
use Yiisoft\Queue\Middleware\Consume\MiddlewareFactoryConsumeInterface;
Expand Down Expand Up @@ -54,14 +56,15 @@
FailureMiddlewareDispatcher::class => [
'__construct()' => ['middlewareDefinitions' => $params['yiisoft/queue']['middlewares-fail']],
],
MessageSerializerInterface::class => JsonMessageSerializer::class,
RunCommand::class => [
'__construct()' => [
'channels' => array_keys($params['yiisoft/yii-queue']['channel-definitions']),
'channels' => array_keys($params['yiisoft/queue']['channel-definitions']),
],
],
ListenAllCommand::class => [
'__construct()' => [
'channels' => array_keys($params['yiisoft/yii-queue']['channel-definitions']),
'channels' => array_keys($params['yiisoft/queue']['channel-definitions']),
],
],
];
11 changes: 0 additions & 11 deletions docs/guide/README.md

This file was deleted.

5 changes: 0 additions & 5 deletions docs/guide/adapter-list.md

This file was deleted.

11 changes: 11 additions & 0 deletions docs/guide/en/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Yii Queue

An extension for running tasks asynchronously via queues.

## Guides and concept explanations

- [Usage basics](usage.md)
- [Migrating from `yii2-queue`](migrating-from-yii2-queue.md)
- [Errors and retryable jobs](error-handling.md)
- [Workers](worker.md)
- [Adapter list](adapter-list.md)
14 changes: 14 additions & 0 deletions docs/guide/en/adapter-list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Queue Adapters
-------------

* [Synchronous](adapter-sync.md) - adapter for development and test environments
* [AMQP](https://github.com/yiisoft/queue-amqp) - adapter over AMQP protocol via [amqplib](https://github.com/php-amqplib/php-amqplib)


There are other queue adapters contributed and maintained by the community and available on GitHub, such as:
* [NATS](https://github.com/g41797/queue-nats) - [NATS](https://nats.io/) JetStream adapter
* [Pulsar](https://github.com/g41797/queue-pulsar) - [Apache Pulsar](https://pulsar.apache.org/) adapter
* [SQS](https://github.com/g41797/queue-sqs) - [Amazon SQS](https://aws.amazon.com/sqs/) adapter
* [Kafka](https://github.com/g41797/queue-kafka) - [Apache Kafka](https://kafka.apache.org/) adapter
* [Valkey](https://github.com/g41797/queue-valkey) - [Valkey NoSQL data store ](https://valkey.io/) adapter
* [Beanstalkd](https://github.com/g41797/queue-beanstalkd) - [Beanstalkd - simple, fast work queue](https://beanstalkd.github.io/) adapter
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ It's configured via constructor parameters, too. Here they are:
- `maxAttempts` - Maximum attempts count for this strategy with the given $id before it will give up.
- `delayInitial` - The initial delay that will be applied to a message for the first time. It must be a positive float.
- `delayMaximum` - The maximum delay which can be applied to a single message. Must be above the `delayInitial`.
- `exponent` - Message handling delay will be muliplied by exponent each time it fails.
- `exponent` - Message handling delay will be multiplied by exponent each time it fails.
- `queue` - The strategy will send the message to the given queue when it's not `null`. That means you can use this strategy to push a message not to the same queue channel it came from. When the `queue` parameter is set to `null`, a message will be sent to the same channel it came from.

## How to create a custom Failure Middleware?
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
44 changes: 44 additions & 0 deletions docs/internals.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Internals

## Unit testing

The package is tested with [PHPUnit](https://phpunit.de/). To run tests:

```shell
./vendor/bin/phpunit
```

## Mutation testing

The package tests are checked with [Infection](https://infection.github.io/) mutation framework with
[Infection Static Analysis Plugin](https://github.com/Roave/infection-static-analysis-plugin). To run it:

```shell
./vendor/bin/roave-infection-static-analysis-plugin
```

## Static analysis

The code is statically analyzed with [Psalm](https://psalm.dev/). To run static analysis:

```shell
./vendor/bin/psalm
```

## Code style

Use [Rector](https://github.com/rectorphp/rector) to make codebase follow some specific rules or
use either newest or any specific version of PHP:

```shell
./vendor/bin/rector
```

## Dependencies

This package uses [composer-require-checker](https://github.com/maglnet/ComposerRequireChecker) to check if
all dependencies are correctly defined in `composer.json`. To run the checker, execute the following command:

```shell
./vendor/bin/composer-require-checker
```
8 changes: 8 additions & 0 deletions phpbench.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"$schema":"./vendor/phpbench/phpbench/phpbench.schema.json",
"runner.bootstrap": "vendor/autoload.php",
"runner.path": "tests/Benchmark",
"runner.revs": 100000,
"runner.iterations": 5,
"runner.warmup": 5
}
Loading

0 comments on commit a699200

Please sign in to comment.