Skip to content
This repository has been archived by the owner on Jul 3, 2024. It is now read-only.

Commit

Permalink
Merge branch 'feature/docs'
Browse files Browse the repository at this point in the history
Conflicts:
	README.md
  • Loading branch information
Jurian Sluiman committed Mar 26, 2014
2 parents 7412a8d + c4c6b08 commit f7af6a4
Show file tree
Hide file tree
Showing 8 changed files with 935 additions and 335 deletions.
399 changes: 64 additions & 335 deletions README.md

Large diffs are not rendered by default.

85 changes: 85 additions & 0 deletions docs/1.Introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
Documentation - introduction
============================

SlmQueue is a job queue abstraction layer. It allows you to easily use job queue systems in a Zend Framework 2
application. Thereby it does not enforce you to specifically use one type of job queue. You can write your code and jobs
independent of the underlying system. This enables great flexibility and decoupling of the systems.

Why a queue system?
-------------------

In PHP applications, the request/response cycle is vital to have a fast response time. The earlier you can return a
response, the better it is for your visitors. However, if you need to perform a lot of processing, the response time will
be extended. If these processing tasks contain pieces of code which can be executed at a later moment (say, 5 minutes
later) they are an excellent candidate for a job queue system.

A job queue system allows you to perform some work in the background, independent from the request/response cycle in your
MVC application. If a client visits a page, the controller will tell the queue to perform a job at a later moment. Next,
the controller can return a response very fast, without the waiting on this job. Some examples of a job are:

1. Send an email
2. Create a PDF file
3. Connect to a third party server over HTTP

A typical sketch of a job queue system is the following:

```
Request __________ _______ ________
------> | | | | | |
| Server | ---> | Queue | -----> | Worker |
Response | | | | ^--| | |
<------ |_________| |_______| |________|
```

The server sends jobs to the queue. There is a worker which waits until a job arrives. If it got one, it executes the
job and waits again until the next job arrives. The worker is in most cases a long running process, processing jobs
one by one. If the workload is too much for one worker, you can let two workers process the same queue.

A queue abstraction layer
-------------------------

SlmQueue is called a job queue *abstraction* layer. This layer sits in between your application and the queue system.
The benefit of such layer, is your application is indepedent from the queue system.

Currently, SlmQueue supports beanstalkd, Amazon SQS and Doctrine. A layer like this makes it possible to use Doctrine
first as your application starts small. If the system grows, you might want to migrate something more mature like
beanstalkd or SQS. Without an abstraction layer, you have to rewrite your complete code to support the new system. With
SlmQueue in place, you simply switch a configuration parameter and all your code still works.

Another typical example are your different application environments. On your local machine, you just run SlmQueue on top
of Doctrine or beanstalkd. With your application hosted at Amazon's EC2, your production environment might use the SQS
adapter (although it is not required to use SQS with EC2 or vice versa). All these different adapters are irrelevant for
your application, making it very flexible to switch, test and try all implementations.

The SlmQueue architecture
-------------------------

SlmQueue consists of three main components: a Job, a Queue and a Worker. Besides these, there is a controller, a CLI
command and several factory classes.

The `Job` class represents a single job. An example from above is sending an email. The job describes this specific task
by wrapping the code to send an email inside a `Job` object. You create your own class (for instance, `EmailJob`) which
has to implement the interface `SlmQueue\Job\JobInterface`. Inside the `execute()` method, all logic is placed to send
this email. If you need to perform various tasks through a job queue, you write for every job a separate class.

The `Queue` class is a direct representation of a queue implementation. All different queues implement the same
`SlmQueue\Queue\QueueInterface`. For every implementation (beanstalkd, SQS or Doctrine, at this moment) there is a class
implementing this interface. The object is the main point of entry to insert jobs in the queue and retrieve them back
from it.

The `Worker` class consumes a `Queue` object and interacts with it to extract jobs from the queue and execute the job.
The worker represents the long running process Every implementation has its own worker, accepting a queue from a certain
type. This way, a worker can handle accordingly to the general flow of jobs from the specific implementation.

Navigation
----------

Next page: [Configuration](2.Configuration.md)

1. [Introduction](1.Introduction.md)
2. [Configuration](2.Configuration.md)
3. [Jobs](3.Jobs.md)
4. [QueueAware](4.QueueAware.md)
5. [Workers](5.Workers.md)
6. [Events](6.Events.md)
7. [Worker management](7.WorkerManagement.md)
132 changes: 132 additions & 0 deletions docs/2.Configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
Documentation - configuration
=============================

For queues and jobs, SlmQueue uses a plugin structure. Both the queue and a job are a plugin in the queue or job
manager. You can pull the queue or job from this manager and base your logic around this structure. In essence, the
plugin structure is similar to Zend Framework 2 controllers or view helpers.

Queues
------

SlmQueue supports at this moment three types of queues: Beanstalkd, Doctrine and Amazon SQS. You can specify this type
by using a factory from one of these systems. A queue is defined in the `queue_manager` key. In the following example,
a queue named "default" is defined.

```php
'slm_queue' => array(
'queue_manager' => array(
'factories' => array(
'default' => 'SlmQueueBeanstalkd\Factory\BeanstalkdQueueFactory'
),
),
),
```

If you get this queue from the queue manager, SlmQueue will configure the instance completely for you and you are ready
to push jobs into the queue. In the following example, access to the queue in your own controller factory is demonstrated.

```php
namespace MyModule\Factory;

use MyModule\Controller\MyController;
use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class MyControllerFactory implements FactoryInterface
{
public function createService(ServiceLocatorInterface $sl)
{
$queueManager = $sl->get('SlmQueue\Queue\QueuePluginManager');
$queue = $queueManager->get('default');

$controller = new MyController($queue);
return $controller;
}
}
```

With a plugin structure, it allows you to create multiple queues. For larger applications, it might be useful to define
different types of queues for different types of tasks. Simple jobs can be pushed into a default queue, but difficult
jobs can be pushed into another queue. The workers of that second queue will run on a larger separate machine, so they
are processed more efficiently.

Jobs
----

Jobs are defined in a similar fashion. The `job_manager` key is available for this configuration.

```php
'slm_queue' => array(
'job_manager' => array(
'factories' => array(
'MyModule\Job\SendEmailJob' => 'MyModule\Factory\SendEmailJobFactory',
),
'invokables' => array(
'MyModule\Job\PrintHelloWorldJob' => 'MyModule\Job\PrintHelloWorldJob',
),
),
),
```

It is not required to use factories for all jobs. If a job does not need any dependency, you can define the job as an
invokable. You can get the job via the job plugin manager, as shown below.

```php
namespace MyModule\Controller;

use SlmQueue\Queue\QueueInterface;
use SlmQueue\Job\JobPluginManager;
use Zend\Mvc\Controller\AbstractActionController;

class MyController extends AbstractActionController
{
protected $queue;
protected $jobManager;

public function __construct(QueueInterface $queue, JobPluginManager $jobManager)
{
$this->queue = $queue;
$this->jobManager = $jobManager;
}

public function fooAction()
{
// Do some work

$job = $this->jobManager->get('MyModule\Job\PrintHelloWorldJob');
$this->queue->push($job);
}
}
```

Jobs without job plugin manager
-------------------------------

There is no need to define all the jobs you use in the job plugin manager. If the job can be invoked (no factory is
defined) you can use the FQCN for the job in the job plugin manager to access it as well.

```php
$job = $jobPluginManager->get('MyModule\Job\PrintHelloWorldJob');
```

Alternatively, in many cases the job manager is not even required to instantiate the job. You can push jobs into queues
where you get the job by using the `new` keyword:

```php
$job = new PrintHelloWorldJob;
$queue->push($job);
```

Navigation
----------

Previous page: [Introduction](1.Introduction.md)
Next page: [Jobs](3.Jobs.md)

1. [Introduction](1.Introduction.md)
2. [Configuration](2.Configuration.md)
3. [Jobs](3.Jobs.md)
4. [QueueAware](4.QueueAware.md)
5. [Workers](5.Workers.md)
6. [Events](6.Events.md)
7. [Worker management](7.WorkerManagement.md)
165 changes: 165 additions & 0 deletions docs/3.Jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
Documentation - Jobs
====================

Every job must implement the interface `SlmQueue\Job\JobInterface`. For ease of use, an abstract class
`SlmQueue\Job\AbstractJob` is provided to implement most of the interface methods. The method, `execute()`, must be
implemented in userland code.

An example job would look like the following:

```php
namespace MyModule\Job;

use SlmQueue\Job\AbstractJob;

class PrintHelloWorldJob extends AbstractJob
{
public function execute()
{
echo 'Hello World';
}
}
```

Job payload
-----------
The job often needs some data to work with, commonly called a payload. For an email job, the contents of the email (to
address, subject and message) are the payload. If you generate an invoice based on an order in your database, you set
the id of this order as the payload.

Internally, a `SlmQueue\Job\AbstractJob` extends the `Zend\Stdlib\Message` class which allows it to set and get content.
These methods are used to set the payload upon pushing and retrieve the payload when the job is executed.

To set the payload in a controller, below is an example:

```php
namespace MyModule\Controller;

use Zend\Mvc\Controller\AbstractActionController;

class MyController extends AbstractActionController
{
public function fooAction()
{
// Do some work

$job = $this->jobManager->get('MyModule\Job\SendEmailJob');
$job->setContent(array(
'to' => 'bob@acme.com',
'subject' => 'Registration completed',
'message' => 'Hi bob, you just registered for our website! Welcome!'
));

$this->queue->push($job);
}
}
```

Then you can fetch this payload from the job itself:

```php
namespace MyModule\Job;

use SlmQueue\Job\AbstractJob;

class SendEmailJob extends AbstractJob
{
public function execute()
{
$payload = $this->getContent();
$to = $payload['to'];
$subject = $payload['subject'];
$message = $payload['message'];

// Do something with $to, $subjet, $message
}
}
```

Job payload are a flexible array structure and will be automatically serialized by SlmQueue. This means you can have
values like a `DateTime` object as payload for jobs which will be serialized and deserialized in the background. However,
this also give you restrictions than payload data *must* be serializable. Doctrine 2 entities, references to
`Zend\Mvc\Application` or other unserializable instances should not be set as payload.

Job dependencies
----------------

Because of the configuration via a job plugin manager, you can inject dependencies you need into the constructor of the
job class. This will require you to define a job factory for that job as well. Assume here we send an email in a job and
this job requires the email transport class as a dependency.

```php
namespace MyModule\Job;

use SlmQueue\Job\AbstractJob;
use Zend\Mail\Transport\TransportInterface;
use Zend\Mail\Message;

class SendEmailJob extends AbstractJob
{
protected $transport;

public function __construct(TransportInterface $transport)
{
$this->transport = $transport;
}

public function execute()
{
$message = new Message;
$payload = $this->getContent();

$message->setTo($payload['to']);
$message->setSubject($payload['subject']);
$message->setContent($payload['content']);

$this->transport->send($message);
}
}
```

To inject the email transport instance, a factory must be created to instantiate the job:

```php
namespace MyModule\Factory;

use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

use MyModule\Job\SendEmailJob;

class SendEmailJobFactory implements FactoryInterface
{
public function createService(ServiceLocatorInterface $sl)
{
$transport = $sl->getServiceLocator()->get('my-transport-service');

$job = new SendEmailJob($transport);
return $job;
}
}
```

The last step is to register this factory for the above job:

```php
'slm_queue' => array(
'factories' => array(
'MyModule\Job\PrintHelloWorldJob' => 'MyModule\Factory\SendEmailJobFactory',
),
)
```

Navigation
----------

Previous page: [Configuration](2.Configuration.md)
Next page: [Queue Aware](4.QueueAware.md)

1. [Introduction](1.Introduction.md)
2. [Configuration](2.Configuration.md)
3. [Jobs](3.Jobs.md)
4. [QueueAware](4.QueueAware.md)
5. [Workers](5.Workers.md)
6. [Events](6.Events.md)
7. [Worker management](7.WorkerManagement.md)
Loading

0 comments on commit f7af6a4

Please sign in to comment.