Master task scheduling with Symfony Scheduler

Section intitulée introductionIntroduction

Nowadays, using a crontab for our recurring tasks is quite common, but not very practical because it’s completely disconnected from our application. The Scheduler component is an excellent alternative. It was introduced in 6.3 by Fabien Potencier during his opening keynote at SymfonyLive Paris 2023 (french publication). The component is now considered stable since the release of Symfony 6.4. Let’s take a look at how to use it!

Section intitulée installationInstallation

Let’s install the component:

composer require symfony/messenger symfony/scheduler

As all the component’s functionalities are based on Messenger, we need to install it too.

Section intitulée the-first-taskThe first task

Let’s create a first message to schedule:

// src/Message/Foo.php
readonly final class Foo {}

// src/Handler/FooHandler.php
#[AsMessageHandler]
readonly final class FooHandler
{
	public function __invoke(Foo $foo): void
	{
    	sleep(5);
	}
}

In the same way as a Message dispatched in Messenger, here we’re dispatching a Message, which Scheduler will process in a similar way to Messenger, except that processing will be triggered on a time basis.

In addition to the Message/Handler pair, we need to define a Schedule:

#[AsSchedule(name: 'default')]
class Scheduler implements ScheduleProviderInterface
{
	public function getSchedule(): Schedule
	{
    	return (new Schedule())->add(
        	RecurringMessage::every('2 days', new Foo())
    	);
	}
}

This will indicate to our application that we have a “default” schedule containing a message launched every two days. Here, the frequency is simple, but it’s entirely possible to configure it more finely:

RecurringMessage::every('1 second', $msg)
RecurringMessage::every('15 day', $msg)

# relative format
RecurringMessage::every('next friday', $msg)
RecurringMessage::every('first sunday of next month', $msg)

# run at a very specific time every day
RecurringMessage::every('1 day', $msg, from: '14:42')
# you can pass full date/time objects too
RecurringMessage::every('1 day', $msg,
	from: new \DateTimeImmutable('14:42', new \DateTimeZone('Europe/Paris'))
)

# define the end of the handling too
RecurringMessage::every('1 day', $msg, until: '2023-09-21')

# you can even use cron expressions
RecurringMessage::cron('42 14 * * 2', $msg) // every Tuesday at 14:42
RecurringMessage::cron('#midnight', $msg)
RecurringMessage::cron('#weekly', $msg)

Here you can see relative formats; more information on this format in PHP can be found on the documentation page.

For cron syntaxes, you’ll need to install a third-party library that allows Scheduler to interpret them:

composer require dragonmantank/cron-expression

Once you’ve defined your Schedule, just as you would for a Messenger transport, you’ll need a worker to listen in on the Schedule as follows:

bin/console messenger:consume -v scheduler_default

The scheduler_ prefix is the generic name of the transport for all Schedules, to which we add the name of the Schedule created.

Section intitulée collisionsCollisions

The more tasks you have, the more likely you are to have tasks arriving at the same time. But if a collision occurs, how will Scheduler handle it? Let’s imagine the following case:

(new Schedule())->add(
	RecurringMessage::every('2 days', new Foo()),
	RecurringMessage::every('3 days', new Foo())
);

Every 6 days, the two messages will collide:

If you only have one worker, then it will take the first task configured in the Schedule and, once the first task is finished, it will execute the second task. In other words, the execution time of the 2nd task depends on the execution time of the 1st.

We often want our tasks to be executed at a precise time. Here are two solutions to this problem:

• Good practice would be to specify the date and time of execution of our task using the from parameter: RecurringMessage::every('1 day', $msg, from: '14:42') for one of the messages and set it to 15:42 for the other task (also possible with cron syntax);
• Have several workers running: if you have 2 workers, then it can handle 2 tasks at the same time!

Section intitulée multiple-workersMultiple workers?

But today, if we run 2 workers, our task will be executed twice!

Scheduler provides the tools to avoid this! Let’s update our Schedule a little:

#[AsSchedule(name: 'default')]
class Scheduler implements ScheduleProviderInterface
{
	public function __construct(
    	private readonly CacheInterface $cache,
    	private readonly LockFactory $lockFactory,
	) {
	}

	public function getSchedule(): Schedule
	{
    	return (new Schedule())
        	->add(RecurringMessage::every('2 days', new Foo(), from: '04:05'))
        	->add(RecurringMessage::cron('15 4 */3 * *', new Foo()))
        	->stateful($this->cache)
        	->lock($this->lockFactory->createLock('scheduler-default'))
    	;
	}
}

We retrieve a service to manage its cache and create locks (remember to install symfony/lock beforehand). Then we indicate that our schedule can now benefit from a state and has a lock thanks to these new elements.

And that’s it 🎉 now we can have as many workers as we want, they won’t launch the same message several times :)

Section intitulée toolingTooling!

Section intitulée debugging-our-schedulesDebugging our Schedules

A console command has been added since this PR, which lists all the tasks in the Schedules you’ve created!

$ bin/console debug:scheduler

Scheduler
=========

default
-------

 -------------- -------------------------------------------------- ---------------------------------
  Trigger   	 Provider                                  		 Next Run               		
 -------------- -------------------------------------------------- ---------------------------------
  every 2 days   App\Messenger\Foo(O:17:"App\Messenger\Foo":0:{})   Sun, 03 Dec 2023 04:05:00 +0000
  15 4 */3 * *   App\Messenger\Foo(O:17:"App\Messenger\Foo":0:{})   Mon, 04 Dec 2023 04:15:00 +0000
 -------------- -------------------------------------------------- ---------------------------------

In addition to seeing the tasks in your Schedules, you’ll also see the next execution date.

Section intitulée change-the-transport-of-your-tasksChange the transport of your tasks

Sometimes a message can take a long time to process. We can therefore say in our Schedule that our message must be processed by a given transport. For example:

(new Schedule())->add(
	RecurringMessage::cron('15 4 */3 * *', new RedispatchMessage(new Foo(), 'async')))
);

Here, when the message is to be dispatched, the worker will send it to the async transport, which will then process it. This is very useful for heavy tasks, as it frees up the scheduler_default worker to process the next messages.

Section intitulée error-handlingError handling

Scheduler allows you to listen to several events via the EventDispatcher component. There are 3 listenable events: PreRunEvent, PostRunEvent and FailureEvent. The first two will be triggered, respectively, before and after each task executed. The latter will be triggered in the event of a task exception. This can be very useful for efficient error monitoring:

#[AsEventListener(event: FailureEvent::class)]
final class ScheduleListener
{
	public function __invoke(FailureEvent $event): void
	{
    	// triggers email to yourself when your schedules have issues
	}
}

With this code, when a FailureEvent occurs, you can send yourself an email or add logs to better understand the problem.

Section intitulée console-as-schedulerConsole as Scheduler

One of the most interesting features of Scheduler in my opinion: the AsCronTask and AsPeriodicTask attributes! They allow you to transform a console command into a Scheduler task in a very simple way! AsPeriodicTask to define a task via a simple recurrence: 2 days for example, and AsCronTask to do the same thing via a cron expression.

#[AsCommand(name: 'app:foo')]
#[AsPeriodicTask('2 days', schedule: 'default')]
final class FooCommand extends Command
{
	public function execute(InputInterface $input, OutputInterface $output): int
	{
    	// run you command

    	return Command::SUCCESS;
	}
}

And that’s it, the command will be executed in Schedule default every 2 days!

There are often duplicates between console commands and your recurring tasks, so this is the perfect feature to link the two!

Section intitulée conclusionConclusion

The Scheduler component is an essential tool to efficiently integrate recurring tasks into Symfony. Its ease of use, flexibility, cron expression management and seamless integration with console commands make it an essential choice.