Castor, a journey across the sea of task runners

At JoliCode we were early adopters of Docker (since late 2013). When we discovered it, we strongly believed that this tool would fit perfectly in our dev environment and allow teams to work under the same clean and reproducible environment.

However, like each new piece of software, learning it requires time and motivation, and forcing it on each developer is always a bad idea as it can bring resignation and exhaustion. Also, even if we were convinced that the idea around Docker was really nice, we were not sure that its API and even the tool would stay the same over time.

What we needed to do then, was to hide the implementation of using this tool behind a functional interface that our team would use. When developing a project, we have a lot of needs as developer:

• Launching / stopping the project environment (we can work on different projects);
• Clearing the cache of the various components;
• Migrating database schema;
• Injecting fake data;
• Launching tests;
• Running quality tools;
• …

Those needs don’t change. However what changes across time is the tools we use.

That’s why we created Docker Starter, a boilerplate that we use internally when we create a new project. We have been using this tool for almost 10 years and we never regretted it.

Section intitulée a-task-runnerA task runner

To create this interface between our needs and the implementation, we need a task runner. Make, with the traditional Makefile, was our first thought, but as we implemented more needs to our project, it quickly became a burden with the difficulty to split it in different files (no more correct listing), requiring to be an expert of bash commands.

When looking at other tools, Fabric became our de facto choice to implement this, we were already using it to deploy some of our small projects with SSH and we really loved the DX around it: simple and effective.

Section intitulée solving-a-problem-by-creating-a-new-problemSolving a problem by creating a new problem

At first it worked like a charm, all our team was using this layer, and even other people on the team who didn’t know Python and Fabric began to add tasks.

However in the long term we began to face difficulties with the update to Python 3 and a lot of dependencies not being compatible with it, different default Python versions on each distribution. Even when using tools like pipenv, we still have a lot of problems on first install or when upgrading. We migrated to Invoke to better support Python 3. But we were still facing random issues with it.

Let’s face it, today, even if Python is really a great language, its ecosystem is a mess. It may have been easier if we were Python developers at the core, so everyone would be able to debug those problems.

But we are PHP developers and having to deal with that on a daily basis is not a cake walk. We are so lucky to have Composer in the PHP world! 😍

Section intitulée a-php-task-runnerA PHP task runner

Since the beginning of this journey, I always wanted to use a PHP tool. Like said before, we are PHP developers. The first criterion for selecting a tool should always be the people who use or create it.

That’s the part where I made a big mistake. In the process of creating such a tool, I thought, wrongly, that it would be better to directly call the Docker API instead of the CLI. I also wanted a SSH connection integrated into the tool and having the possibility to run all of this in an async way to parallelize tasks.

In order to achieve that, I created docker-php and a lot of tools around it and implemented a SSH client in pure PHP on the amphp project. All those libraries were consuming so much time that I didn’t have the bandwidth to create this task runner.

In the meantime, new tools were created by the PHP community, like the amazing robo task runner. However, we don’t really like it. The DX to create tasks with it feels cumbersome and in our opinion there is too much boilerplate, especially compared to the Python Invoke library that we use in docker-starter.

We also looked at the laravel envoy, but I never understood why we would have to learn a new DSL when we already have everything necessary in PHP, so this was definitely a no go for us.

Section intitulée castor-a-modern-php-task-runnerCastor, a modern PHP task runner

Based on our experience, and our failures, we began to understand what we wanted and what we didn’t.

That’s why we built Castor, a PHP task runner with a lot of love on the DX part.

Here is an example on how to create a task :

#[AsTask]
function foo(string $param) {
     run("echo Hello ${param}");
}

Then you will be able to execute this task with the following command:

$ castor foo "world"
Hello world

Section intitulée real-life-usage-examplesReal life usage examples

In a project you may have a backend in PHP which delivers an API to a SPA application. In order to ensure the quality of this project you will use various tools :

#[AsTask]
function test_php() {
     run("./vendor/bin/phpunit", path: './backend');
}

#[AsTask]
function test_js() {
     run("yarn run jest", path: './app');
}

The command doesn’t mention the tool used. Then if we want to change it (using pest instead of phpunit ? using prettier instead of eslint ? adding a new qa tool for php ?, …), other developers don’t need to know which commands to execute; they will always use the same task.

Those checks are rather common to execute when you want to push your changes to git. We can imagine a situation where we want to run all those tools before the commit :

#[AsTask]
function check_before_commit() {
    parallel(
       fn() => test_php(),
       fn() => test_js(),
    );
}

This would run all previous commands in parallel, and you could add this task to the pre-commit hooks of your project.

We could also build a more complex command with interaction to better help the developer, like how to clean the docker infrastructure of a project :

#[AsTask(description: 'Cleans the infrastructure (remove container, volume, networks)')]
function destroy(
    #[AsOption(description: 'Force the destruction without confirmation', shortcut: 'f')]
    bool $force = false,
): void {
    if (!$force) {
        io()->warning('This will permanently remove all containers, volumes, networks... created for this project.');
        io()->note('You can use the --force option to avoid this confirmation.');
        if (!io()->confirm('Are you sure?', false)) {
            io()->comment('Aborted.');

            return;
        }
    }

    run(['docker', 'composer', 'down', '--remove-orphans', '--volumes', '--rmi=local']);

    $files = finder()
        ->in(variable('root_dir') . '/infrastructure/docker/services/router/etc/ssl/certs/')
        ->name('*.pem')
        ->files()
    ;
    fs()->remove($files);
}

It makes use of Symfony’s Console helpers to nicely output texts, ask interactively for confirmation but also Symfony’s Finder and Filesystem to find and remove files, etc.

Section intitulée using-it-in-the-ciUsing it in the CI

Another advantage is that you can use the same tasks in your dev environment and in your CI. A change would then impact both environments without having to know how the CI or dev environment works.

As we use GitHub Actions for many projects, we’ve contributed to shivammathur/setup-php to provide castor as pre-installed binary if you need to use it in your GitHub Actions workflows. It will be available in the next release of the Github Action.

Section intitulée conclusionConclusion

We are really fans of the “Less is more” concept, Castor allows you to create powerful tasks using a minimalist API without having to learn a new language or DSL if you are a PHP developer.

Under the hood, it uses powerful and robust Symfony components, like Console and Process. By using them, we have access to all the features of those libraries, like command listing, progress bar output, validation, and a lot more.

We also provide a lot of useful functions, like watching a file or directory for changes, running concurrent programs or sending desktop notifications.

Check out our documentation if you want to learn more about castor.