Translation workflow with Symfony2

I have an issue with Symfony2 translations. No in fact, let’s say I have an issue with app translation in general. It’s painful to write, to sync, to contextualize and to maintain.

I’ve built many web applications with Symfony (1 & 2) and have yet to find the perfect workflow to please the developers, the product owner and the translators all at the same time.

This article is about the workflow I’m currently using and the issues I face in Symfony2, this is opinionated but I will explain why I’m using it as default from now on.

Section intitulée the-translation-issue-in-symfony2The translation issue in Symfony2

The Translator component is like a gettext, you give it messages and you can strtr the translated strings back, with some more options like plurals, domains and locales fallback. It works perfectly.

But how do you populate those messages catalogs? There is three solutions I’m aware of:

• create translations files by hand;
• use the translation:update command, built-in FrameworkBundle (but may disappear someday);
• use the translation:extract command from JMSTranslationBundle.

The first one is obviously the worst: every time you write some text in the app, you have to add the string in an XML, Po or whatever file. As you are a team with multiple developers all working at the same time on the same application, this may lead to file conflict. Yet you are also gonna miss some, and your product owner is not going to be happy. Your pull request is not going to be accepted and you will waste some time fixing everything, just for a stupid piece of string.

Let’s talk about the built-in translation:update command. Not even mentioned in the official documentation, TranslationUpdateCommand run a service called translation.extractor, only capable of reading simple trans calls in Twig templates by default. You can add your own extractors but good luck with that (parsing files is hard).

Looking only in templates for strings to translate is wrong – with Symfony2 they can come from a hell lot of places. That’s part of what JMSTranslationBundle tries to solve with extractors for:

• all calls to the trans, or transChoice method within PHP files;
• all classes implementing the TranslationContainerInterface;
• all form labels that are defined as options to the ->add() method of the FormBuilder;
• messages declared in validation constraints.

This is already much more usable. This Bundle (sadly not updated since 2013 and under Apache License) is what we have best so far to extract translations from static files. But is that enough? I don’t think so.

Sometimes, we use variables (const values) as key. Other time, we would like to translate Exception messages (that’s what the Security Component do). We could also need to translate some form options, dynamically generated keys, etc. Parsing is not, and will never be, the full answer. You will still miss some keys and be mad, and not everyone like or can work with scalar string only.

Section intitulée working-with-translation-placeholderWorking with translation placeholder

It’s a common need to have placeholders in translation string:

Hello my name is %name%!

It’s also a good practice to use abstract keys instead of plain string:

acme.homepage.greeting

There is lots of advantages in using abstract keys, but then the translator has no way to see there is a %name% placeholder available. And none of the static extractor mentioned earlier will help with that. The only solution at the moment is a special JMSTranslationBundle annotation called Desc where you write yourself hint for the translator… That’s not something I want to do because I’m a developer and I’m lazy.

Section intitulée what-i-m-doing-to-improve-the-workflowWhat I’m doing to improve the workflow

Symfony 2.7 came with a new Translation Profiler thanks to Abdellatif Ait boudad. It’s a simple TranslationDataCollector storing all the calls to the translator. It means this collector is going to have all the translation keys, at run-time!

We need to do something with those keys, like saving them at the push of a button!

The TranslationDataCollector also miss something quite important: the placeholders. So we added them, simple as that.

We have a custom translation panel in our profiler, with a checkbox for each missing message. When the submit button is pressed, we call a custom Controller to save the new strings wherever we want. It’s only 2 new files and a bit of configuration.

This custom controller could save translation files, or call an external API, it does not matter. You can check the code on this gist, with the “writing part” being your own responsibility for now.

Our testing and staging instance are running with the dev environment, so when testing a feature, it’s easy for everyone (developers, product owner, QA peoples) to see the red flag and click a button to add the missing keys in the system.

In our case, we send the translations keys to an online translation service called Loco via their API. The translators can see there is new un-translated keys, and that’s totally asynchronous, it does not involve any developers for them to work and update translations.

We also have a command to fetch the translations from Loco and install them in the Symfony instance. So every-time we deploy something, new translations are downloaded (of course, we can also update the translation whenever we want, without doing a new release). Again, no developer needed for this. This is the end of tickets about wording and small typo wasting the team time.

It’s been a month that we are already working with this system and it works like a charm.

We ship new features without having to worry about the translations, all we have to do is write the appropriate keys, use transchoice when appropriate, inject the placeholders: a developer’s job. While testing, we can send the missing keys to our translation platform but we don’t have to.

Then the product owner can test the feature. He may discover un-translated keys and can save them himself.

Then the translation team can contribute the new translations, whenever they want.

And finally, we can deploy to production, the translations are downloaded and everyone is happy ☺!

Section intitulée my-recommendationsMy recommendations

I’m pleased about my translations workflow today because of those practices, so here is a list:

• always use abstract keys:
  • developers will not have to write text anymore (no more lipsum, no more typo…);
  • the key will (must) never change over time;
  • the key give translator a context.
• enforce a key writing standard: do not let developers go crazy with the key names, try to follow the same pattern everywhere, like vendor.controller.action.place.the_actual_key for example;
• do not commit the translation file. This one is not going to be enjoyed by everyone but I think that updating some text in production should not require to tag a new release. And who never had a translation file conflict with git?
• use an external translation service. Don’t talk to me about poedit again, online services are way better, they offer real tools for translators and some of them are free. They all have their pros and cons. I’m using Loco at the moment, but you should also have a look at:
  • Openl10n by Matthieu Moquet ❤
  • Transifex;
  • Crowdin;
  • OneSky.
• disable translator in the test environment: I don’t want my functional tests to broke if a translation is changed. So I have this in my config_test.yml file:

    framework:
        translator:
            enabled: false
  

It’s much more clean to me to have asserts on the keys in my tests than the translated strings… Another good idea could be to have some dedicated translation file only for the test environment, because we may want to actually test translation results.

Section intitulée what-to-do-nextWhat to do next?

The end of translation pain was just in front of me, this new TranslationDataCollector opened the door to a lot of cool stuffs. It could be used for a lot of things, like integration tests (do not deploy if there is missing translations) for example.

There still are functional issues, like the fact that you can’t have two different validation messages for the same Assert (one for the backend, one for the frontend), or the fact that translation catalog names are often hard-coded and forced on you (to fully translate a login page, you may have to dive into 3 different catalogs…).

I may propose this as a Symfony feature – or at least a bundle – in a near future, but I’m looking for feedback and comments. What tools do you use? How to do manage translations? What do you think about this workflow? Let’s talk.