Written by Niko Stotz
on 

MPS language design patterns: Multiple generators for same language

If we need more than one generator for the same language, things can become complicated. With help of generation plans, we can sort generators apart cleanly, and let the user decide which ones to engage.

This article follows Sergej Koščejev’s excellent idea on language design patterns.
The example used in this article is available on GitHub. We refer to concrete names of our example in parenthesis.
example structure

Rationale

Let’s assume we have a nice domain-specific language, our users enjoy working with it. However, at some point the fruits of their labor needs to be available outside of MPS — and often times in more than one format. For example, we might want to generate some executable Java code, some HTML documentation, and JSON representation from the same model. Moreover, some users might want only a subset of these outputs.

Each of these targets can easily be implemented as MPS generator. As soon as we tried to handle all outputs in the same generator, we run into issues:

  • We need to copy the input once for each output.

  • Generator rules to different target languages are intermixed.

  • We cannot use $COPY_SRC$ or $COPY_SRCL$ macros any more.

  • It’s hard to control which outputs are actually generated.

To solve this issue, we propose clean separation of our language and generators.

Overview

We design our actual DSL (Entities) as usual; however, the DSL language does not contain a generator.

For each output format, we create a separate language (Entities.gen.baselanguage and Entities.gen.xml), dubbed generator language. A generator language defines only one concept called generator enabler (GenerateXml). Other than that, the generator language contains a generator from our actual DSL to one target language.
In the generator’s is applicable section, we make sure the whole generator gets toggled by presence of a generator enabler.

mapping config is applicable

We create one genplan solution (Entities.genplan). It contains one model with one genplan for each generator language. In our example, this means two models: Entities.genplan.baselanguage and Entities.genplan.xml. One additional model (Entities.genplan.genplan) contains the main generator plan. The main generator plan forks to each of the generator language genplans, i.e. creates a copy of the current input model for each fork and processes them independently.

genplans

Finally, we create a devkit (Entities.devkit). It contains our actual DSL, all generator languages, and the genplan solution. Make sure to attach the main generator plan to the devkit.

In a user model (Entities.sandbox), our user can create DSL concepts as they please (my.special.entities). As long as they don’t create any of the generator enablers, generating the model will yield to no outcome. As soon as our user adds an instance of a generator enabler (Generate BaseLanguage, Generate XML), the corresponding generator will execute on the next model generation.

This design solves all original issues:

  • MPS' genplans take care of copying the input.

  • Each generator language is cleanly separated from other generators, and focuses on one target language.

  • All macros work as if there’s only one generator.

  • We control a generator’s activation at one central place.

Alternative for generator enablers

In real-world applications, users are often confused by generator enablers as root nodes. They prefer to have one place to configure all generation concerns; that might include additional aspects like an output directory.

We can accommodate for that by an additional language:

  1. Create a generator config language, e.g. Entities.gen.

  2. Create an interface IGeneratorEnabler.

  3. Create a rootable concept GeneratorConfig with a list of IGeneratorEnabler as children.

  4. Make each generator enabler implement IGeneratorEnabler. The generator enablers themselves stay in their own generator language.

  5. Adjust the generator’s is applicable sections to look inside the GeneratorConfig for their enablers.

  6. Instead of several root nodes, the user creates one GeneratorConfig in their model, and adjust its contents to their needs.

Both the GeneratorConfig and each generator enabler might define additional members to configure the generation process.

Multiple generators: a checklist

  • Delete generator from actual DSL. (If the actual DSL already contains a generator, create one generator language first, then move the generator’s contents there.)

  • For each target language, create a new language, dubbed generator language.

    • Add generator enabler concept (e.g. GenerateXml): If an instance of this concept is present in a model, this generator will execute.

      Structure

      • Set instance can be root to true.

      • Set alias to e.g. Generate YourTargetLanguage.

      Editor

      Add #alias# editor component.

    • Add an is applicable constraint to the generator language's mapping configuration:

      is applicable:
  (genContext)->boolean {
    genContext.inputModel.roots(GenerateXml).isNotEmpty;
  }

      GenerateXml refers to the previously added generation enabler.

    • Implement the generator for one target language.

  • Create a new solution, dubbed genplan solution.

  • For each generation language, add one model to the genplan solution.
    Add jetbrains.mps.lang.generator.plan to Used Languages.

    • Create a new Plan instance.
      Name it actual DSL name - target language name.

    • Add an ApplyGenerators step.
      Refer to the generator in generation language.

      Entities - XML

apply
Generator Entities.gen.xml.generator

  • Add a model for the main generation plan to the genplan solution.
    Add jetbrains.mps.lang.generator.plan to Used Languages.

  • Create a new Plan instance inside the main generation plan model.
    Name it actual DSL name default.

  • For each generator language, add one fork step to the main generation plan.

    • Add a dependency to the corresponding genplan model. Example: model Entities.genplan.genplan depends on Entities.genplan.baselanguage and Entities.genplan.xml.

    • In the fork step, refer to the corresponding genplan.

      Entities default

fork with Entities - BaseLanguage Entities.genplan.baselanguage
fork with Entities - XML Entities.genplan.xml

  • Create a new devkit.

  • Add the actual DSL, all generator languages, and the genplan solution to the devkit.
    Press Apply button.

  • Assign the main generation plan to the devkit.

    devkit properties

  • In any user model’s Used Languages, replace the actual DSL by the devkit.

About Niko Stotz

I head the Model Driven Engineering team at F1RE, and support our colleagues and customers with language engineering and DSL expertise.

You can contact me at niko@f1re.io.