T4 Tutorial: Generating Code from DSL Models


This post is a part of the series that introduces code generation with Text Templates (also known as T4 Templates) in Visual Studio using C# and Visual Basic; explains how to create reusable templates and combine them in complex code generators. In order to follow examples in this article, you need to have Visual Studio 2008 SP1 Standard Edition or higher, Visual Studio 2008 SDK 1.1, T4 Toolbox and T4 Editor installed on your computer.

Domain-Specific Language Tools

As the name implies, a Domain-Specific Language (DSL) is a programming language dedicated to a particular problem domain. Although we usually associate the term programming language with a textual language, such as C# and Visual Basic, DSLs also take a form of graphical modeling languages, such as UML diagrams, or Excel spreadsheets. Visual Studio SDK includes Domain-Specific Language Tools, which allow you to build DSLs of the graphical variety. Shown below, is a Class Designer built using DSL tools and included as a sample in the Visual Studio SDK.

Class Designer

This sample DSL allows you to model a set of classes in a graphical design environment, similar to a UML diagrams in Visio. The model it creates is stored in XML format.

image

The Class Designer example also includes T4 templates that you can use as a starting point to generate code from this model. These T4 templates rely on a custom directive processor called ClassDiagrams to make the model data available for code generation. The directive processor ensures that metadata assembly is referenced when the template is compiled and generates a special property called ModelRoot which provides access to the metadata loaded from the model file - Sample.classes.

C#

<#
/***************************************************************************
    Copyright (c) Microsoft Corporation. All rights reserved.
    This code is licensed under the Visual Studio SDK license terms.

    THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF
    ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY
    IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR
    PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

***************************************************************************/
#>
<#@ template inherits="ModelingTextTransformation" #>
<#@ output extension=".txt" #>
<#@ ClassDiagrams processor="ClassDiagramsDirectiveProcessor"
  requires="fileName=’Sample.classes’"  #>

Report template

<#= this.ModelRoot.Name #>

<#
  // When you change the DSL Definition, some of the code below may not work.

  foreach (ModelType type in this.ModelRoot.Types)
  {
#>
    <#= type.GetType().Name #> <#= type.Name #>
<#
  }
#>

Visual Basic
<#
REM    Copyright (c) Microsoft Corporation. All rights reserved.
REM    This code is licensed under the Visual Studio SDK license terms.
REM
REM    THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF
REM    ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY
REM    IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR
REM    PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.
#>
<#@ template inherits="ModelingTextTransformation"language="VB" #>
<#@ output extension=".txt" #>
<#@ ClassDiagrams processor="ClassDiagramsDirectiveProcessor"
  requires="fileName=’Sample.classes’"  #>

Report template

<#= Me.ModelRoot.Name #>

<#
  REM When you change the DSL Definition, some of the code below may not work.

  For Each type As ModelType In Me.ModelRoot.Types
#>
    <#= type.GetType().Name #> <#= type.Name #>
<#
  Next
#>

The output produced by these templates is shown below. In a real-world scenario, you would probably generate something more useful, such as C# or Visual Basic class declarations or, perhaps, SQL table scripts.

Report template

T4Toolbox.Examples.OrderEntry

    ModelClass Customer
    ModelClass Address
    ModelClass Product
    ModelClass Supplier
    ModelClass Order

Because the DSL samples in Visual Studio SDK use standard T4 templates, they can only produce a single output file per template. As a result, you are either limited to generating all artifacts in a single large file, or having multiple templates to generate smaller files. Luckily, T4 Toolbox can be used to generate multiple files from DSL models as we will see later in this article.

Domain-Specific Language Implementation

Before we take a closer look at generating code, we will review some of the basics you will need to understand in order to follow discussion of code generation. For complete details, please refer to the Visual Studio SDK and Domain-Specific Development with Visual Studio DSL Tools written Gareth Jones and other people who lead creation of this technology at Microsoft.

The best way to follow this discussion is by opening the ClassDiagrams example from Visual Studio SDK.

image

The Class Designer DSL itself is also defined by a DSL model. The screenshot below shows a fragment of its definition located in the DslDefinition.dsl file inside of the Dsl project.

image

The diagramming notation is similar enough to UML class diagrams for you to be able to interpret it without reading a book on DSLs. As you can see, the sample DSL has a ModelRoot that contains a collection called Types of ModelType objects. ModelClass inherits from ModelType and stores information about a single class, including its collections of Attributes and Operations. In addition to the domain types, Class Diagrams Sample Solutionlike ModelRoot and ModelClass, the DSL definition also includes diagram elements, such as ClassShape and AssociationConnector, which are not shown here.

The Dsl project contains most of the code required to implement the DSL itself and its design environment in Visual Studio. The GeneratedCode folder contains a set of T4 templates that generate most of the required code. In particular, it generates strongly-typed .NET code for accessing the resulting DSL models programmatically. Here is the template called DomainClasses.tt that generates metadata classes for the sample Class DSL.

<#
/***************************************************************************
    Copyright (c) Microsoft Corporation. All rights reserved.
    This code is licensed under the Visual Studio SDK license terms.

    THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF
    ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY
    IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR
    PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

***************************************************************************/
#>
<#@ include file="Dsl\DomainClassCodeGenerator.tt" #>
<#@ Dsl processor="DslDirectiveProcessor"
  requires="fileName=’..\DslDefinition.dsl’" #>

The screenshot below shows an extract from the code this template produces. The ModelClass shown in it is generated from the ModelClass element defined in the sample Class DSL diagram above. An instance of the ModelClass represents a particular class defined in the class model, such as Customer or Product shown in the beginning of this article.

image

ModelRoot, ModelType, ModelClass, ModelAttribute and other classes in this file are what our code generator will use to access model definition created by the Class Designer.

Generating Code From DSL Models

For our DSL code generator, we will use the standard T4 Toolbox design pattern based on Template and Generator classes.

C#

<#@ template language="C#v3.5" hostspecific="True" debug="True" 

    inherits="ModelingTextTransformation" #>
<#@ output extension="txt" #>
<#@ include file="T4Toolbox.tt" #>
<#@ include file="ClassGeneratorCS.tt" #>
<#@ include file="ClassTemplateCS.tt" #>
<#@ ClassDiagrams processor="ClassDiagramsDirectiveProcessor" 

    requires="fileName=’Sample.classes’"  #>
<#
    ClassGenerator generator = new ClassGenerator();
    generator.ModelRoot = this.ModelRoot;
    generator.Run();
#>

Visual Basic

<#@ template language="VBv3.5" hostspecific="True" debug="True" 

    inherits="ModelingTextTransformation" #>
<#@ output extension="txt" #>
<#@ include file="T4Toolbox.tt" #>
<#@ include file="ClassGeneratorVb.tt" #>
<#@ include file="ClassTemplateVb.tt" #>
<#@ ClassDiagrams processor="ClassDiagramsDirectiveProcessor" 

    requires="fileName=’Sample.classes’"  #>
<#
    Dim generator As ClassGenerator = New ClassGenerator()
    generator.ModelRoot = Me.ModelRoot
    generator.Run()
#>

The sample generator consists of two separate classes. ClassTemplate, defined in ClassTemplateCs.tt or ClassTemplateVb.tt, takes ModelClass and generates a class declaration. ClassGenerator, defined in ClassGeneratorCs.tt or ClassGeneratorVb.tt, takes ModelRoot as a parameter, iterates through its list of Types and renders ClassTemplate for each ModelClass it finds. There is nothing special about design or implementation of ClassTemplate and ClassGenerator, except for the ModelClass and ModelRoot types they use as parameters for code generation. These types are provided by the Class Diagrams DSL.

This code generator produces class declarations for types defined in Sample.classes in separate source files.

C#
namespace T4ToolboxDslCs
{
    using System;
    using System.Collections.Generic;

    public class Address
    {
        public String Street { get; set; }
        public String City { get; set; }
        public String State { get; set; }
        public String Zip { get; set; }
    }
}

Visual Basic
Imports System
Imports System.Collections.Generic

Namespace T4ToolboxDsl
    Public Class Address
        Public Street As String
        Public City As String
        Public State As String
        Public Zip As String
    End Class
End Namespace

The main code generation file, SampleCs.tt or SampleVb.tt depending on your language preference, does have several important differences from a typical T4 Toolbox template. First important difference is the <#@ ClassDiagrams #> custom directive placed in the beginning of the template. This custom directive has a requires parameter that expects a value in form fileName=’<FilePath>’, where <FilePath> is a relative path to the DSL model file. The ClassDiagrams directive processor is provided by the Class Diagrams DSL and registered in the experimental registry hive of Visual Studio when you run the Dsl project in the Visual Studio SDK sample. Once the DSL is deployed this directive processor would be registered under standard Visual Studio registry hive.

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0Exp\Configuration\TextTemplating\DirectiveProcessors\ClassDiagramsDirectiveProcessor]
@="A directive processor that provides access to ClassDiagrams files"
"Class"="Microsoft.Example.ClassDiagrams.ClassDiagramsDirectiveProcessor"
"CodeBase"="file:///C:/Users/you/Desktop/DslPackage/bin/Debug/Microsoft.Example.ClassDiagrams.Dsl.DLL"

Second important difference is the use of ModelingTextTransformation in the inherits parameter of the template directive. This base class provides additional methods, such as AddDomainModel and ConvertModelRelativePathToTemplateRelativePath, which are required for the code generated by the ClassDiagrams directive processor to be compiled correctly during template transformation.

Running Sample Source Code

Debugging Sample SolutionComplete source code is available for download in the end this article. It is based on the ClassDiagrams sample from Visual Studio SDK. In order to see the code generation in action, you will need to start debugging the Dsl project in Example.ClassDiagrams solution.

This will start a second instance of Visual Studio in the experimental hive and load Debugging solution. This solution contains several projects. Debugging project came in Visual Studio SDK with this sample and contains standard T4 templates shown at the beginning of this article.

T4ToolboxDslCs and T4ToolboxDslVb are the C# and Visual Basic projects that contain code generators based on the T4 Toolbox. SampleCs.tt and SampleVb.tt are the actual code generators that produce class declarations in separate source files. Remember that you need to click the Show All Files button in the toolbar of Solution Explorer in order to see the output files in Visual Basic projects.

Conclusion

Graphical Domain-Specific Languages are a very powerful tool that can be used to build complete solutions for modeling and generating code for complex problem domains. DSLs are typically developed by Microsoft and other companies who specialize in building development tools. Due to inherent complexity of designing a graphical language and implementing a complete modeling environment integrated in Visual Studio IDE,  DSL development is not feasible in scope of a typical application development project. The investment required to implement and support a viable DSL can only be recouped on some of the largest application development projects that involve dozens of people and last for several years.

These larger projects, where DSL development is justified, typically require generating large volumes of code. Due to the T4 limitation of generating a single output file per template, developers are forced to generate multiple classes in each output file and use multiple template files to generate code from a single model to keep the size of output files manageable. A great example of this problem is the implementation of a DSL itself, which has a single model definition file and multiple code generation templates files. Each of the generated files contains numerous class definitions, making them difficult to read and debug.

The traditional T4 templates are also impossible to extend without resorting to modifying their source code directly. On a large application development project, the team developing the DSL quickly becomes a bottleneck for customization requests. Alternatively, the teams using the DSL start customizing the main T4 templates which requires additional effort to maintain them over time as new versions of the DSL are released.

Although the issues around template extensibility and output file management not be difficult for authors of the DSL, the large number and size of the T4 templates required for non-trivial scenarios present a serious challenge for application developers using it. T4 Toolbox can help you address these problems. By implementing your DSL code generator as a set of Template and Generator classes as shown in this article, you can generate all required output files from a single T4 file and allow developers to extend it with the help of inheritance and encapsulation instead of modifying the main code generator directly.

Download

Source Code, C# and Visual Basic

Resources


Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

Hello Oleg,

this is a great primer. Very nice. Indeed there is some pain involved in creating own DSLs or actually in shipping them to other machines. Therefore we did integrate the most common diagrams into T4 Editor. We will post additional T4 templates for code generation in the T4 template gallery. I am also looking forward to contribute those to T4 toolbox if you like.

[…] T4 Tutorial: Generating Code from DSL Models - Oleg Sych shares a nice tutorial on getting started with T4 code generation from a Domain Specific Language definition in Visual Studio […]

It’s great that the DSL Tools team chose to keep their model as XML, being iterable by T4 templates. After learning T4, I found applying it to DSL Tools a breeze. That said, the Tools themselves aren’t exactly piece of cake. Speaking of which, I have an article on them here (http://www.codeproject.com/KB/architecture/asyncdsl.aspx), if anyone’s interested.

That’s a nice introduction to DSLs and code generation.
I actually took the completely opposite direction. Started using T4 as a code generator for DSL models, but now I see it more and more as a general purpose code generation engine.
Your posts have been very helpful in understanding the ins and outs of T4 :)

Updated content and source code to be compatible with version 9.10 of the T4 Toolbox.