Oleg Sych

Text templates run within the constraints established by the T4 architecture. Well designed templates take advantage of the benefits offered by T4 and sometimes need to work around its limitations.

Template Transformation Process

Here is a collaboration diagram that illustrates the template transformation process and its participants. Sections below provide a step by step description of the process. References to objects and steps from this diagram use italic font.

Custom Tool

Template transformation process begins in Visual Studio when you right-click a .tt file in Solution Explorer and select “Run Custom Tool” from the context menu. Custom Tool is a special COM object that implements IVsSingleFileGenerator interface, defined by Visual Studio. The T4 Custom Tool is implemented by a .NET class called TemplatedCodeGenerator and registered in Visual Studio settings under name TextTemplatingFileGenerator. Visual Studio locates the Custom Tool by its name, creates the COM object and calls its Generate method (step 1).

ITextTemplating Service

The custom tool uses IServiceProvider interface provided by Visual Studio to locate a service provided by T4, called ITextTemplating. This service is implemented in Microsoft.VisualStudio.TextTemplating.VSHost assembly by an internal class called TextTemplatingService. Once the service is located, the Custom Tool calls its ProcessTemplate method, passing it template contents and some other parameters (step 1.1).

Host

TextTemplatingService serves as a Host for the code generation Engine. While the Engine is responsible for code generation, the Host is responsible for providing all external resources it requires. The Host implements ITextTemplatingEngineHost interface defined by T4. During code generation, the Host passes this interface along with the template contents to the ProcessTemplate method of the Engine for code generation (step 1.2).

Engine

The T4 Engine is responsible for parsing the template (step 2), generating and compiling a GeneratedTextTransformation (steps 3, 4 and 5),  running it and returning output it produced to the Host (step 6). After parsing the template contents (step 2), the Engine processes T4 directives used in the template. While processing a built-in directives (step 3) the Engine calls the Host to perform the required action. For example, to process the include directive the Engine calls the LoadIncludeText method of the Host (step 3.1).

DirectiveProcessor

When processing custom directives (step 4), the Engine first calls the ResolveDirectiveProcessor of the Host (step 4.1) to determine the Type of DirectiveProcessor for each of them. Directive Processors work by adding code to the GeneratedTextTransformation. For example, property directive creates a property that can be used to pass a parameter value to the template from the outside. Once processors for all types of custom directives are resolved, the Engine instantiates and calls the ProcessDirective of an appropriate processor for each custom directive (step 4.2).

GeneratedTextTransformation

The Engine combines parsed template blocks and output of custom directives to produce source code for the GeneratedTextTransformation, a special class that inherits from TextTransformation class provided by T4 and represents a "compiled" version of the template (step 5).

The Engine calls the ResolveAssemblyReference method of the Host to locate all assemblies referenced by the template and required to compile it (step 5.1). It then calls the ProvideTemplatingAppDomain method of the Host to obtain the AppDomain where the code will be compiled (step 5.2). Visual Studio Host will create a new AppDomain called TemplatingAppDomain (step 5.3). If the TemplatingAppDomain already exists, Visual Studio Host will reuse it up to 25 times to improve performance.

After obtaining a TemplatingAppDomain, the Engine uses a CodeDomProvider (C# or Visual Basic, depending on the language specified in the template directive) to compile the the source code of the GeneratedTextTransformation in a temporary, in-memory assembly (step 5.4). This temporary assembly remains cached in the TemplatingAppDomain. If the same template is transformed again, the Engine will find find a cached assembly instead of generating a new one.

Once GeneratedTextTransformation is compiled, the Engine creates a new instance of this class and calls its TransformText method (step 6). This method returns output content generated by the template, which then goes back to the Engine, the Host, the Custom Tool and finally to the Visual Studio.

Analysis

Sections below highlight some of the aspects of T4 architecture. While some of them represent constraints that T4 architecture places on template design, others represent opportunities that may be overlooked by template developers.

T4 is designed to generate one output file per template

A close look at the Generate method of the IVsSingleFileGenerator interface reveals that a Custom Tool is responsible only for generating content. The custom tool returns generated content to Visual Studio, which is responsible for saving it to a file, adding the output file to the current project and placing it under source control. It’s a great design that allows custom tool to know very little about Visual Studio internals, however, it also limits T4 to producing a single output file per template. T4 templates need to use special workarounds to generate multiple output files from a single template.

ITextTemplating service can be used in Visual Studio add-ons

Use of ITextTemplating service is not limited just to the T4 Custom Tool. It is possible to use this service directly from a Visual Studio add-on. For example, a DSL designer could use this service to generate code directly from domain model using one or more standalone templates, without requiring users to add and manage .tt files in their projects.

Behavior of built-in directives can be changed by implementing a custom host

Keep in mind that the Engine handles built-in directives without creating directive processors. For example, it handles the output directive by calling the SetFileExtension and SetOutputEncoding methods. In order to change behavior of built-in directives you would need to create a custom T4 host.

Behavior of T4 directives varies between different hosts

Ability to create a custom T4 host is a double-edged sword. Three different T4 host implementations are available from Microsoft - Visual Studio host (described in this article), command line host and GAT host. Functionality provided by these hosts varies significantly and affects behavior of assembly, include and property directives, making it more difficult to reuse the same template with different hosts.

Custom directives can extend T4 template syntax

You can create a custom directive processor for your T4 templates and register it with Visual Studio Host. Unfortunately, command line host does not support the same registration mechanism, which may limit your ability to use custom directives. 

Assemblies referenced by T4 template become locked

Remember that Visual Studio Host reuses the templating AppDomain up to 25 times to improve performance. Unfortunately, this comes at a cost. Once the assembly containing GeneratedTextTransformation class is loaded in the templating AppDomain, the .NET Framework locks all of the assemblies it references. If your template uses one of your own assemblies, you will not be able to recompile it after running the template. This problem cannot be easily solved if your template needs run code from your assembly. However, if your template simply uses Reflection to generate code based on types in your assembly, you can work around this problem by using a different API, such as Introspection or CodeModel to access type metadata.

GeneratedTextTransformation runs in Visual Studio process

Ultimately, GeneratedTextTransformation runs in the Visual Studio process (devenv.exe), which hosts the TemplatingAppDomain. One of the side effects of this design is that debugging the compiled template requires you to attach a debugger to the Visual Studio itself. In other words, you typically need a second instance of Visual Studio to debug your template running inside the first one.

Another side effect of this design is that Visual Studio process defines the CurrentDirectory for the running GeneratedTextTransformation, which doesn’t match the directory where the template itself is located. This makes it more difficult to use relative paths referring to external files in T4 templates. As described in the next section, template authors need to use ResolvePath method to work around this problem.

Templates can access the Host

Templates can access the Host using the hostspecific parameter of the template directive. This parameter adds a Host property to the GeneratedTextTransformation class giving the template access to the instance of the ITextTemplatingEngineHost interface. This gives to the template author to use the ResolvePath method to convert relative paths to absolute paths. For example, AzMan template uses this approach to allow you to specify a relative path to the model (XML file that contains AzMan security definitions) used to generate a strongly-typed .NET wrapper. Without this ability, use of templates in team projects would be difficult because file paths would need to be hard-coded.

Templates can access Visual Studio services

Visual Studio Host implements IServiceProvider interface, which template can obtain by type-casting the instance of ITextTemplatingEngineHost available via its Host property. Having this interface instance, template can use its GetService method to access services provided by Visual Studio. For example, template can access DTE object and add multiple output files to a Visual Studio project.

About T4

T4 (Text Template Transformation Toolkit) is a template-based code generation engine. It is available in Visual Studio 2008 and as a download in DSL and GAT toolkits for Visual Studio 2005. T4 engine allows you to use ASP.NET-like template syntax to generate C#, T-SQL, XML or any other text files.

Additional information about T4 architecture is available in MSDN article Architecture of the Text Template Transformation Process. For more information about T4, check out my previous article.