How to generate multiple outputs from single T4 template


Update: A new version of the code described in this article is available in T4 Toolbox. For details, click here.

Overview

For some code generation tasks, the exact number of code artifacts to be generated may not be known upfront. For example, when generating CRUD stored procedure for a given database table, you may want to have one SELECT stored procedure created for each index. As new indexes are added to the table, additional SELECT procedures need to be generated.

Unfortunately, T4 was designed to generate a single output file per template. This forces developer to either generate all code artifacts in a single file, or create an additional T4 template for each new artifact that needs to be generated. In the CRUD stored procedure example, we need to either generate all stored procedures in a single .sql file, or add a new template to the code generation project for each additional SELECT stored procedure.

Both of these approaches are less than ideal. Generating a single source file with multiple artifacts tends to produce large files, which are difficult to understand and track changes in. Many development teams adopted a practice of creating a separate source file per code artifact (i.e. one C# class per .cs file, one stored procedure per .sql file, etc.) This allows to bring physical structure of a project (source files and folders on hard drive) closer to its logical structure (types and namespaces) making it easier to understand. Changes made in smaller files are also easier to track with version control tools. Most Visual Studio project item templates encourage this practice and Database Professional edition in particular is strongly geared toward using a single .sql file per database object.

Strongly-typed DataSet generator compromised between having one source file per code artifact and producing multiple artifacts from a single project item by generating a single DataSet class with multiple DataTable and DataRow classes nested in it. This can produce huge source files for non-trivial DataSets and long class names that don’t fit on a single line. LINQ DataContext generator addresses the problem with long class names by generating multiple classes in a single source file without nesting, but stops short of splitting individual generated classes into separate source files.

Ideally, a code generation tool should allow producing multiple output files from a single Visual Studio project item. Additional output files should be automatically added to the Visual Studio project to which the original project item belongs and previously generated output files that are no longer needed should be automatically removed from the project.

This article demonstrates how to accomplish this goal with T4 text templates. It discusses alternative approaches of generating multiple outputs and briefly describes code required to automatically add/remove output files from a Visual Studio project. Ready-to-use code is included at the end of the article with step-by-step usage instructions.

Producing multiple outputs from a single T4 template

T4 compiles a text template into a class descending from TextTransformation and calls its TransformText method. This method returns a string that contains output produced by the template, which is then written to the output file by the T4 engine. The only way to produce multiple output files is to handle this explicitly, in the code of the template itself. While writing a code block that creates a file is trivial, the main challenge is to take advantage of T4 capabilities to generate its contents.

Saving accumulated content

Compiled TextTransformation accumulates output of text blocks, expression blocks, Write and WriteLine methods by appending it to an internal StringBuilder object exposed by its GenerationEnvironment property. We can write code that save all currently accumulated content to a file. Here is a helper method that does that:

SaveOutput.tt
<#@ template language=C#hostspecific=true#>
<#@ import namespace=System.IO#>
<#+
  void SaveOutput(string outputFileName)
  {
      string templateDirectory = Path.GetDirectoryName(Host.TemplateFile);
      string outputFilePath = Path.Combine(templateDirectory, outputFileName);
      File.WriteAllText(outputFilePath, this.GenerationEnvironment.ToString()); 

      this.GenerationEnvironment.Remove(0, this.GenerationEnvironment.Length);
  }
#>

This template turns on the hostspecific option to make T4 generate the Host property, which SaveOutput method uses to determine output directory.

Here is a template that uses this helper method to generate two output files.

Example1.tt
<#@ include file=SaveOutput.tt#>
<#
    GenerateFile1();
    SaveOutput(”File1.txt”);  

    GenerateFile2();
    SaveOutput(”File2.txt”);
#>
<#+
    void GenerateFile1()
    {
#>
This is file 1
<#+
    }  

    void GenerateFile2()
    {
#>
This is file 2
<#+
    }
#>

Template in Example1.tt uses class feature blocks to separate generation of content for different files into different methods - GenerateFile1 and GenerateFile2. In this example, these methods are very simple and contain a single text block each. However, we can easily extend them by adding method parameters, expanding the methods with additional text blocks and expression blocks, calling other helper methods, etc. As the example below illustrates, we can move GenerateFile1 and GenerateFile2 methods into separate template files (File1.tt and File2.tt) and use the include directive to make them available in the main template (Example2.tt). This would be beneficial if the individual methods become too complex or if we need to reuse them in several multi-output templates.

Example2.tt
<#@ include file=SaveOutput.tt#>
<#@ include file=File1.tt#>
<#@ include file=File2.tt#>
<#
    GenerateFile1(”parameter 1″);
    SaveOutput(”File1.txt”);  

    GenerateFile2(”parameter 2″);
    SaveOutput(”File2.txt”);
#>

File1.tt
<#+
    void GenerateFile1(string parameter)
    {
#>
This is file 1, <#= parameter #>
<#+
    }
#>

File2.tt
<#+
    void GenerateFile2(string parameter)
    {
#>
This is file 2, <#= parameter #>
<#+
    }
#>

Saving accumulated content is an effective method for generating multiple files. It is simple, but can scale up to handle complex code generation scenarios. On the other hand, this approach doesn’t allow reuse of standalone templates. In other words, a T4 template that already produces a particular output file cannot be reused “as is” in a multi-output template and needs to be converted into an “include” template with a class feature block that defines a “template” method. And vice versa, an “include” template cannot be used in standalone mode to produce a single output file.

Calling standalone template

Instead of saving output accumulated by a single compiled template (TextTransformation class), we can have T4 engine compile and run another, separate template; collect it’s output and save it to a file. We can repeat this process as many times as necessary. Here is a helper method that does that.

ProcessTemplate.tt
<#@ template language=C#hostspecific=True#>
<#@ import namespace=System.IO#>
<#@ import namespace=Microsoft.VisualStudio.TextTemplating#>
<#+
  void ProcessTemplate(string templateFileName, string outputFileName)
  {
    string templateDirectory = Path.GetDirectoryName(Host.TemplateFile);
    string outputFilePath = Path.Combine(templateDirectory, outputFileName);  

    string template = File.ReadAllText(Host.ResolvePath(templateFileName));
    Engine engine = new Engine();
    string output = engine.ProcessTemplate(template, Host);  

    File.WriteAllText(outputFilePath, output);
  }
#>

This template also turns on the hostspecific option to generate the Host property. ProcessTemplate method uses this property to determine full path of the standalone template file as well as the output directory. ProcessTemplate method creates a new instance of T4 Engine class, which it uses to compile and run the standalone template.

Here is a template that uses this helper method to generate two output files from two standalone templates.

Example3.tt
<#@ include file=ProcessTemplate.tt#>
<#
    ProcessTemplate(”Standalone1.tt”, “StandaloneOutput1.txt”);
    ProcessTemplate(”Standalone2.tt”, “StandaloneOutput2.txt”);
#>
Standalone1.tt
<#@ output extension=txt#>
This is file 1

Standalone2.tt
<#@ output extension=txt#>
This is file 2

Unfortunately, there is no standard way to provide parameters to a standalone template built into T4 engine itself. GAX includes a custom T4 host that provides <#@ property #> directive processor and T4 editor installs a standalone <#@ property #> directive processor which can be used with standard T4 host. Discussion of these options is outside of scope of this article since both of them require having additional components, which are not included with Visual Studio 2008 by default.

One possible way to pass parameters to a standalone T4 template is via remoting CallContext. Here is how previous example can be expanded to achieve that.

Example4.tt
<#@ import namespace=System.Runtime.Remoting.Messaging#>
<#@ include file=ProcessTemplate.tt#>
<#
    CallContext.SetData(”Standalone1.Parameter”, “Value 1″);
    ProcessTemplate(”Standalone1.tt”, “StandaloneOutput1.txt”);  

    CallContext.SetData(”Standalone2.Parameter”, “Value 2″);
    ProcessTemplate(”Standalone2.tt”, “StandaloneOutput2.txt”);
#>
StandaloneWithParameter1.tt
<#@ template language=C##>
<#@ output extension=.txt#>
<#@ import namespace=System.Runtime.Remoting.Messaging#>
This is file 1 <#= Parameter #>
<#+
    string Parameter
    {
        get { return (string)CallContext.GetData(”Standalone1.Parameter”); }
    }
#>
StandaloneWithParameter2.tt
<#@ template language=C##>
<#@ output extension=.txt#>
<#@ import namespace=System.Runtime.Remoting.Messaging#>
This is file 2 <#= Parameter #>
<#+
    string Parameter
    {
        get { return (string)CallContext.GetData(”Standalone1.Parameter”); }
    }
#>

Compared to saving accumulated content to multiple output files, calling standalone templates is more flexible, but also more complex. It is more flexible because it allows authoring templates that can be used in standalone mode to produce a single output file, but can also be called from another template which produces multiple output files. This approach is more complex because it requires custom code or additional (third-party) components to pass parameters from calling template to the standalone template. Calling a standalone template is also slower than saving accumulated output, because T4 has to compile standalone template separately from the calling template. Ad-hoc tests show that saving of accumulated content appears to be 50% faster than execution of a similar standalone template. Actual impact on performance will depend on complexity of the templates, frequency of their compilation and may not be noticeable.

Adding/removing files from a Visual Studio project

Adam Langley provides a good overview of creating a custom code generator that adds multiple output files to a Visual Studio project in his article on CodeProject. While it is certainly possible to build your own custom tool that generates multiple output files from T4 templates, it requires a separate Visual Studio project and has to be installed on each developer’s computer. Fortunately, it is possible to access Visual Studio extensibility APIs from a T4 template directly, which allows us to put the entire code generation solution into T4 template files that developer can get with the rest of the application source from their version control system.

As Adam Langley explains in his article, the task of adding a generated file to a Visual Studio project boils down to obtaining a ProjectItem that, in our case, represents the T4 template which generates multiple output files. Then it’s a simple matter of calling AddFromFile method to add each output file as a nested sub-item to the project.

MultiOutput.tt template contains Adam’s code adapted for execution within a T4 code block. The only significant change that was required was in the way it obtains the DTE service. Instead of calling Package.GetGlobalService, which from a T4 code block returns null, it needs to request it through the Host property:

IServiceProvider hostServiceProvider = (IServiceProvider)Host;
EnvDTE.DTE dte = (EnvDTE.DTE)hostServiceProvider.GetService(typeof(EnvDTE.DTE));

For complete details, please refer to Adam’s article and __getTemplateProjectItem helper method in the attached MultiOutput.tt. Here is an updated version of SaveOutput and ProcessTemplate helper methods.

<#+
  List<string> __savedOutputs = new List<string>();
  Engine __engine = new Engine();  

  void DeleteOldOutputs()
  {
      ProjectItem templateProjectItem = __getTemplateProjectItem();
      foreach (ProjectItem childProjectItem in templateProjectItem.ProjectItems)
      {
          if (!__savedOutputs.Contains(childProjectItem.Name))
              childProjectItem.Delete();
      }
  }  

  void ProcessTemplate(string templateFileName, string outputFileName)
  {
      string templateDirectory = Path.GetDirectoryName(Host.TemplateFile);
      string outputFilePath = Path.Combine(templateDirectory, outputFileName);  

      string template = File.ReadAllText(Host.ResolvePath(templateFileName));
      string output = __engine.ProcessTemplate(template, Host);
      File.WriteAllText(outputFilePath, output);  

      ProjectItem templateProjectItem = __getTemplateProjectItem();
      templateProjectItem.ProjectItems.AddFromFile(outputFilePath);  

      __savedOutputs.Add(outputFileName);
  }  

  void SaveOutput(string outputFileName)
  {
      string templateDirectory = Path.GetDirectoryName(Host.TemplateFile);
      string outputFilePath = Path.Combine(templateDirectory, outputFileName);  

      File.WriteAllText(outputFilePath, this.GenerationEnvironment.ToString());
      this.GenerationEnvironment = new StringBuilder();  

      ProjectItem templateProjectItem = __getTemplateProjectItem();
      templateProjectItem.ProjectItems.AddFromFile(outputFilePath);  

      __savedOutputs.Add(outputFileName);
  }
#>

As you can see, ProcessTemplate and SaveOutput methods add generated file to the current Visual Studio project as a nested item under the template. They also save the name of the output file in a list, which is then used by DeleteOldOutputs method which removes all nested project items that weren’t generated by the current template transformation. DeleteOldOutputs helper method allows templates that generate a varying number of output files to automatically remove obsolete files from the project. For example, if the template generates multiple SQL files with a SELECT stored procedure for each index in a given database table, calling DeleteOldOutputs will automatically remove obsolete SELECT stored procedure when an index is removed from the table.

Usage

  • Add MultiOutput.tt from the attached ZIP archive to the Visual Studio project you are using for code generation.
  • If you are using the SaveOutput approach, add to the Visual Studio project a new .tt file similar to the example below. Examples of File1.tt and File2.tt templates are available above.
<#@ output extension=txt#>
<#@ include file=MultiOutput.tt#>
<#@ include file=File1.tt#>
<#@ include file=File2.tt#>
<#
    GenerateFile1(”parameter 1″);
    SaveOutput(”File1.txt”);  

    GenerateFile2(”parameter 2″);
    SaveOutput(”File2.txt”);  

    DeleteOldOutputs();
#>
  • If you are using the ProcessTemplate approach, add to the Visual Studio project a new .tt file similar to the example below. Examples of Standalone.tt and Standalone2.tt templates are available above.
<#@ output extension=txt#>
<#@ import namespace=System.Runtime.Remoting.Messaging#>
<#@ include file=MultiOutput.tt#>
<#
    CallContext.SetData(”Standalone1.Parameter”, “Value 1″);
    ProcessTemplate(”Standalone1.tt”, “StandaloneOutput1.txt”);  

    CallContext.SetData(”Standalone2.Parameter”, “Value 2″);
    ProcessTemplate(”Standalone2.tt”, “StandaloneOutput2.txt”);  

    DeleteOldOutputs();
#>
  • Solution Explorer showing multiple output files You should see the generated output files listed under their templates in Solution Explorer.

Download

References

About T4

T4 (Text Template Transformation Toolkit) is a template-based code generation engine. It is available in Visual Studio 2008 and as a download in DSL and GAT toolkits for Visual Studio 2005. T4 engine allows you to use ASP.NET-like template syntax to generate C#, T-SQL, XML or any other text files.

For more information about T4, check out my previous article.


Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

Humm…. great post but I can’t seem to get the CallContext to work. When I try and get the data out it keeps comeing back and saying that it is empty.
Any ideas??

It is hard to say without looking at the code. What type of object are you trying to pass as a parameter?

Does changing the code from using CallContext GetData/SetData methods to LogicalGetData/LogicalSetData solve the problem?

I am using the utility templates you attached, and they look very promising, but I am getting some errors using vs2005.

1) In the MultiInput.tt @Template directive, I think you need the standard “inherits” parameter. I get a bunch of errors without this.

2) SaveOutput() seems to work, but won’t work in my situation, due to some of the constraints I have on the templates I wish to combine (duplicate variable names etc)

3) ProcessTemplate seems perfect, but doesn’t work. I get an error in the generated text transformation class, saying that my DSL assembly cannot be found (in the using statement of the text generation class). If I just call my sub-template directly, it works fine, but when using the template in a combined fashion, it cannot resolve. Help?

Jason, let me try to answer your questions.

1) I wrote this code for templates that don’t require DSLs, which is why the template directive in MultiOutput.tt doesn’t have the inherits parameter. Perhaps it would be best to remove the template directive from MultiOutput.tt altogether and rely on the including template to provide the hostspecific parameter it expects.

2) Perhaps encapsulating your templates using the Nested Template Class technique would help to overcome this.

3) It’s hard for me to tell what this problem may be. Does the error occur when T4 is trying to compile the template or when it is actually running the GeneratedTextTransformation?

My error occurs when trying to compile the text generation class. the “using” statement at the top of the file that references my DSL assembly is the culprit. Note, that this is the generated class for the “included” template that fails, not the outer “multi template” template.

If I generate the “included” template directly, it works fine. It seems like the DSL assembly is not in scope or passed into the context of the inner compilation engine.

As for removing the template directive, that does not work, as then the multiOutput.tt does not compile, because “host” is not defined.

Ill take a look at the nested template class.

[…] How to generate multiple outputs from single T4 template […]

Well, as a partner of mine said “I’ve seen light!”. The article is great as a very handful example of running code for text-based template code generation, and helps me a lot in order to understand the concept.
I repeat, excellent work!!!

I have a question now, regarding the topic you mention here. I tried to “take outside” the ProcessTemplate procedure, right into a C# console program. When I process a template, an error CS0006: Could not find metafile “System.XML” (I´m using XML inside the template I´m trying to process)… any idea why is that?
Greetings

It’s hard to tell without knowing how you adapted the code to run in a console program. If you implemented a custom host, it may not be resolving assembly references properly.

One thing I found while trying this out. Could be usefull to anyone using foreighn characters, on the lines with File.WriteAllText(), it is usefull to add the encoding parameter at the end, so that the accent marks and other characters can apear correctly. In my case, I was generating aspx files, and all though the text output from the transformation was correct, the rendered output on the browser wasn’t. Adding System.Text.Encoding.UTF-8 at the end of the parameter list on the WriteAllText resolved the problem.

Thanks for this great article, which helpes ALOT!!!

[…] In this instance we generate all the SQL into one file, if we wanted to we could easily generate each constraint into its own file, Oleg Sych has an example of how to do this. […]

Hi,

When i use DeleteOldOutputs(); This function deletes the standard output file for the t4 template. But this files isn’t removed from the solution.

Any ideas?
Thx in advance

Glenn, Visual Studio always recreates the standard output file after execution of the T4 template is finished. I haven’t found a way to suppress it.

I’m experimenting with the attached project files, but SaveOutput.tt doesn’t seem to produce any output.

Hi!

I am using Rob Conery’s Subsonic template but the section in the middle where he writes out a table of all properties in his List.tt template I want to output into a UserControl.ascx but cannot get it to work. Could you help?

[…] Quando usamos T4 pode ser necessário gerar vários arquivos com um uníco template, um exemplo clássico é quando estamos gerando o mapeamento para as tabelas do SGBD. Neste post vamos ver um exemplo de como gerar vários arquivos usando templates T4. O exemplo é baseado no post do Oleg. […]

[…] How to generate multiple outputs from single T4 template […]

Hello

Thank you for all your effort.
You have certainly helped me to get started with T4.
Could I ask the following question:
I have used the code below in an attempt to incrementally process templates to build up a set of wpf xaml windows.

I however get an error when referencing Host.

( string output = engine.ProcessTemplate(template, Host);
)

Could you help.
I want to use the default Host in visual studio.

[…] Oleg Sych, How to generate multiple outputs from single T4 template. […]

Great post! I used your first example approach combined with PetaPOCO generator T4 scripts, which create a POCO class for each table in a database and put them into a single file. With your code I modified the script to allow the generation of a single file for each class. It worked like a charm with Visual Studio 2010 Ultimate, 4.0 Framework. Thanks so much!

Hi
I am looking for feasibility to create WCF generator to include as follows:

Using T4 template ready to use Linq to Sql generator, for a particular database, Template generates class files.

Looking at approach to read the linq to sql generated classes and dynamically create WCF service and include methods by extracting from linq to sql generated classes.

Could you please provide few inputs on this.

Thanks,
Ravi