.Config File Transformation


This article provides an overview of configuration files in .NET applications, discusses common problems of managing environment-specific settings, reviews different approaches for managing configuration settings and introduces .config file transformation in Visual Studio 2010.  The downloadable code example and shows how to use config transforms not only with web applications, but also with regular app.config files in other types of projects.

Overview of Configuration Files

Today, most non-trivial applications use configuration settings. Commercial software products must be able to run in different environments for different users. At the very least that means being able to run from a folder specified by the user during installation. Non-trivial applications often need dozens of settings, including database connection strings, network locations and credentials. These configuration settings can be stored and accessed in a number of different ways. On the Microsoft Windows platforms, we have used INI files, database tables and registry keys. Applications built using the .NET Framework commonly use XML-based configuration files, such as the one shown below.

App.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="MySetting" value="MyValue"/>
  </appSettings>
</configuration>

When building a C# or Visual Basic project that contains this configuration file, Visual Studio will copy it to the output directory and rename to match the name of the executable, such as ConsoleApplication.exe.config. The application can access the settings from this file using ConfigurationManager and other classes in the System.Configuration namespace. Here is an example.

using System;
using System.Configuration;

namespace ConsoleApplication
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(ConfigurationManager.AppSettings["MySetting"]);
            Console.WriteLine("Press any key to close the application …");
            Console.ReadKey();
        }
    }
}

Managing Configuration Settings

Although applications developed by companies for internal use may not need to be as configurable, mature development teams take advantage of configuration settings to run their applications in different environments for development and production purposes so that the application can be modified and tested without affecting the real users. Because internal applications are typically installed and configured by people with technical background who are comfortable with editing the XML-based configuration files, building a robust GUI for configuration settings does not add much business value. However, the task of configuring the application for a particular environment still remains and falls onto the plate of either developers or administrators. When done manually this process is tedious and error-prone.

Multiple .Config Files

Without resorting to writing an installation program or script, one of the options you have is to maintain a separate copy of the application configuration file for each environment. For example, in addition to the App.config shown above which you would use in development environment, you might also have two more configuration files for your test and production environments.

Test.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="MySetting" value="MyTestValue"/>
  </appSettings>
</configuration>
Production.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="MySetting" value="MyProductionValue"/>
  </appSettings>
</configuration>

When deploying the application to test environment, you would replace the default ConsoleApplication.exe.config with the Test.config, and for production environment you would use the Production.config. The process is simple enough to perform manually and easy to automate with a simple batch script. However, it only works well when the configuration file is small and contains only environment-specific settings, such as database connection strings. As configuration files become larger maintaining the common settings duplicated in each configuration file becomes more and more difficult.

ConfigSource Attribute

To solve this problem, the .NET Framework 2.0 introduced ConfigSource attribute for all configuration sections. Using this attribute we can extract environment-specific sections from the App.config to a separate file.

App.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings configSource="AppSettings.config"/>
</configuration>
AppSettings.config
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="MySetting" value="MyValue"/>
</appSettings>

When a section file, such as AppSettings.config, is copied to the output directory, ConfigurationManager will be able to automatically load settings from it instead of the main configuration file, ConsoleApplication.exe.config in our example. To change this configuration for a particular environment, we now only need to replace the section file, AppSettings.config, and not the entire App.config. Here are the environment-specific versions of the section configuration files for our sample project.

AppSettings.Test.config
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="MySetting" value="MyTestValue"/>
</appSettings>
AppSettings.Production.config
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="MySetting" value="MyProductionValue"/>
</appSettings>

Extracting sections into separate files allows us to manage settings at a more granular level. It works well when environment-specific settings are concentrated in a particular section, such as AppSettings or ConnectionStrings. However, once the number of sections that need to be replaced for different environments begins to grow, the section files become more and more difficult to manage.

Web.config File Transformation

Visual Studio 2010 introduced a new solution for managing environment-specific settings in Web application projects – config file transformation. With this approach, your web.config file stores settings required to run the application in your local development environment.

Web.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="MySetting" value="Debug Value"/>
  </appSettings>
</configuration>

imageOn the other hand, all settings specific to a particular shared environment (such as test, staging or production) are stored in a separate transformation file.The example below shows a transformation file for the Release environment that changes MySetting defined in the Web.config.

Web.Release.config
<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appSettings>
    <add key="MySetting" value="Release Value"
         xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
  </appSettings>
</configuration>

imageVisual Studio updates Web.Config using the transformation file when packaging or publishing the application. You can perform both of these actions from the project’s context menu in Solution Explorer or using MSBuild to build the Package target of the web application project.

msbuild WebApplication.csproj /target:Package /p:Configuration=Release

New web application projects you create in Visual Studio 2010 will have both Debug and Release transform files created automatically. If you have an existing web application project, perhaps upgraded from Visual Studio 2008, you can add transform files to it by right-clicking the web.config in the Solution Explorer and selecting “Add Config Transforms” from the context menu. However, keep in mind that config transformation does not occur during regular build or debugging of the web application project. If Debug configuration represents local development environment, you may want to remove the Web.Debug.config transformation file to avoid confusion.

Transformation file syntax is based on the standard syntax of configuration files, with addition of the xdt attributes, such as Locator, which specifies how to find a particular setting in the original .config file, and Transform, which specifies how to modify it. In our example, the add element will be found in the appSettings section using its key (MySetting); once the original element is found, all of its attributes will be replaced with those specified in the transformation file (SetAttributes). Here is the resulting Web.config after transformation.

Web.config (transformed)
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <appSettings>
    <add key="MySetting" value="Debug Value"/>
  </appSettings>
</configuration>

Unlike the approach based on ConfigSource attribute and separate section files, you only have a single transform file per environment. And because config transformation is integrated in the web application publishing process and allows you to use standard Visual Studio build configurations, such as Debug for development and Release for production, as well as define custom environments, such as Test, as necessary. Configurations/environments can be switched quickly, without leaving Visual Studio or running external tools.

image

On the flipside, in order to use config transforms you need to learn a new XML-based language that defines the transformations. However, you will typically use a minimum subset of its capabilities and find that, on most internal application projects, the initial effort will be well justified by the simplicity and reduced cost of ongoing maintenance of the environment-specific settings.

App.config File Transformation

Out of the box, .config file transformation is only available for Web application projects in Visual Studio 2010. Console, WinForms, WPF and other applications projects don’t support this capability out of the box. Luckily, you can add it to any Visual Basic or C# project by manually changing the project source file as shown below.

<!– –>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />

<UsingTask TaskName="TransformXml"
  AssemblyFile="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.Tasks.dll" />

<Target Name="AfterCompile" Condition="exists(’app.$(Configuration).config’)">
  <!– Generate transformed app config in the intermediate directory –>
  <TransformXml Source="app.config"
    Destination="$(IntermediateOutputPath)$(TargetFileName).config"
    Transform="app.$(Configuration).config" />
  <!– Force build process to use the transformed configuration file from now on. –>
  <ItemGroup>
    <AppConfigWithTargetPath Remove="app.config"/>
    <AppConfigWithTargetPath Include="$(IntermediateOutputPath)$(TargetFileName).config">
      <TargetPath>$(TargetFileName).config</TargetPath>
    </AppConfigWithTargetPath>
  </ItemGroup>
</Target>
<!– –>

In this MSBuild code snippet, we are using the TransformXml task defined in the Microsoft.Web.Publishing.Tasks.dll. This is the task responsible for transforming Web.config files in Web application projects. It takes parameters that specify names of the original .config file, the transform file and the output (transformed) file. We use TransformXml task to override AfterCompile target if a transform file exist for the current build configuration. For example, if our project contains a file called App.Debug.config and the current project configuration is Debug, the AfterCompile target will be called. Finally, we use ItemGroup element to change the built-in AppConfigWithTargetPath item collection. This is necessary in order for the ClickOnce publishing process and the Visual Studio debugging process (vshost.exe) to discover and use the transformed configuration file instead of the original one.

image

Unlike with Web application projects, you will need to add the transform files to the project manually, following the naming convention of App.<Configuration Name>.config, such as App.Release.config. Also, the transform files will not be automatically nested under the main App.config. To achieve this effect, you can manually edit the project file and specify the DependentUpon attribute for transform files as shown below.

<!– –>
<ItemGroup>
  <None Include="App.config" />
  <None Include="App.Release.config">
    <DependentUpon>App.config</DependentUpon>
  </None>
</ItemGroup>
<!– –>

With these changes made, the App.config file will be transformed at build time, using the configuration-specific transformation file you provided. If the transformation file doesn’t exist, the original App.config file will be used. Unlike web applications, which are compiled at run-time by ASP.NET, regular applications are fully compiled by Visual Studio during project build. This also means that config transformation occurs during development build/debug cycle, so if the Debug project configuration represents local environment on each developer’s machine, you could use config transformation for Debug configuration too. However, for consistency with web projects and to reduce confusion for less experienced developers on your team, it may be best to avoid this.

XML File Transformation

Config transformation is not limited to only .NET configuration files. You can transform any valid XML files in Visual Studio projects based on MSBuild, such as Windows Azure Project. This project type comes with Windows Azure tools for Visual Studio and is used to build service packages that describe one or more virtual machines (called roles) and include the compiled applications that will be running on these machines. Each service package is accompanied by a service configuration file, which is typically used to specify connection strings for SQL Azure databases and storage accounts as well as the number of virtual machine instances required for each role.

ServiceConfiguration.cscfg (Production)
<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject" osFamily="1" osVersion="*"
  xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">

  <Role name="WebApplication">
    <Instances count="2" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
        value="DefaultEndpointsProtocol=https;AccountName=…;AccountKey=…" />
    </ConfigurationSettings>
  </Role>

</ServiceConfiguration>

As you might expect, Visual Studio also provides the ability to debug Windows Azure Projects on your local machine, without uploading the service package to the cloud. When running in local development environment, called Dev Fabric, your service package often needs to have different configuration. For example, instead of connecting to a SQL Azure database it would connect to your local SQL Server instance, instead of cloud-based table storage it would write diagnostic logs to the local storage emulator, instead of having 2 or more role instances required for high-availability it would need to run only one to improve debugging experience, and so on. Here is a development version of the service configuration file you might want to use.

ServiceConfiguration.cscfg (Debug)
<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject" osFamily="1" osVersion="*"
  xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
  <Role name="WebApplication">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
        value="UseDevelopmentStorage=true" />
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

imageWith a little customization we can get config transformation work for the Windows Azure Projects too. Since Visual Studio doesn’t support config transformation for this type of projects yet, we again need to modify it manually as shown below.

WindowsAzureProject.ccproj
<!– –>
<Import Project="$(CloudExtensionsDir)Microsoft.CloudService.targets" />

<UsingTask TaskName="TransformXml"
  AssemblyFile="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.Tasks.dll" />

<PropertyGroup>
  <ServiceConfigurationTransform>ServiceConfiguration.$(Configuration).cscfg</ServiceConfigurationTransform>
</PropertyGroup>

<Target Name="TransformServiceConfiguration"
  BeforeTargets="CopyServiceDefinitionAndConfiguration"
  Condition="exists(’$(ServiceConfigurationTransform)’)">

  <!– Generate transformed service config in the intermediate directory –>
  <TransformXml Source="@(ServiceConfiguration)"
    Destination="$(IntermediateOutputPath)%(Filename)%(Extension)"
    Transform="$(ServiceConfigurationTransform)" />

  <!– Force build process to use the transformed configuration file from now on. –>
  <ItemGroup>
    <ServiceConfiguration Remove="ServiceConfiguration.cscfg" />
    <ServiceConfiguration Include="$(IntermediateOutputPath)ServiceConfiguration.cscfg" />
  </ItemGroup>
</Target>
<!– –>

Just like with the console application we customized previously, we use the TransformXml MSBuild task defined in the Microsoft.Web.Publishing.Tasks.dll to transform the service configuration file in a custom target called TransformServiceConfiguration that will be executed before the built-in CopyServiceDefinitionAndConfiguration. One the transformed configuration file is generated in the intermediate (obj) output directory, we force the build process to use it instead of the original one.

WindowsAzureProject.ccproj
<!– Items for the project –>
<ItemGroup>
  <ServiceDefinition Include="ServiceDefinition.csdef" />
  <ServiceConfiguration Include="ServiceConfiguration.cscfg" />
  <None Include="ServiceConfiguration.Release.cscfg"/>
</ItemGroup>

We also need to add the transform file to our project by manually changing the project file as shown above. Unlike C# and VB projects, Windows Azure Projects in the 1.3 version of the Azure Tools for Visual Studio don’t support the DependentUpon metadata and we can’t make the transform files nested under the configuration file in Solution Explorer. Here is the transformation file that changes our debug version of the service configuration to match the production version shown above.

ServiceConfiguration.Release.cscfg
<?xml version="1.0" encoding="utf-8"?>
<sc:ServiceConfiguration
  xmlns:sc="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration"
  xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">

  <sc:Role name="WebApplication" xdt:Locator="Match(name)">
    <sc:Instances count="2" xdt:Transform="SetAttributes"/>
    <sc:ConfigurationSettings>
      <sc:Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
        value="DefaultEndpointsProtocol=https;AccountName=youraccount;AccountKey=yourkey"
        xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
    </sc:ConfigurationSettings>
  </sc:Role>
</sc:ServiceConfiguration>

In this file we change the number of instances required for our web application to 2 and replace the diagnostics connection string with a string pointing to a Windows Azure storage account.

Notice that we explicitly qualify the service configuration namespace in this transform file. This is necessary in order for the current version of the TransformXml task to correctly recognize and match elements in the transform file to the elements in the original XML file.

Conclusion

Config file transformation offers an effective solution for managing environment-specific configuration settings for internal applications. Compared to other approaches, such as maintaining environment-specific configuration files and section files, config transformation has the advantage of a single file per environment with minimum duplication of common settings. Although this approach comes at the cost of having to use a new XML-based language to define transformations, its integration into the Visual Studio build process and simplicity of ongoing use make it very attractive for internal application development. Although it is only available in Visual Studio 2010 for Web application projects, other project types can be easily customized to take advantage of this functionality.

Just like its alternatives discussed in this article, config file transformation requires environment-specific configuration information to be stored in source control, with the rest of the application source code. Any sensitive information, such as user names and passwords embedded in connection strings or other settings, will be visible to all developers who have access to the source control repository. In order to avoid putting production systems at risk, your first choice should be to use authentication mechanisms, such as Integrated Windows Authentication, that don’t rely on storing secrets such as passwords in the configuration settings. If this is not an option, the next choice is to encrypt the configuration settings as described here. Editing the configuration files manually on each deployment can also be done as a last resort when the organization’s IT policy prevents developers from accessing production environment and from storing production configuration in source control. In such cases config transformation can still be used for continuous deployment to development and test environments the team has access to.

Because config file transformation occurs at build time, as opposed to installation or run-time, it cannot be used for shrink-wrapped software products that will be installed and configured by end-users. For such applications, having a robust and well documented installation and configuration program is still the best option.

Download

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

[…] .Config File Transformation (Oleg Sych) […]

XDT Transformation Tool, based on web transormation. It is useful for build servers, like CCNet.

Changed the approach to use AfterCompile target in order to be compatible with ClickOnce publishing.

Added XML File Transformation section with an example of service configuration transformation

Would it be possible to extend the T4 compiler so that there is a way to transform regular text into a C# string? That is, Similar to the Text Output but converting into a C# string instead of outputting directly to the generated file.

I generate my code by building up a StringBuilder because I use helper functions to generate C# constructs(classes, enums, etc…). This makes it easier to format and represent the code in a TT. The main problem is when I want to directly insert code I must insert all the special characters such as \t, \r\n, etc. This makes it not only difficult to read but to modify. A simple directive that will take the text and format it exactly as is and then represent it as a string literal would make life so much easier. e.g.,

In a tt,

string s = “”;

string s = “This is a string.\r\n\tAnd it is equivalent to …\r\n”;

I’ve always found the config transformations in .net to be too restrictive, I usually have more than one level of variance: if test server 1 and 2 share 90% of the config but differ in 10% I would still need to duplicate the 90% that is common and make one compile for each of them. And how are we supposed to deal with variance from one dev machine to the next? I have recently started using T4 to manage this for me. If you are interested you can read it here: http://ilearnable.net/2010/08/02/t4-for-complex-configuration/

Anyway: I Love your articles on T4! Keep up the good stuff!

The code in section “App.config File Transformation” will cause the following issue with ClickOnce publishing:

“…different computed hash than specified in manifest…”

Seems that the hash for the config file is computed before “AfterCompile” target. What would be the fix for this?

I would like my web.config in my Web Project to be transformed on ALL builds, just like my App.config and Azure service cscfg files.

Is this possible? Any tips on disabling the builtin generation and using this?

Additionally… someone has taken the Target XML necessary for App.config transformations and placed it in a nice includable file: http://exceptionalcode.wordpress.com/2010/06/21/visual-studio-app-config-xml-transformation/

Any chance something similar could happen with the Azure cscfg transformations?

Cole, Unlike for regular apps, where bin\debug folder contains a separate copy of the binaries and configuration files, web applications are built in the same folder. Because of this, I don’t think it is possible to have Web.config transformation done at build time - it would overwrite the original Web.config every time. Unless someone can figure out how to run webdeploy during build and make it fast enough, I think we are out of luck.

Hi oleg,
Thanks for sharing the Code and its not working for ClickOnce App. getting following execption “…different computed hash than specified in manifest…” at the time of installation on Relase Mode.

I am having the same issue as marc above. Seems like it needs to happen before GenerateClickOnceManifests

Very nice to have this in one place like this. Thanks.

I tried this with a library project that is linked in an Azure file… IT totally ignores the transformed config file and re-does the app.config => dll.config conversion again… Any ideas?

Hi,

I have implemented this on my Azure project and localy on my laptop it works, but when building on an other computer the transformation isn’t done.

The only difference is I am using VS Ultimate and the otherones are using VS premium. Is there a difference between the version in combinaction with config transforms?

Hope you can help.

Regards,
Marcel

Re: getting all of the web.config files to compile at runtime.
The easiest way I’ve found to do it, is by simply adding this NuGet package
http://nuget.org/List/Packages/CodeAssassin.ConfigTransform

It only works with web.config files for the moment. But at compile time it will create
web.debug.config.transformed
web.release.config.transformed
web.customconfiguration.config.transformed

[…] However, this makes it difficult to remove the initializer for release builds or to define a different strategy for Integration Tests. Fortunately, you can define the DbContext Initializers in the config file and change or remove them using XLST transformations (For an excellent primer on Web and App config file transformations see Oleg Sych’s article Here). […]

Re App.Config transformations…. If you have no transform specified for Debug configuration, and if you have only one bin\ folder for built DLLs, you must REBUILD to get the Debug config file to overwrite any existing Release config file. Build is not sufficient.

Thanks for this!
Any idea how to make it work in a click-once publish scenario?
The transforms occur, but at the end the app.config file and not the transformed config is copied to the publish directory.

Hi,
I need tt file code to get value from app.config. Existing code looks like below. and it throwing error. Is i hard code the same, its working fine. Please help me on this. Thanks in advance..
string InputsXMLPath = ConfigurationManager.AppSettings[”InputsXMLPath”];
//var doc = XDocument.Load(”c:\\InputControls.xml”);
var doc = XDocument.Load(InputsXMLPath);