Running xUnit tests as a part of a TFS 2010 build

Getting TFS 2010 to include xUnit tests in its build process can be a tricky process. I recently used this post by Karsten Strøbæk to edit several of our continuous integration builds to do just that and thought I would expand upon and update Karsten’s process.

Use the latest version of NUnit for Team Build

The first thing you’ll need to do is install xUnit and NUnit for Team Build. Several changes have been made to the latter since the last release was done so I recommend you download the source code and do a build from that rather than downloading the binaries.

Use .Net 4 version of xUnit

If you’re using .Net 4 or above you’ll want to use xunit.console.clr4.exe instead of xunit.console.exe.

Target TFS 2010

One of the things that has been (re-)added to NUnit for Team Build is the version command line argument. This allows you to target either TFS 2008 or TFS 2010. We’ll assume you’re using TFS 2010, so you will need to change the arguments for the Nunit for Team Build invoke process activity in the build workflow, like so:

    String.Format(
        "-n {0} -t {1} -p ""{2}"" -f {3} -b ""{4}"" -v 2010",
        "results.xml",
        BuildDetail.TeamProject,
        BuildSettings.PlatformConfigurations(0).Platform,
        BuildSettings.PlatformConfigurations(0).Configuration,
        BuildDetail.BuildNumber
)

Set build to Failed or Partial Success when tests fail

You can use the xUnitResult variable, which contains the return value from the xUnit console, to set the build phase status and build status. To do this, immediately after the NUnit for Team Build invoke process add an If activity with condition “xUnitResult > 0″. In the “Then” area add a SetBuildProperties activity and set the following:

Execute tests in multiple assemblies

The biggest change I made from Karsten’s original post was adding support for tests in multiple assemblies. The xUnit console can only run tests from a single assembly, so in order to do this you need to edit the build workflow to loop through your test assemblies, executing xUnit and NUnit for Team Build for each.

The workflow becomes quite busy, but here is the basic structure of the “Run Tests” activity.

Test Results

Firstly, a FindMatchingFiles activity is used to find test assemblies. I added an argument to the workflow called xUnitAssembly pattern and then used this in the MatchPattern property of the activity like so:

    String.Format(
        "{0}\\{1}",
        outputDirectory,
        xUnitAssemblyPattern
    )

I set the default value of the xUnitAssemblyPattern argument to “*.Tests.dll” as that matched the convention for most of our projects.

Then we use a ForEach activity to loop through the found assemblies, executing xUnit console and NUnit for Team Build for each. The arguments for both of these activities are tweaked to take into account the fact there are now multiple test assemblies.

The xUnit activity’s arguments now become:

    String.Format(
        """{0}"" /silent /nunit ""{0}.xml""",
        assemblyName
    )

Where assemblyName is the current test assembly filenmae. Similarly NUnit for Team Build’s arguments are updated to read this file:

    String.Format(
        "-n ""{0}.xml"" -t {1} -p ""{2}"" -f {3} -b ""{4}"" -v 2010",
        assemblyName,
        BuildDetail.TeamProject,
        BuildSettings.PlatformConfigurations(0).Platform,
        BuildSettings.PlatformConfigurations(0).Configuration,
        BuildDetail.BuildNumber
    )

Finally, our logic to fail / partially succeed the build needs is tweaked so that it will only invoke the SetBuildProperties activity if the build hasn’t already been failed / partially succeeded.

Now when your build completes you should see the results from all of your test assemblies listed:

Test Results

(Tools exist that can combine the results of multiple NUnit test runs if you wanted all of your tests to appear as a single run.)

A warning about build numbers

One problem I ran into was that we have multiple solutions in a Team Project, and they all use the same build number format meaning that builds from different definitions could have the same build number. This caused problems for NUnit for Team Build which uses the build number to post your test results – if multiple builds exist in the same team project with the same build number, even if they are from different build definitions, this will cause an error. Updating the build numbers to include a project reference solved the issue easily enough.

Comments