One of our main requirements when performing the migration was to have the same level of integration with our Team Build system. This means that the build should run all tests and fail if any test fails. Also we should be able to see in the build log which tests have failed.
This step is the most complicated one so here is a quick breakdown of all the steps involved:
- Setting up the build server
- Changing the build template
- Add a new ForEach sequence to iterate through all test assemblies
- Add a sequence to run all tests in a single test assembly
- Invoke NUnit
- Publish NUnit results
- Mark the build as broken if not all tests pass
- Modify the workflow to support projects without a test settings file
- Check in the build template file.
Setting up the build server
The first step is to setup the build server. I have created a folder called C:tfstools with two subdirectories: NUnit and NUnitTfs.
The NUnit folder contains the latest release of NUnit (just the binaries, no installer) and the NUnitTfs folder contains the latest release of NUnitTFS (http://nunit4teambuild.codeplex.com/releases/view/45765). After placing the files in there, make sure that the configuration file for NUnitTFS has the correct server address in each service endpoint.
There are no other configuration steps to be done on the build machine. The following steps must be done for each TFS project that uses NUnit:
Changing the build template
The process to setup this step was mainly inspired by this blog post (http://www.heikura.info/blog/publish-nunit-test-results-as-part-of-team-build-in-team-foundation-server-2010). Unfortunately, by following only the steps there I wasn’t able to setup everything correctly (that post also doesn’t go into how to run all test projects during a build), so after a lot of trial-and-error I finally managed to get the whole process running:
Setup build arguments
Open the DefaultTemplate file and on the bottom click the Arguments button. In the grid that shows up add two entries as in the following screenshot:
- NUnitBinaryDirectory – [In], String, “C:tfstoolsNUnitbinnet-2.0”
- NUnitTfsPublisherDirectory [In], String, “C:tfstoolsNUnitTfs”
Add a new foreach sequence to iterate through all test assemblies
Locate the sequence with the title “If testAssembly.HasTeestSettingsFile” (should be around half-way down on the whole flow).
In the “true” branch of the if statement you should have these two actions:
Insert a ForEach<string>action above them and change the properties as follows:
Add a sequence to run a set of tests
Inside the body of the foreach, add a sequence that will hold all following steps. Change the DisplayName property to “Run NUnit tests”. Select the new sequence and open the variables grid. Add a new variable called NUnitResult, Int32. The scope should be “Run NUnit tests”:
Add the Invoke NUnit action
In the newly created sequence add an InvokeProcess action, and inside the new action add a WriteBuildMessage and a WriteBuildError action like in the following screenshots.
The properties for the WriteBuildMessage and WriteBuildError should look like this:
Add the publish NUnit results action
After the Invoke NUnit action (in the same sequence) add a new Invoke process action. Call it “Publish NUnitTests” and copy paste the inner WriteBuildMessage and WriteBuildError actions from the “Invoke NUnit” action created at the previous step.
Set the Arguments to:
Set the FileName to:
And finally, set the WorkingDirectory to outputDirectory.
Add an if statement that will set the test status to error if the tests don’t pass
At the end of the same sequence add an If statement. Set the condition to NUnitResult <> 0 and in the “Then” branch add an assign action. Set the properties as follows:
The if action should look like this in the end:
If you have followed all the above steps, you should get a sequence as in the following screenshot:
Wait, there’s more
The above steps ensure that the Team Build server will run your NUnit tests when you have a test settings file. You have to do one more step (I promise it’s an easy one ) to get the tests to run also when you don’t have that file.
Scroll a bit above to the “If testAssembly.HasTestSettingsFile” action. Cut the “Run MSTest for Test Assemblies” action from the “Else” branch and replace it with a Sequence. Paste back the action inside the sequence. Then, Copy the “Run all NUnit tests” action from the “Then” branch and paste it above the “Run MSTest for Test Assemblies action”:
The final result of the “Else” branch should look like the screenshot above.
The end (for real now)
Check in the DefaultTemplate.xaml file and you’re done. When you build your project now it should run all NUnit tests as part of the build process.
Check out the next parts of this series here: