Part 1: Continuous Integration using MSBuild, CruiseControl.NET, FxCop, NUnit, NCover + Subversion
I've spend the last week or so refactoring our Continuous Integration (CI) process at work. Our current build was causing quite a few headaches - from not showing the build stats correctly to being too slow etc. I decided to redo the whole CI process and thought I'd document the process as a series of blog posts to serve as a guide for creating a CI process using MSBuild, VS 2005, InstallShield, NCover, NDepend and Mercury Quick Test Professional together with open source tools like CruiseControl.NET, FxCop, NUnit, Sandcastle and Subversion.
The series consists out of the following posts:
- Part 1 covers the background, requirements, process and tools required for the whole CI process.
- Part 2 covers the common build targets and tasks that are used by the DeveloperBuild, DeploymentBuild, CodeStatisticsBuild, CodeDocumentationBuild and QtpBuild
- Part 3 covers the DeveloperBuild
- Part 4 covers the DeploymentBuild
- Part 5 covers the CodeStatisticsBuild
- Part 6 covers the CodeDocumentationBuild
- Part 7 covers the QtpBuild
Lastly I end off the series by showing you how to use some additional community extensions to add some further panache to your CI build.
You can also find a complete, downloadable pdf booklet of the series here 
.
Change History
- 12/12/2007: Added the GetEnvironment target to InitialTargets; Changed environment specific builds of DeploymentBuild to also depend upon the Environment.txt file.
- 11/12/2007: Changed the CodeStatisticsBuild to run NDepend as part of the build; Consolidated all the targets into a single build file; Updated the CC.NET configuration to make use of new build queue features available in CC.NET 1.3 and removed duplication from the CC.NET configuration through using DTD entity references.
- 28/09/2007: Changed the QtpBuild to make use of incremental builds to install/uninstall the application and to run the test cases, identified in a file, from within Mercury Quality Center.
- 17/09/2007: Added the QtpBuild and updated the series to include links to the new content.
- 31/08/2007: Added the link to the downloadable booklet for the complete series.
- 17/08/2007: Updated content to include the CodeDocumentationBuild and the link to the CCStatistics post.
- 25/04/2007: Fixed broken CruiseControl.NET link.
Resources
I found the following resources helpful for getting to grips with the power of MSBuild:
- Book: Deploying .NET Applications: Learning MSBuild and ClickOnce
- Channel 9 Wiki: MSBuild.Links
- Channel 9 Wiki: MSBuild.HomePage
- MSDN Library: MSBuild Overview
- MSDN Library: MSBuild Reference
- Code: MSBuild Community Tasks
The
documentation for CruiseControl.NET was sufficient for setting up the build server to use CC.NET with MSBuild.
Background
Before delving into the CI process itself, I need to provide some background on our application to create the context for our CI process. The application, developed using VS 2005, is a stand-alone WinForms application with integration points to various other systems/applications via web services/third party API's. We use Subversion for source control and some third party libraries/controls from vendors like Infragistics, Xpo. We have tried to develop the application using Agile practices like TDD and we are currently running on 1 week iterations. At the end of each iteration we produce a build (.msi) that is released to our testers. We treat all our databases as just-another-code-artifact and have scripts that allow us to create the databases' structure and content. We are able to at any point in time setup a new development environment by simple getting the latest version from our repository.
To facilitate all of this, we have standardized on using the following directory structure for our solution:
- Builds folder contains a separate folder for every build and its build artefacts.
- Code folder contains all our VS 2005 projects as a flat hierarchy of sub-folders. All the .sln files reside in the Code root folder. We have a Code\Deploy sub-folder to which all the non-unit test project outputs are compiled.
- Code Metrics folder contains all the metrics that we generate for our system like the code coverage results.
- Documentation folder contains the help file settings and MSDN style help file we generate from our XML code comments.
- Install folder contains our InstallShield .ism file and various merge modules and other files required to create a .msi install for our system.
- Lib folder contains all the third party libraries (Infragistics, NUnit, TypeMock, XPO etc.) that are used as project references.
- Qtp folder contains the RunQTP.vbs script file and a TestCases sub-folder that contains a list of test suite files. Each test suite file contains a list of individual Qtp test cases to run.
- Sql folder contains a sub-folder for every database that we use. Each database sub-folder contains all the .sql script files to create the structure and content for the database as well as batch files to automate this creation process using osql/sqlcmd. We support both sql 2000 + sql 2005.
Process
One of the requirements for our CI process is that a developer should be able harvest the same process as the build server to work in his/her sandbox/private workspace before committing their changes to the repository. The build server will obviously use a few additional tasks for deployment, but the same compile/test process should be re-usable from within both the developer sandbox/private workspace and build server.
Our build needs to do the following tasks:
- Get the latest source from the repository
- Build the databases
- Compile the source
- Run the unit tests
- Produce code coverage results
- Produce FxCop results
- Produce NDepend results
- Build MSDN style help documentation
- Backup the databases
- Version the source code
- Build a msi
- Deploy the msi
- Run the regression test pack using QTP
- Notify QTP testers via e-mail
- Tag the repository
Not all of these tasks need to be completed on a continuous basis. Some (like steps 9-15) are only required to run once weekly as part of the iteration build to create the .msi. We also found that creating MSDN style documentation as well as running code coverage and code analysis on every check-in caused our solution build, that currently consists out of 83 projects, to take way too long. We therefore decided to create a staged build/build pipeline and to split our CI build process into 5 separate builds on two separate servers.
- DeveloperBuild (Runs on BuildServer) - This build is set to monitor the repository every 60 seconds for changes. It builds the databases, compiles the source code and runs the unit tests. It does incremental builds for building the databases, compiling the source code and for running the tests to get quicker build times.
- DeploymentBuild (Runs on BuildServer) - The deployment build is run once weekly at 12:30 on Friday afternoons to create the .msi installation that is deployed to our QTP testers. In addition to creating and deploying the .msi it also tags the repository to create a tag for the work done for the iteration.
- CodeStatisticsBuild (Runs on BuildServer) - The statistics build is set to run on successful completion of the deployment build and produces the Code Coverage, FxCop and NDepend results for the week. The developers can still evaluate the code coverage, FxCop and NDepend results continuously in their sandbox, but we incur the overhead of this on the build server only once a week. This way we can have the stats to spot trends on a weekly basis.
- CodeDocumentationBuild (Runs on BuildServer) - The documentation build is set to be invoked manually and creates MSDN style help documentation from the XML code comments.
- QtpBuild (Runs on QtpServer) - The QtpBuild runs on a dedicated machine as the regression test pack takes quite some time to run through (3 hours in our scenario). The build triggers on the completion of a successful DeploymentBuild on the normal build server.
As evident, the first server (aka the BuildServer) will do all builds except for the QtpBuild. The second server (aka the QtpServer) will only run the regression test pack using QTP.
Build Server Tools
These are the tools that are used by our CI process and that needs to be installed on our build server running Windows Server 2003:
- Visual Studio 2005 Team Edition for Software Developers to run Managed Code Analysis (aarrgghhhh!). You can use the standalone version of FxCop, but there are some differences between this and the version used for VS 2005 Code Analysis. Unfortunately you therefore need to install the complete VS 2005 Team Edition for Software Developers.
- NCover v1.5.8 for Code coverage.
- NCoverExplorer v1.4.0.7 and NCoverExplorer.Extras v1.4.0.5 to create Code coverage reports.
- NDepend v2.6 for additional code metrics.
- MSBuild.Community.Tasks for some additional MSBuild tasks.
- NUnit 2.0 2.4.5 to run the unit tests.
- .NET Framework 2.0
- .NET Framework SDK 2.0
- CruiseControl .NET 1.3
- InstallShield Standalone Build v12 - The standalone build is the InstallShield build engine without the GUI and you are allowed to install this on a build server for free granted you have a valid InstallShield license.
- Subversion 1.4.5 for source control.
- HTML Help Workshop for building HTML help 1.x files.
- Visual Studio .NET Help Integration Kit 2003 for building help files that can integrate into Visual Studio.
- Sandcastle June 2007 CTP to generate the help file content from the XML code comments.
- Sandcastle Help File Builder (SHFB) to set the options for the help file and to bootstrap the document creation process.
- Sandcastle Presentation File Patches to solve some issues with the Sandcastle June CTP. Extract the contents of the ZIP archive to the Sandcastle installation directory.
- Mercury QuickTest Professional 8.2 for automated regression testing.
Next Steps
Well, that takes care of the all the background information. The next post will delve into all the common build targets that are used by the DeveloperBuild, DeploymentBuild, CodeStatisticsBuild, CodeDocumentationBuild and QtpBuild. Keep watching this space... 
B.t.w, I'm a big fan of ReSharper and the MsBuild/NAnt support of ReSharper is oh so sweet!!