December 2007 - Posts - From the software development trenches

December 2007 - Posts

Izindaba #21
07 December 07 05:35 AM | cjlotz | with no comments

image91235Time for another weekly round-up of developer news that focuses on .NET, agile and general development content.  (Izindaba = Zulu for News).

.NET

  1. Parallel Extensions to the .NET Framework CTP - News from the BCL Team on the CTP and newly created Parallel Computing Developer Center.
  2. ASP.NET MVC Framework (Part 2): URL Routing and ASP.NET MVC Framework (Part 3): Passing ViewData from Controllers to Views - Scott Guthrie continues with his series on the new ASP.NET MVC Framework. CTP coming soon! smile_shades
  3. ASP.NET AJAX Control Toolkit and Web Deployment Project Releases for VS 2008 - Scott Guthrie announces the updates that work with VS 2008 and .NET 3.5.
  4. Should I (Still) Learn 1.0?  - Jesse Liberty reflects on whether it is worth learning Silverlight 1.0.
  5. Improve your C#! Borrow from F# - Dustin Campbell shows how to create more elegant C# code by borrowing concepts from F#.
  6. Visual Studio 2008 and .NET Framework 3.5 Training Kit - Hands-on-labs, presentations and demos on all things new in VS 2008 and .NET 3.5.  Great for getting your teams up to speed with the new technologies.  clap
  7. A Detailed look at Overriding the Equality Operator - InfoQ shows how to implement this correctly.

Design

  1. Deconstructing Software - Patrick Smacchia continues with another post that illustrates how to use NDepend to ensure that your code is really layered as it should be.
  2. A Microcosm of Agile Design - Jeremy Miller goes about explaining and defending Agile Design principles.

SDLC

  1. Do Dedicated Team Rooms Make for More Productive Teams? - InfoQ looks at answering the question.

SQL Server

  1. New File Storage Mechanism for SQL Server - Some SQL Server news from InfoQ.

Software

  1. VSTS Web Access Power Tool for Team System 2008 released
  2. ADO.NET Entity Framework Beta 3 Released - New visual designer capabilities, bug fixes, performance improvements and feature requests.  Also see the list of breaking changes.
  3. Expression Blend SP1 - Still no integration with TFS.
  4. VisualSVN 1.3.2 - Bug fixes.
  5. MbUnit V3 and Galileo alpha v1 - Andrew Stopford announces the exciting progress being made towards a major update to MbUnit.
  6. TeamCity 3.0 Released - JetBrains announces a new version of this powerful continuous integration and build management solution for Java and .NET.  A free version is available for small teams.
  7. Mono on OSX, Win32 - Miguel de Icaza announces some plans for improved Mono support on OSX.
  8. MSBuild Wish List vs FinalBuilder Feature Set - Folks at VSoft Technologies illustrate that the excellent FinalBuilder already provides most of the features requested recently by MSBuild users.
  9. Package your Own MSDN Library content - GUI Tool for making personalized e-books (.chm and .hxs format) from the MSDN Library or TechNet Library content.

Misc

  1. Selection Criteria for JavaScript Frameworks
  2. December '07 DevDiv Dogfood Statistics - Just in case you were worried about TFS scaling smile_wink
  3. Shuffling - Jeff Atwood looks as the most efficient algorithm for shuffling a deck of cards.

Happy reading smile_regular

Continuous Integration: From Theory to Practice
03 December 07 08:08 PM | cjlotz | 9 comment(s)

image Continuous Integration (CI) is a popular incremental integration process whereby each change made to the system is integrated into a latest build. These integration points can occur continuously on every commit to the repository or on regular intervals like every 30 minutes.  They should however not take longer than a day i.e. you need to integrate at least once daily.

In this article I take a closer look at CI.  The article is divided into two main sections: theory and practice.  In the theory section, I consider some CI best practices; look at the benefits of integrating frequently and reflect on some recommendations for introducing CI into your environment.  The practice section provides an in-depth example of how to implement a CI process using .NET development tooling.  I conclude the article by providing some additional, recommended reading.

Change History

References

I used the following references in creating the article:

Practices

Martin Fowler presents the following set of practices for CI [Fowler]:

  • Maintain a single source repository - Use a decent source code management system and make sure its location is well known as the place where everyone can go to get source code.  Also ensure that everything is put into the repository (test scripts, database schema, third party libraries etc.)
  • Automate the build - Make sure you can build and launch your system using MSBuild/NAnt scripts in a single command.  Include everything into your build.  As a simple rule of thumb: anyone should be able to bring in a clean machine, check the sources out of the repository and issue a single command to have a running system on their machine.
  • Make your build self-testing - Include automated tests in your build process. 
  • Everyone commits every day - "A commit a day keeps the integration woes away" [Duvall].  Frequent commits encourage developers to break down their work into small chunks of a few hours each. Before committing their code, they need to update their working copy with the mainline, resolve any conflicts and ensure that everything still works fine.  Jeremy Miller refers to this process as the "Check In Dance" [Miller].
  • Every commit should build the mainline on an integration machine - Use a CI server like or CruiseControl.NET or Team Foundation Build.  If the mainline build fails, it needs to be fixed right away.
  • Keep the build fast - If the build takes too long to execute, try and create a staged build/build pipeline where multiple builds are done in a sequence.  The first build (aka ''commit build'') executes quickly and gives other people confidence that they can work with the code in the repository.  Further builds can run additional, slower running unit tests, create code metrics, check the code against coding standards, create documentation etc.
  • Test in a clone of the production environment - Try to set up your test environment to be as exact a mimic of your production environment as possible.  Use the same versions of third party software, the same operating system version etc.   Consider using virtualization to make it easy to put together test environments.
  • Make it easy for anyone to get the latest executable - Make sure that there is a well known place where people can find the latest executable.
  • Everyone can see what's happening - Make the state of the mainline build as visible as possible. Use various feedback mechanisms to relate build status information [Duvall].  Some fun examples include using lava lamps, a big screen LCD and an ambient orb.
  • Automate deployment - Have scripts that allow you to automatically deploy into different environments, including production.  For web applications, consider deploying a trial build to a subset of users before deploying to the full user base.

Jeremy Miller adds the following advice [Miller]:

  • Check in as often as you can - Try breaking down your workload into meaningful chunks of code and integrate these pieces of code when you have a collection of code in a consistent state. Checking in once a week seriously compromises the effectiveness of a CI process.
  • Don't leave the build broken overnight - Developers need to be immediately notified upon a build breakage and make it a top priority to fix a broken build.
  • Don't ever check into a busted build.
  • If you are working on fixing the build, let the rest of the team know.
  • Every developer needs to know how to execute a build locally and troubleshoot a broken build.

Paul Duvall also warns against some additional CI anti-patterns that tend to produce adverse effects [Duvall]. The ones that have not been covered above are:

  • The cold shoulder of spam feedback - Team members sometimes quickly become inundated with build status e-mails (success and failure and everything in between) to the point where they start to ignore messages. Try to make the feedback succinctly targeted so that people don't receive irrelevant information.
  • Don't delay feedback with a slow machine - Get a build machine that has optimal disk speed, processor, and RAM resources for speedy builds.

Benefits

Numerous benefits result from integrating continuously [McConnell]:

  • Errors are easier to locate - New problems can be narrowed down to the small part that was recently integrated.
  • Improved team morale - Programmers see early results from their work.
  • Better customer relations - Customers like signs of progress and incremental builds provide signs of progress frequently.
  • More reliable schedule estimates & more accurate status reporting - Management gets a better sense of progress than the typical "coding is 99% percent complete" message.
  • Units of the system are tested more fully - As integration starts early in the project the code is exercised as part of the overall system more frequently.
  • Work that sometimes surfaces unexpectedly at the end of a project is exposed early on

Where do I start?

Here are some steps to consider for introducing a CI process [Fowler]:

  • Get the build automated - Get everything into source control and make sure you can build the whole system with a single command.
  • Introduce automated testing in the build - Identify major areas where things go wrong and start adding automated tests to expose these failures.
  • Speed up the build - Try aiming at creating a build that runs to completion within ten minutes.  Constantly monitor your build and take action as soon as your start going slower than the ten minute rule.
  • Start all new projects with CI from the beginning

.NET Implementation & Tools

Earlier this year I wrote a set of articles that describe a complete setup of a .NET CI solution.  The CI process uses a wide variety of tools including CruiseControl.NET, Subversion, MSBuild, Visual Studio 2005, NUnit, FxCop, NCover, InstallShield, SandCastle and QTP.  You can download the series as a fully searchable PDF with PDF bookmarks and PDF links.

Team Foundation Server users can check out the new and improved out-of-the-box support for continuous integration provided by Team Foundation Server 2008.  This, coupled with the recent announcement on the TFS Licensing Change for TFS 2008, makes using Team Foundation Server less expensive and a more viable option to consider.

Recommended Reading

  1. Continuous Integration: Improving Software Quality and Reducing Risk by Paul Duvall.
  2. Software Configuration Management Patterns by Steve Berczuk and Brad Appleton.
  3. Refactoring Databases by Scott Ambler and Pramodkumar Sadalage.
  4. Visual Studio Team System: Better Software Development for Agile Teams by Will Stott and James Newkirk.