Willy-Peter Schaub's Cave of Chamomile Simplicity

VSTS 2008 Team Test ... a few notes and thoughts from our internal workshop

Overview

We covered VSTS Team test features using the great training material created by Brian Randell (MVP) and in essence the test environment currently supports the following features:

• Unit Testing by adding attributes to classes and methods to be tested.
• Web Testing allowing us to record a set of web actions, replaying, extracting and validating information from the HTTP traffic.
• Load Testing which can execute any type of tests, based on user load and time, for example; capturing of performance counters and distributing load across servers.
• Manual Testing, which documents manual test steps and allows tester to capture a pass/fail result. The manual tests can then be published against build. Watch a great video by Chris Menegay (MVP) that covers the basics of manual testing, or track his blog.
  
• Generic Testing, which caters for all the non-MS-VSTS test options, allowing us to create generic tests that integrate with the test harness.
• Ordered Testing, which combines different test types in a specific order.
• Code Coverage, which assesses the effectiveness of tests and provides line-by-line execution details.
  
• We can define the colours to be used in the Visual Studio options:
  
• The standard colours are as follows:
• Coverage Not Touched RED
• Coverage Partially Touched YELLOW
• Coverage Touched BLUE

• We can also re-sign assemblies after instrumentation:
  

Some issues/questions/wishes that were raised during our test session

Lack of out-of-the-box

• testing tools for rich and smart clients.
• load testing for WCF based solutions ... see WCF Load Test for a great VSTS Rangers product.
• complete test cycle management, i.e. use case -> test case -> test code -> dev -> testing -> match issues against use|test cases, or determine impact of code changes on use|test cases, i.e. feature set.

References

• TFS - Using the Team Edition (2008) for Testers to automate the drudgery
• WCF Load Test
• How Do I ... Videos

Conclusion

The general feeling is that the VSTS Test tools are comprehensive and invaluable if used in a managed test environment. Have a look at the test tools and watch the space for forthcoming posts on Rosario features, which may or may not address some or all of the concerns raised so far.

Today we are covering Team Build and TFS integration with non-Microsoft environments today. I will summarise and post my notes tonight.