Eric Swanson on Technology Services

Today, I presented Microsoft Automated Testing Methodologies & Frameworks for an informal discussion, covering a number of technologies along with a source code demo to highlight most of the popular testing projects.

Three primary discussion questions arose and a final summary question is posed for readers:

1. Is TDD going to take more time?
2. Where do I start?
3. How do I convince my customers and teams TDD is worthwhile?
4. What were some of the key technology discussion points?

Question: Is TDD going to take more time?

Answer: That depends. This is a loaded question. Time for who and to do what exactly? Read more… Time for the developer? The project team? QA? Business analysts? The end-customer? How are you approaching, implementing, and verifying TDD? Also, is quality a measurable output of the product? If you are not going to be around when the bugs show up, what is your incentive to deliver quantifiable quality? And, no, writing tests alone does not guarantee quality. Over time, quality TDD will reduce the total cost of software. It will also reduce the time to fix software bugs. Good TDD makes certain bugs NEVER reappear (I make it an explicit point to write a test for every bug at a minimum). Tying TDD to business requirements minimizes the impact of developer assumptions on the product. Mistakes will still happen and the business requirements can also be flawed from the start or through translation. Writing tests as a specification before the implementation further reduces impacts by forcing the developer to understand and codify requirements up-front before fullfilling them through implementation.

Question: Where do I start?

A: Where do you start without TDD? Hopefully the answer is business requirements or technical requirements tied to business requirements.

• For non-developers, evangelize TDD and enable developers in whatever way you can.
• For the developer, start with a requirement that can be solved in minutes (break down the task), open up a code file, write a few lines of code that will call a non-existant code API that could do what is expected (you are codifying the actual design of your API explicitly as a test client of the API). Now, code the implementation. Compile and run the test, tweaking code until the initial test expectations of the API are met (and nothing extra!). Next requirement… Of course, supporting software will influence the actual approach taken.
  *For bug fixes, write a test to verify the bug exists and then modify the software to fix the bug.

Question: How do I convince my customers and teams TDD is worthwhile?

A: There are a lot of variables here…

1. The Bug Argument: The simplist, primary argument is this: How does your customer / team ensure that a bug never reappears in the software after it has been fixed?
2. The Reverse-Maintenance Argument: Developers will utilize and modify code that wasn’t authored by them, or that they wrote, but have become unfamiliar with over time. When they use or modify the code, they will do so with certain and often necessary assumptions (conscious or subconscious). Assumptions, especially under pressure, are ripe with potential for creating bugs. If the code being modified has associated tests, the developer can test and verify their assumptions against existing expectations of the software; expectations as business requirements, technical requirements, as well as the explicit and codified assumptions of previous authors.
3. The Forward-Maintenance Argument: Developers are forced to consider future maintenance of their solutions and document expectations and assumptions up-front, because writing test clients forces developers to use the implementation API from a different perspective - the perspective of consumers and other developers.

Question: What were some of the key technology discussion points?

A: …

• Microsoft “Code Contracts” was emphasized for two reasons: 1) It allows developers to state expectations and assumptions as embedded code specifications that can be optionally verified under specified runtime configurations and optionally at compile-time through static analysis. 2) It will be included in .NET 4.0 and Visual Studio 2010.
• Fitnesse.NET opened eyes toward better testing integration throughout the development lifecycle and greater testing visibility for QA and optionally the business analysts and/or stakeholders.

View: Microsoft Automated Testing Methodologies & Frameworks