John Cleese
"Testing generates 80% of customer satisfaction..."
Using the term "eXtremal Testing" and a number of books I'm trying to promote the idea of "tester-centered" Offshore Outsourcing. My goal is to show you the key pitfalls in Software Projects, related to testing and verification of Custom Software Product, and to help you to avoid such pitfalls. The article is intended for Teams and Customers of Software Projects - two always-fighting counterparts.
History Of EXtremal Programming, Brief Overview
History of eXtremal Programming (XP) is rather short. XP was invented in the 1990s as a new software development methodology. Kent Beck, a project manager in Daimler-Chrysler, first used it in one of its Software Projects. In fact, XP inherits major concepts of Agile Manifesto, published earlier. In fact, XP is the simplification of classic concepts, created for lazy-bones.
In comparison with more documentation-centric software project management processes, like Rational Unified Process (RUP), Microsoft Solutions Framework (MSF) and IEEE set of standards (SWEBOK), XP is customer and programmer centered. I mean that documentation in XP plays less important role than knowledge, possessed by individuals (Software Development Team members) [myers04].
eXtremal Testing (XT) is the approach to Custom Software Product verification in front of system requirements. eXtremal Testing combines classic methods (system testing, performance testing, stress testing, volume testing, usability testing) and Agile best practices of project management [beck03].
Pitfalls In Custom Software Product Testing
In Software Projects Software Development Team is not communicating with Customer personally and requirements are specified on paper (sometimes with low quality). Using XP/Agile best practices is rather difficult. Cost of quality (cost of Defect removal) is rather high and may impact scope creep.
I see a number of pitfalls in Software Testing, when fixed-cost or time&material contract is outsourced to a remote Software Development Team:
- "Custom Software Product quality mostly depends on Coders' attention and skills" - this is wrong.
- "Tester can't understand the subject area and all Custom Software Product features" - wrong.
- "Customer is the best Tester" - wrong.
Five Steps Of Custom Software Product Verification
I'm sure you know that cost of quality (defined by ASQ as cost of defect removal) increases with Custom Software Product development through phases: requirements, architecture, design, coding, testing, deployment, maintenance. I want to define five major steps in Custom Software Product verification, which should be done by Software Development Team on each iteration, in each phase, for each new build:
- Step 1. Compilation,
- Step 2. Unit-Testing,
- Step 3. Code Peer Review,
- Step 4. System Testing,
- Step 5. User Acceptance Testing (UAT).
Compilation is done by Coders and integrators by means of software compilers. Existing tools (Microsoft Visual Studio, Eclipse, IBM Software Architect and others) are very powerful and help to reveal major code syntax Defects.
Unit-Testing is a done by Coders and unit-tests (testability elements). Test-Driven Development (TDD) explains the approach to unit-testing [beck03].
Code Peer Review is a method of Defect revealing without Custom Software Product running. Peer Review is done by Coders from another Custom Software Development Project [wiegers01].
System Testing (including manual testing and automated testing) is done by Testers and is reported through Defect tracking system. Also called as alpha-testing, this step is performed before showing the Custom Software Product to Customer [kaner99].
User Acceptance Testing (UAT) is performed by Customer and is known as beta-testing. Each Defect reported on this step may cost you money. Your major goal in any Custom Software Development Project is to minimize the amount of Defects found at this step by Customer [kaner02].
Fire Your Coders And Hire Testers
I know that some of the sentences below are provocative. I know that you love your Coders and you respect their work. I know that you feel that they are the people who create the Custom Software Product. This is not true. Start with the small changes:
- Limit Your Coders in Time
- Motivate Testers to Find Defects
- Create Your Coding Manual
Limit Your Coders in Time. If they work until the result is achieved - they will work a lot, they will do their best, they will re-factor code, they will do add extra functionality, they will fill themselves happy and proud of the result. Your Custom Software Development Project will suffer. Limit them in time and ask to implement requirememts as fast as they can - then give them another task. Project manager, don't let your Coders to manager your project! Stop them as soon as functionality is implemented - no matter how many Defects the code contains.
Motivate Testers to Find Defects. Give $5.00 for each new Defect found by your Testers and tell them, that if Custom Software Product doesn't have Defects - they will be fired. Custom Software Product without Defects means that your Software Testing team is wasting your money. Motivate them to crash the Custom Software Product.
Create Your Coding Manual. Create a 10-page document with instructions on how to name variables, how to comment code, how to create cycles, loops, functions, build arrays, etc. Work with this document, not with your "professional" Coders. The time you spend with your Coder is wasted. The time you spend for Coding Manual is your asset. Make sure you award the best Coder - the person who is the most compliant to your Coding Manual.
Published on 6/17/2006
