We're paid to create business value (amen).
Finding bugs does not create any business value.
Fixing bugs creates the business value.
Preventing bugs creates more business value.
Brian Marick's test type matrix (http://www.testing.com/cgi-bin/blog/2003/08/22).
A lot of bugs are found through exploratory testing (Business Facing & Critiquing the Product).
Depending on your requirements (business, industry, etc.) some types of testing may be more important to start early (i.e. security testing for defense industry software).
Tests tell us where we are from a business point of view.
If we're here to primarily provide business value, we need to be concerned with what the issues that endanger providing. Test what's risky.
Risks
- Handling Schedule Risk
- Use short iterations
- allows to measure the completion of work in a fixed amount of time
- not possible without all different kinds of tests
- think about breaking down iterations into sub releases
- iteration includes the following tests
- acceptance
- usability
- exploratory
- unit
- property
- shipping an iteration should not mean it's been passed to testing. It should be done, done, done.
- nothing happens outside an iteration
- don't reopen your iteration planning once you've set it and started work on it
- If it's not in the iteration you don't do it
- testing is a lifecycle issue
Customers don't know what they need and they're horrible at prioritizing. If you can't ship, you've failed.
Unit tests while writing code. Acceptance tests as soon as part of the feature works. Because of this you'll have lots of tests. Decide what to run and when to run them. This is for unit and integration tests. i.e. What is run locally by the dev, nightly, at checkin, etc.
Regressions are everyone's nightmare. Automating helps a lot for keeping bugs fixed and not letting them creep back into the process. They can happen because of lack of developer's running all the tests all the time. Automated unit, Fit and acceptance tests can help, but not with security, performance or usability.
Test interactions with mock objects. You still have to run all the tests all the time or you won't see the benefits. Decrease coupling (aka. designing for testability). Focus on smaller things.
Test Coverage is a metric that can be deceiving. You have 100% coverage, but did you write all the code for all the features? How good are the tests creating the coverage.
Nail down the testing requirements prior to working on the code. If you don't have clear expectations you can not work towards meeting or exceeding them.
Make some of the testing (i.e. performance) part of every iteration.
Shipping is bigger than just features.
posted @ Wednesday, May 16, 2007 11:35 AM