Why Bugs Escape

A test team’s worst nightmare is having a flood of critical bugs discovered by customers at a new release of a product. It seems that you can never test too much nor can you test everything. So it is always a fine balance, driven by many factors, of when to stop testing and deliver the software to the customer. But invariably bugs escape—it is only the quantity and severity that we, as testers, can hopefully control.

First we can start with attempting to understand why bugs escape testing by looking at how we test in relation to how customers use our product. Concepts from complexity and chaos theories will be introduced in light of the variables used within testing. Then some ideas on how to do effective root cause analysis with the bugs that customers do find and report will be discussed. Additional notions on mitigating the escapes by reviewing the testing approaches and techniques will be illustrated.

The takeaways from this presentation include approaches to root cause analysis, inclusion of chaos into testing and thoughts on viewing our products through the customer’s eyes.

Session Takeaways:

  • An understanding of some of the reasons why bugs escape testing in spite of best intentions.
  • How to mitigate the escapes by learning from customers.
  • How to incorporate some concepts of complexity theory to improve the testing process.
  • Ideas on the potential dangers of being too lean in the testing approach.

Agile Test Strategy
Location: Wayland Date: October 7, 2015 Time: 2:30 pm - 3:30 pm Jim Sivak Jim Sivak