When most people think of automated tests they picture automating what a manual tester does in running the tests. Sometimes this is what we desire, but it isn’t the most powerful way to use test automation. This tutorial is about extending our reach to do testing that cannot be done manually; working beyond the automated execution of manual tests by evaluating massive volumes of data, monitoring program activities not observable by humans, or exploring ever-changing areas of the program. This tutorial covers these types of tests with a focus on exploratory automated tests (EAT) and automated test oracles for evaluating software behavior for the tests.
EAT is a testing approach that uses the power of the computer to look for bugs that functional testing misses. Unlike regression tests that do the same thing each time they are run, exploratory tests do something different each time. Through the power of the computer, many of these tests can run and check millions or billions of functions – numbers unthinkable with manual tests, automated scripts, or even table-driven automation. These techniques go after bugs that are virtually impossible to expose or isolate in manual testing.
Automated test oracles increase the power of the tests by potentially exposing multiple types of errors, often without modifying the tests themselves.