Conferences and supporting programme
The Impact of Test Case Quality
When it comes to testing, a lot of effort is spent selecting the “right” testing tool. But often this effort is expended for a second-class goal. Certainly you need a tool that works for you, your development environment, your project, and your process. But what is paramount for good testing is not the testing tool, but the quality of the test cases. Only “good” test cases will find defects in the software. This talk will demonstrate using simple examples, that even “good looking” sets of test cases miss to detect defects, even if the tests achieve 100% code coverage. But how do we develop “good” tests? The requirements are certainly a good starting point. But it is usually not enough to test just the requirements. It is necessary to use test cases incorporating boundary values, extreme values, illegal values, unexpected values. Can such test cases be created automatically from the software to be tested? In my opinion, this makes no sense, e.g. because omissions in the software are not found this way. In my opinion, nowadays (and for some time in the future) it is still the task of a person thinking carefully about the requirements and crafting carefully test cases from them. However, often too many test cases emerge from this process (“combinatorial explosion”). A means against this effect is equivalence partitioning. Equivalence partitioning is highly recommended by ISO 26262 for ASIL B to D as method for deriving test cases. Equivalent partitioning is fundamental for the Classification Tree Method (CTM), a method for test case specification. This method is shortly introduced and it is explained how it reveals, if extreme values are used or not. If 100% code coverage do not guarantee for a high quality of test cases (as stated above), how can we rate the quality of our test cases? Mutation testing (called “error seeding” by IEC 61508) is presented as a solution for this problem.
--- Date: 01.03.2018 Time: 4:00 PM - 4:30 PM Location: Conference Counter NCC Ost