Open
Description
Assume
- The user assesses test quality by ensuring that each test kills some mutant.
- The user has written a test case that contains both correct and incorrect assertions.
- When running mutation testing, some assertions kill mutants.
Problem: Incorrect assertions will not be detected.
Example 1: Incorrect test evaluation
EXPECT_EQ(some_actual_value_returned_from_the_sut, some_expected_value_set_by_the_test); // OK
EXPECT_EQ(some_expected_value_set_by_the_test, some_expected_value_set_by_the_test); // NOT OK
Example 2: Incorrect pre-condition evaluation
ASSERT_EQ(some_pre_condition, some_pre_condition); // NOT OK
Example 3: Overgenerelized test code
if (some_cond) {
EXPECT_EQ(...); // Is this executed?
}
if (some_other_cond) {
EXPECT_EQ(...); // Is this executed?
}
if (some_third_cond) {
EXPECT_EQ(...); // Is this executed?
}
Example 4: Complex test code
for (cond){
for (cond) {
if (some_cond) {
if (some_other_cond) {
if (some_third_cond) {
EXPECT_EQ(...); // Is this executed?
} else
EXPECT_EQ(...); // Is this executed?
}
}
}
}
}
Example 5: Test code that can be removed
// TODO: Remove depricated EXPECT_EQ when feature X is removed
Feature_X = getDepricatedFeature();
if (Feature_X.active == True){
EXPECT_EQ(...); // Is this executed?
}
Suggestion:
Add functionality that lets the user verify that each assertion in the test is able to detect a bug.