Clear testing goals

Question 1

Software Metrics

Answer 1

System or standard of measurement which gives a value to some property

Question 2

Coverage

Answer 2

A measure of proportion of a structure that a program, test case, test suite exercises.

Question 3

List test coverage methods based on rigour from low to high.

Answer 3

1. Statement coverage
2. Decision coverage
3. Condition Coverage
4. Condition/Decision Coverage.
5. Modified Condition/decision coverage.

Question 4

What can analysis of structural coverage discover?

Answer 4

• Shortcomings in requirements-based test cases
• inadequacies in requirements
• dead code/deactivated code
• unintended functionalities.

Question 5

Path coverage

Answer 5

Test suite achieves path coverage if it results in every path in the program taken at least once.

Question 6

Path coverage v. Exhaustive testing

Answer 6

Not the same. Path coverage ensures each control flow path is seen once. not that all paths have seen all possible values.

Question 7

Statement Coverage

Answer 7

Test suite achieves Statement coverage if it results in each statement in the program being run at least once.

Question 8

Statement coverage for:
int abs(int x) {
    int ans = x;
    if x < 0 {
       ans = -x
    }
    return ans;
}

Answer 8

test case: abs(-4).

Hits every statement in the code but doesn’t hit every branch in the program.

Question 9

Branch Coverage

Answer 9

Test suite achieves branch coverage if it results in each branch in the program being run at least once.

Question 10

Branch coverage for:
int abs(int x) {
    int ans = x;
    if x < 0 {
       ans = -x
    }
    return ans;
}

Answer 10

test case: abs(-4)
test case: abs(4)

All branches taken in the program.

Question 11

Path coverage v. Branch coverage?

Answer 11

Path coverage ensures each control flow path is taken.
Branch coverage ensures that each INDIVIDUAL branch is taken.
But there are combinations of branches that may not have been seen!

Question 12

Condition Coverage

Answer 12

Test suite achieves path coverage if it results in each condition in each branching instruction being both true and false. (true, true) and (false, false) for if (a || b)

Question 13

Combinatorial coverage

Answer 13

Test suite achieves combinatorial coverage if it results in each condition in each branching instruction being tried in all combinations of truth values.

Question 14

What is required for combinatorial coverage:

if (avrageMark >= 50 or hardFailCount == 0 or elephantCount > 3) {
…

Answer 14

(50, 0, 4) = (true,true,true)
(50, 0, 3) = (true,true,false)
(50, 1, 4) = (true, false, true)
…

Question 15

Modified Condition/Decision Coverage

Answer 15

100% branch coverage
100% condition coverage
each entry/exit point is exercised
each condition affects the behaviour independently.

if (a or b)
(TF), (FT), (FF)

Question 16

Limitations of MC/DC

Answer 16

• For large number of inputs, A LOT OF TEST CASES.
• MC/DC is expensive and not a lot of evidence that it is valuable
• Used as a check box. Have you done it or not?

Question 17

Source vs Object code coverage

Answer 17

Structural coverage at source level can differ from that at the object code level. Ie. multiple object code statements can be generated from a single source code statetement.

Question 18

Structural coverage vs structural testing
SC: Statement, Decision, MC/DC
ST: Statement, branch, Condition, Path

Answer 18

SC: determine which code structure was not exercised by the requirements based test procedures.
ST: process of exercising software with test scenarios written from the source code not from requirements.

Question 19

Structural testing limitations

Answer 19

provides no info about whether the code is doing what it’s supposed to be doing as specified in the requirements.

Question 20

What is Criteria Subsumption

Answer 20

A Subsumes B if every test suite that satisfies A also satisfies B.

Branch coverage subsumes statement coverage.

Question 21

If A subsumes B, will test suite satisfying A find all failures for B than a test suite satisfying B?

Answer 21

NO! in general this is true that test suites meeting subsuming criteria do find more failures but no guarantee.

Question 22

Does test suite for branch coverage find all failures that a test suite satisfying statement coverage does?

int abs(int x) {
  int ans = x;
  if (x < 2) {
    ans = -x
  }
  return ans;
}

Answer 22

test suite for branch coverage:
abs(10) = 10 pass
abs(-10) = 10 pass
achieves both statement and branch coverage.

test suite for statement coverage:
abs(-1) = -1 fail