Verification

Question 1

What is validation?

Answer 1

Validation ensures that the software systems meets the customers expectations

(are we building the right system?)

Question 2

What is verification?

Answer 2

Verification ensures correctness against system specification

(are we building the system right?)

Question 3

Static Verification Techniques

Answer 3

do not require executing the program

• code review
• static checking (automated analyses)
• formal verification (program satisfies formal property)

Question 4

Dynamic Verification Techniques

Answer 4

require a program to be executed

• Testing
• runtime monitoring (check for safety/security vialoations)

Question 5

Code review - definition

Answer 5

structured inspection process, performed in a team, to find possible defects

Question 6

Code review - advantages and obstacles

Answer 6

+ distribute knowledge among team
+ can find defects before they show in tests
+ can improve code quality
+ no executable system required

• only work well when properly conducted
• team members may feel criticized or not productive

Question 7

Steps in test Design

Answer 7

1. identify and analyze the responsibilities of the IUT
2. add test cases based on
   - use case, design, minimal successs guarantess
3. determine how verdict is reached through providing expected results

-> testing must be based on fault model

Question 8

Which aspects of testing can be automated?

Answer 8

• running the tests
• generating test inputs
• generating test verdict (test oracle)

Question 9

What can be achieved by testing?

Answer 9

establish sufficient trust into the system

Question 10

Definition - Test (Data) Point

Answer 10

is a specific value for

• a test case input or
• a state variable

Question 11

Definition - Test Case

Answer 11

consists of

• pretest state of the IUT
• test point/ condition
• expected results

Question 12

Definition - Test Suite

Answer 12

Collection of test cases

Question 13

Definition - Test Run

Answer 13

execution of a test suite on the IUT

Question 14

Structural Coverage - Statement Coverage (SC)

Answer 14

all statements were executed at least once

Question 15

Structural Coverage - Basic Block Coverage (BBC)

Answer 15

(Basic Block: max sequence of instructions without branch points)
all basic block are executed at least once

Question 16

Structural Coverage - Branch Coverage (BC)

Answer 16

all edges in the CFG were executed at least once

Question 17

Structural Coverage - Path Coverage (PC)

Answer 17

all path in the CFG were executed at least once

unachievable in practice

Question 18

Logic Coverage - Decision Coverage (DC)

Answer 18

any decisions in the program were at least once false and once true
(desicion: a < b && c == d)

Question 19

Logic Coverage - Condition Coverage (CC)

Answer 19

any conditions in the program were at least once false and once true
(condition: a < b, c == d (from a < b && c == d))

Question 20

Logic Coverage - Modifies Condition-Decision Coverage (MCDC)

Answer 20

for one occurrence condition c in decision d

• d evaluates at least twice
• once where c is false
• once where c is true
• d evaluates differently in both cases
• and other conditions in d evaluated identically or not evaluated at least once

(often required when developing safety-critical software)

Question 21

Automated Test Case Generation

Answer 21

White Box

• Syntactic approaches: scanning for contitions, evaluation -> suitable to achieve logic based coverage criteria
• Symbolic execution: unwinding CFG with symbolic values -> suitable to achieve also structural coverage criteria

Black Box
- analysis of input data or model of IUT

Question 22

Automatic Static Verification Techniques

Answer 22

Static Checking (based on CFG)

• Runtime exceptions, liveness, information flow
• fully automated
• many false positives
• scales reasonably well

Bug Finding