Lecture 6 Flashcards
What is validation and what is verification?
- verification: Are we developing the product right? (Often performed as internal procedure)
- validation: Are we developing the right product? (Between stakeholders and costumer)
What is program testing?
Testing aims to reveal errors in the program
It’s aimed at breaking the system.
Testing can only reveal the presence of errors, not their absence.
Testing is part of a general verification and validation process.
How can we deal with faults?
- Fault avoidance (before the system is released):
- Fault detection (while system is running)
- Fault tolerance (recover from failure once the system is released)
What is fault, erroneous state and failure?
Fault: The mechanical or algorithmic cause of an erroneous state (“bug”).
Erroneous state: The system is in a state such that further processing by the system can lead to a failure.
Failure: Any deviation of the observed behaviour from the specified behaviour.
Shortly what is black-box and white boc testing?
Black-box testing focuses on testing software functionality without knowledge of internal code structure, while white-box testing involves testing internal code structures and logic.
Example of fault (inteface specification):
Mismatch between what the client needs and what the server offers
Mismatch between requirements and implementation
What is the focus and goal of black-box testing?
Focus: Input/output behaveior. if for any given input, we can predict the output, then the unit passes the test.
Goal: Reduce test cases using equivalence partitioning. Group similar inputs and test one from each group. Example: Test one negative number if all negatives are expected to behave the same.
Example of algorithmic faults:
Missing initialisation
Incorrect branching condition
Missing test for null
What is the focus and goal of white-box testing?
Internal code logic. Test cases are designed to check if the software works correctly for various code paths and conditions.
Goal: Ensure all paths and conditions in the code are tested. (CHAT)
Examples of mechanical faults (hard to find):
- Operating temperature outside of equipment specification
- Power failure
What is COVERAGE?
Degree to which sum of test cases
covers program.
E.g., 75% path coverage, if 4
Example of erroneous state:
- Wrong user input
- Null reference errors
- Concurrency errors
- Exceptions
Name the three types of coverage:
Types of coverage
* Statement Coverage (C0)
* Branch Coverage (C1)
* Path Coverage (C2)
How do we deal with them (fault, erroneous state, failure)?
- Patching
- Declaring the bug as a feature
- Testing
- Modular redundancy
Explain C1
C1 is BRANCH COVERAGE:
Full coverage
* At every decision, all branches are executed, e.g.,
for if statement, both then and else part
* Considers “empty” alternatives, e.g., omitted else
branches
Benefits
* Reveals unreachable branches
* Good functional test usually achieves high branch
coverage
Metric:
* C1 = Covered (Primitive) Branches / All (Primitive)
Branches
What can testing only show?
“Testing can only show the presence of bugs,
not their absence” (Dijkstra)
Explain C2
Full coverage
* All sequences whose elements relate to directly
connected nodes in the control flow graph
(potentially infinitely many)
* Considers (repeated) loops
Benefits
* theoretically optimal testing strategy but
combinatorial explosion
Metric:
* C2 = Covered Execution Paths / All Execution Paths
(theoretically)
Static analysis vs. dynamic analysis?
Static analysis (testing without code execution):
- Hand execution
- Walk-through
- Code inspection
Automated tools checking for
- syntactic and semantic errors
- departure from coding standards
Dynamic analysis: (testing with code execution):
- Black-box testing (Test the input/output behaviour)
- White-box testing (Test the internal logic of the subsystem or class)
Name the four testing styles:
Unit Testing, System Testing, Integration Testing, Acceptance Testing
How to control flow graphs?
Statement (C_0)
- One language construct
- “Line of code”
Branch (C_1)
- Caused by conditions (if, switch, while etc.)
- E.g., A at first choice but not B
Path (C_2)
- One potential execution
- E.g., sequence creating A, C
Explain unit testing
- Individual components (class or
subsystem) are tested - Carried out by developers
- Goal: confirm that the component or subsystem is correctly coded and carries out the intended functionality
What is statement coverage?
Full coverage, as every statement is executed at least once.
Benefits are that it’s easy to achieve and reveals unreachable code
Metric is C_0 = Covered Statements/All statements
Explain System Testing
- The entire system is tested
- Carried out by developers
- Goal: determine if the system meets the requirements
(functional and nonfunctional)
Describe the characteristics of C_0 (name, execution, effort):
Statement coverage test
Execute each statement at least once
Relatively low effort