Lecture 6 Flashcards
What is validation and what is verification?
- verification: Are we developing the product right? (Often performed as internal procedure)
- validation: Are we developing the right product? (Between stakeholders and costumer)
What is program testing?
Testing aims to reveal errors in the program
It’s aimed at breaking the system.
Testing can only reveal the presence of errors, not their absence.
Testing is part of a general verification and validation process.
How can we deal with faults?
- Fault avoidance (before the system is released):
- Fault detection (while system is running)
- Fault tolerance (recover from failure once the system is released)
What is fault, erroneous state and failure?
Fault: The mechanical or algorithmic cause of an erroneous state (“bug”).
Erroneous state: The system is in a state such that further processing by the system can lead to a failure.
Failure: Any deviation of the observed behaviour from the specified behaviour.
Shortly what is black-box and white boc testing?
Black-box testing focuses on testing software functionality without knowledge of internal code structure, while white-box testing involves testing internal code structures and logic.
Example of fault (inteface specification):
Mismatch between what the client needs and what the server offers
Mismatch between requirements and implementation
What is the focus and goal of black-box testing?
Focus: Input/output behaveior. if for any given input, we can predict the output, then the unit passes the test.
Goal: Reduce test cases using equivalence partitioning. Group similar inputs and test one from each group. Example: Test one negative number if all negatives are expected to behave the same.
Example of algorithmic faults:
Missing initialisation
Incorrect branching condition
Missing test for null
What is the focus and goal of white-box testing?
Internal code logic. Test cases are designed to check if the software works correctly for various code paths and conditions.
Goal: Ensure all paths and conditions in the code are tested. (CHAT)
Examples of mechanical faults (hard to find):
- Operating temperature outside of equipment specification
- Power failure
What is COVERAGE?
Degree to which sum of test cases
covers program.
E.g., 75% path coverage, if 4
Example of erroneous state:
- Wrong user input
- Null reference errors
- Concurrency errors
- Exceptions
Name the three types of coverage:
Types of coverage
* Statement Coverage (C0)
* Branch Coverage (C1)
* Path Coverage (C2)
How do we deal with them (fault, erroneous state, failure)?
- Patching
- Declaring the bug as a feature
- Testing
- Modular redundancy
Explain C1
C1 is BRANCH COVERAGE:
Full coverage
* At every decision, all branches are executed, e.g.,
for if statement, both then and else part
* Considers “empty” alternatives, e.g., omitted else
branches
Benefits
* Reveals unreachable branches
* Good functional test usually achieves high branch
coverage
Metric:
* C1 = Covered (Primitive) Branches / All (Primitive)
Branches
What can testing only show?
“Testing can only show the presence of bugs,
not their absence” (Dijkstra)
Explain C2
Full coverage
* All sequences whose elements relate to directly
connected nodes in the control flow graph
(potentially infinitely many)
* Considers (repeated) loops
Benefits
* theoretically optimal testing strategy but
combinatorial explosion
Metric:
* C2 = Covered Execution Paths / All Execution Paths
(theoretically)
Static analysis vs. dynamic analysis?
Static analysis (testing without code execution):
- Hand execution
- Walk-through
- Code inspection
Automated tools checking for
- syntactic and semantic errors
- departure from coding standards
Dynamic analysis: (testing with code execution):
- Black-box testing (Test the input/output behaviour)
- White-box testing (Test the internal logic of the subsystem or class)
Name the four testing styles:
Unit Testing, System Testing, Integration Testing, Acceptance Testing
How to control flow graphs?
Statement (C_0)
- One language construct
- “Line of code”
Branch (C_1)
- Caused by conditions (if, switch, while etc.)
- E.g., A at first choice but not B
Path (C_2)
- One potential execution
- E.g., sequence creating A, C
Explain unit testing
- Individual components (class or
subsystem) are tested - Carried out by developers
- Goal: confirm that the component or subsystem is correctly coded and carries out the intended functionality
What is statement coverage?
Full coverage, as every statement is executed at least once.
Benefits are that it’s easy to achieve and reveals unreachable code
Metric is C_0 = Covered Statements/All statements
Explain System Testing
- The entire system is tested
- Carried out by developers
- Goal: determine if the system meets the requirements
(functional and nonfunctional)
Describe the characteristics of C_0 (name, execution, effort):
Statement coverage test
Execute each statement at least once
Relatively low effort
Explain Integration Testing
- Groups of subsystems (collection
of subsystems) and eventually the
entire system are tested - Carried out by developers
- Goal: test the interfaces among
the subsystems.
Describe the characteristics of C_2 (name, execution, effort):
Path coverage test:
Complete test (C_2a)
All possible paths are executed
nerally not possible with loops
Boundary-interior test (C_2b)
Similar to C_2a but loops are executed via special rules
High effort
Structured test (C_2c)
Similar to C_2a but loops are executed exactly n-times
High effort
Explain Acceptance Testing
- Evaluates the system delivered by
developers - Carried out by the client. May
involve executing typical
transactions on site on a trial basis - Goal: demonstrate that the
system meets the requirements
and is ready to use.
Describe the characteristics of C_1 (name, execution, effort):
Branch coverage test
Each edge is visited at least once
Realistic min. requirement, moderate effort
Mention the three INTEGRATION TEST
STRATEGIES
- Big bang
- Bottom-up
- Top-down
Explain shortly the levels of tests using car as an example
Unit Testing: Checks small parts like a car’s engine. Done by creators to ensure these parts work as intended.
Integration Testing: Tests how these parts fit together (like wheels to the engine). Done by creators to verify the connections.
System Testing: Tests the whole car. Done by creators to see if the whole thing meets the plan.
Acceptance Testing: Done by the person buying the car to make sure it fits their needs and works properly in real use.
1.
Mention the four different doubles
- Dummy Object: Only used to fill parameter spaces; not used in actual testing logic.
- Fake Object: A simpler, working implementation for testing, like an in-memory database.
- Stub: Returns fixed responses to specific calls made during testing.
- Mock Object: Simulates behavior of real objects, checking interactions and sequences in tests.
What are test drivers?
Simulates the part of the system that calls the component under test.
Helps in setting up the conditions necessary for testing a specific component by mimicking the behavior of other components it depends on.
Calls the Systems Under Test (SUT) or TestedUnit
What is a test double?
A test double in software testing is similar to a stunt double in movies. It’s used to replace a real component in the system when that component is difficult or impractical to use during testing
What are test stubs?
Simulates a component that’s being called by the tested component
A component that System Under Test (SUT) or TestedUnit depends on
Partial implementation and returns fake values
Explain the big bang approach:
Explain layered integration shortly (NOT DONE)
Her
Tests the interactions between different layers of an application before testing the entire system as a whole.
Layer I: Likely the user interface
Layer II: May include business logic with components like “Entity Model,” “Calculator,” and “Currency Converter”.
Layer III: The bottom layer, typically the data layer, with components “BinaryFile Storage,” “XMLFile Storage,” and “Currency DataBase”.
The approach suggests that integration testing should be done starting from the bottom layer up. First, ensure that each component within a layer (Layer III) works as expected. Next, test the interactions between components in Layer III and Layer II. Once Layer II components are working with Layer III, proceed to test the integration between Layer II and the top Layer I.
This method helps isolate issues within and between layers, making it easier to identify and fix problems than testing everything at once (as in the Big Bang approach).
Explain bottom up in integration testing. Draw it?
What is bottom up?
An integration strategy in the lowest layer of the call hierarchy (tested individually)
The subsystems above this layer are tested and call the previously tested subsystems
This is repeated until all subsystems are included.
Test from the bottom and up.
Explain top down and draw it:
What integration strategy is this?
Her
Bottom up testing
Explain agile process and acceptance test in comparesion together:
The point of the acceptance testing process in Agile methods is to ensure that the software meets the user or customer’s expectations and requirements.
What is top down?
An integration strategy in the top layer
Combine all the subsystems that are called by the tested subsystems and tets the resulting collection of subsystems
Do this until everything is incorporated into the tests
test from the top down
What is regression testing?
Regression testing is testing the system to check that changes have not ‘broken’ previously working code
Pros and cons of bottom up
Pros:
- No stubs needed
- Useful for integration testing of these systems: Object oriented systems, real-time systems, systems with strict performance requirements
Cons:
- Drivers are needed
- Tests an important subsystem last (the user interface
Explain Continuous Integration (integration strategies):
a software development technique where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day.
What are the two types of system testing?
Functional testing:
- Goal: Tests the functionality of system
- Test cases are designed from the requirement analysis and centers the requirements and key functions (use cases)
- The system is treated as black box
Performance testing:
- Goal: Try to violate non-functional requirements
- Test how the system behaves when overloaded
- Tries unusual orders of execution
- Check the system’s response to large volumes of data
Mention some types of performance testing
- Stress Testing
- Volume testing
- Configuration testing
- Compatibility test
- Timing testing
- Security testing
- Environmental test
- Quality testing
- Recovery testing
- Human factors testing
Pros and cons of top down
Pros:
- Tests can be defiend in terms of the functionality of the system (functional requirements)
- No drivers needed
Cons:
- Stubs are needed
- Stubs are difficult to write. Stubs must be allowed all possible conditions to be tested
- Large number of stubs may be required if the lowest layer contains many methods
- Some interfaces are not tested separately
Say the test-driven development cycle
Test-driven development cycle
* Add a new test to the test model
* Run the automated tests
* => the new test will fail
* Write code to deal with the failure
* Run the automated tests
* => see them succeed
* Refactor code
What are some types of user testing (type of acceptance testing)?
Alpha testing: Users of the software work with the developers to test the software
Beta testing: Release of the software is made available to the users (raise problems they find)
Acceptance testing: Customers test and decide whether it’s ready or not
What is the proper way to set up a unit test in Java?
Use @BeforeEach to create fresh instances of the class for each test, ensuring tests are independent and repeatable.
When should you write your tests?
Normally after the source code is written.
But in XP (or when using TDD), before the source code is written
What is the typical structure of a unit test in Java?
A unit test in Java typically consists of the @Test annotation followed by a test method that invokes the method to be tested and asserts expected outcomes.
How do you write a test method for a Java class?
Instantiate the class, call the method to be tested with specific inputs, and use assertEquals to verify the output matches the expected result.
What are some JUnit annotations
What is a JUnit assert statement?
An assert statement formulates an assumption about a test case’s expected behaviour.
It has to hold for the test case to pass.
What are some JUnit assert statements?
How do you test for expected exceptions in Java unit tests?
Use Assertions.assertThrows() to verify that the correct exception is thrown for invalid inputs.
How do you test the expected behavior of a method?
Write test cases that reflect the method’s specification, such as input-output pairs, and use assertions to ensure the method behaves as intended.
What are some integration strategies risks?
- The higher the complexity of the software system, the more difficult is the integration of its components
- The later integration occurs in a project, the bigger is the risk that unexpected faults occur
- Bottom up or top down integration strategies don’t do well with later integration
- Continuous integration addresses these risks by building as early and frequently as possible
Additional advantages:
- There is always an executable version of the system
- Team members have a good overview of the project status
