Quality Assurance

Question 1

Testing is…

Answer 1

show program does what is intended
discover defects before it is put into use
finding differences between expected behaviour and observed behaviour
is aimed at breaking the system
is part of a more general verification and validation process

Testing can reveal the presence of errors - not their absence!

Question 2

Validation

Answer 2

Are we developing the right product?

“The assurance that a product, service, or system meets the needs of the customer and other identified stakeholder”

Often involves external stakeholders/customers.

Question 3

Verification

Answer 3

Are we developing the product right?

“The evaluation of whether or not a product, service or system complies with regulation, requirement, specification, or imposed condition.”

Often performed as internal procedure.

Question 4

Static analysis

Answer 4

Hand execution (reading source code)
Walk-through (informal presentation)
Code inspection (formal presentation)
Automated tools (i.e. in the IDE)

Question 5

Dynamic analysis

Answer 5

Black box
White box testing

Question 6

Black-box testing

Answer 6

We don’t focus on the program but only input and getting the expected output.

Focus on I/O behaviour. If for any given input we can predict the output, the the unit passes the test. (almost impossible to generate all possible inputs (“test cases”))

Question 7

Equivalence partitioning

Answer 7

Done in black-box testing bc there are infinite many possible inputs. A way to reduce the number of test cases by dividing inputs into equivalence classes.

Example: if an object is supposed to accept a negative number, testing one negative number is enough.
Or could be: negative number, 0, positive small number, positive large number

Question 8

White-box testing

Answer 8

White box testing is a form of application testing that provides the tester with complete knowledge of the application being tested, including access to source code and design documents.

Question 9

Statement

Answer 9

One language construct
‘Line of code” checking true or false

Question 10

Branch

Answer 10

Caused by conditions (if, switch, while, etc.)
E.g. A at first choice but not B.

Question 11

Path

Answer 11

One potential execution.
E.g. sequence creating A,C

Question 12

Coverage

Answer 12

Test quality criteria
Degree to which sum of test cases cover program.
e.g. 75% path coverage

Statement coverage C_0
Branch coverage C_1
Path coverage C_2

Question 13

Statement coverage

Answer 13

Basically check all conditions.
Full coverage: every statement executed at least once.

Easy to achieve
reveals unreachable code

C_0 = covered statements / all statements

Question 14

Branch coverage

Answer 14

Full coverage == all branches are executed.
For if statement, both then and else part.

Reveals unreachable branches.
Good functional test usually achieves high branch coverage.

C_1 = covered (primitive) branches / all (primitive) branches

Question 15

Path coverage

Answer 15

Full coverage == all sequences whose elements relate to directly connected nodes in the control flow graph (potentially infinitely many)
Considers repeated loops.

Theoretically optimal testing strategy but combinatorial explosion.

C_2 = covered execution paths / all execution paths (theoretically)

Question 16

The three different path coverage tests

Answer 16

Complete test: all possible paths (generally not possible with loops)
Boundary-Interior Test: Similar to above but loops are executed via special rules (high effort)
Structure test: Similar to above, but loops are executed exactly n-times (high effort)

Question 17

Unit testing

Answer 17

Individual components (class or subsystem) are tested.

Question 18

System testing

Answer 18

The entire system is tested
Goal: determine if the system meets the requirements (functional and non-functional)

Question 19

Integration testing

Answer 19

Groups of subsystems (collections of subsystems) and eventually the entire system are tested.

Goal: test the interfaces among the subsystems. Verify units interacting properly.

Question 20

Acceptance testing

Answer 20

Evaluates the system delivered by developers.

Carried out by the client.
Goal: demonstrate that the system meets the requirements and is ready to use.

Question 21

Test driver

Answer 21

Simulates part of the system that calls the component under test.
So the driver calls whatever component we are testing.
A component that calls the System Under Test (SUT) or TestedUnit
Often used in Bottom-up integration
Drivers are used to invoking the component that needs to be tested.

Question 22

Test double

Answer 22

Used if the collaborator in the system is awkward to work with.
4 types: dummy object, fake object, stub, mock object

Question 22

Test stub

Answer 22

simulates a component that is being called by the tested component.

provides/implements the same API as the component
a component, the SUT depends on
returns fake values

Question 23

Dummy object

Answer 23

Test double
passed around but never actually used. Usually used to fill parameter lists

Question 24

Fake object

Answer 24

Test double
is a working implementation, but usually contains some type of ‘shortcut’.
example: a database stored in memory instead of a real database

Question 25

Stub (test double)

Answer 25

Provides canned answers to calls made during the test, but is not able to respond to anything outside what it is programmed for

Question 26

Mock object

Answer 26

test double
Mocks are able to mimic the behaviour of a real object. They know how to deal with sequences of call they are expected to receive.

Question 27

Big bang approach

Answer 27

Integration test strategy
Test all object at the same time.

Question 28

Bottom up test

Answer 28

Integration test strategy
The subsystem in the lowest layer of the call are tested individually.
Then the subsystems above this layer are tested that call the previously tested subsystems. Repeated until all subsystems are included.

Question 29

Top down test

Answer 29

Test the subsystems in the top layer first.
Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems. Do this until all subsystems are incorporated into the tests.

Question 30

Functional testing

Answer 30

System test. We want to test the functionality of the system.
Test cases are designed from the requirement analysis document.
System is treated as black box.

Question 31

Performance testing

Answer 31

System test
Try to violate non-functional requirements.

I.e. could be test how the system behaves when overloaded. (stress testing)
Try unusual orders of execution.
Check the system’s response to large volumes of data. (volume testing)

Security testing
Recovery testing etc.

Question 32

Alpha testing

Answer 32

Acceptance test
Users of software work with the development team to test the software at the developer’s side.

Question 33

Beta testing

Answer 33

Acceptance test
A release of the software is made available to users to allow them to experiment and raise problems that they discover with the system developers.

Question 34

Acceptance testing

Answer 34

Acceptance test
Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems.
Not present in agile processes.

Question 35

When should you write your tests?

Answer 35

Traditionally, after the source code is written.
In XP or when using TDD (test-driven development), before the source code is written.

Question 36

Regression testing

Answer 36

All tests rerun every time a change is made to the program. Tests must run successfully before the change is committed.

Question 37

How to write one simple test in jUnit.
Testing a method:
public int mul(int x, int y){
// ???
}

Answer 37

1. @Test
2. Define class, use descriptive name
   public void testMulPositiveNumbersReturnPositive
   In function:
3. Arrange: Initialise the object you are testing on (unless @BeforeEach)

Calculator c = new Calculator();

2. act: perform the method

int result = c.mul(2,2);

3. assert: check if result is expected result

assertEquals(4, result);

Question 38

How to set up several test cases using @BeforeEach.

Testing a method:
public int mul(int x, int y){
// ???
}

Answer 38

public class CalculatorTest {
private Calculator c;
@BeforeEach
public void setup(){
c = new Calculator();}
@Test
public void testMul1(){
int result = c.mul(2,2);
assertEquals(4, result);}
}

Question 39

@Test

Answer 39

Makes method a test method

Question 40

@BeforeEach

Answer 40

Prepare test environment before each test (e.g. load data)

Question 41

@AfterEach

Answer 41

Clean up test environment after test.

Question 42

assertEquals([message,], expected, actual)

Answer 42

Checks if values are equivalent. Message not necessary.

Question 43

assertTrue([message,],condition)

Answer 43

Asserts if the condition is true, if not throws assertionError with the message.
Same for assertFalse, assertFalse, assertNull, assertNotNull