4.2 Black-box Test Techniques

Question 1

What is black-box testing?

Answer 1

Test techniques based on analysis of the specified behavior of the test object without reference to its internal structure. Test cases are independent of code and software implementation, and therefore test cases remain the same even if the structure changes, so long as the required behavior does not change.

Question 2

How does equivalence partitioning work?

Answer 2

Data is divided into groups based on the expectation that all the elements of a given partition are to be processed in the same way. Therefore if a test case that checks one value detects a defect, cases that test any other values should also produce a defect. Therefore, only one test case per partition is needed.

Question 3

What is the difference between valid and invalid partitions?

Answer 3

Valid partitions contain “valid” data, as defined by the team. Usually this means data that should be successfully processed by the test object.

Invalid partitions contain “invalid” data, as defined by the team. Usually this means data that should be ignored or rejected by the test object.

Question 4

What is Each Choice coverage?

Answer 4

Each Choice coverage requires each set of test cases to exercise each partition from each set of partitions at least once, not taking into account combinations of partitions.

It is used because many test objects include multiple sets of partitions (more than one input parameter).

Question 5

What is Boundary Value Analysis?

Answer 5

A technique based on testing the boundaries of equivalence partitions. The minimum and maximum values of a partition are its boundaries; all elements between them therefore belong to the same partition.

Developers are more likely to make errors with boundary values, often where the implemented boundaries are a step above or below their intended positions. In some cases, boundaries are inadvertently omitted altogether.

Question 6

What is 2-value BVA?

Answer 6

For each boundary value, there are two coverage items: this value, and its closest neighbor in the adjacent partition.

To achieve 100% coverage with 2-value BVA, test cases must exercise all coverage items, i.e., all identified boundary values. Coverage is measured as the number of boundary values that were exercised, divided by the total number of identified boundary values, and is expressed as a percentage.

Question 7

What is 3-value BVA?

Answer 7

For each boundary value, there are three coverage items: this boundary value, and both its neighbors. Therefore in 3-value BVA, some coverage items may NOT be boundary values.

To achieve 100% coverage, test cases must exercise all coverage items. Coverage is the number of boundary values and their neighbors, divided by the total number of identified boundary values, and is expressed as a percentage.

Question 8

Which version of boundary value analysis is more rigorous?

Answer 8

3-value BVA is more rigorous than 2-value BVA as it may detect defects overlooked by 2-value BVA. For example, if the decision “if (x ≤ 10) …” is incorrectly implemented as “if (x = 10) …”, no test data derived from the 2-value BVA (x = 10, x = 11) can detect the defect. However, x = 9, derived from the 3-value BVA, is likely to detect it.

Question 9

What is a Decision Table?

Answer 9

Decision Tables are used to test implementation of system requirements that specify how different combinations of conditions result in different outcomes. They are an effective way of recording complex logic, such as business rules.

The conditions and resulting actions of the system are defined as the rows of the table. Each column corresponds to a decision rule that defines a unique combination of conditions along with the associated outcomes.

Question 10

What is a limited entry Decision Table?

Answer 10

All the values of conditions and actions are boolean (true / false).

Question 11

What is an extended entry Decision Table?

Answer 11

Some or all conditions and actions may have values beyond boolean, such as ranges of numbers, equivalence partitions, or discrete values.

Question 12

Describe the notation for Decision Table conditions and actions.

Answer 12

For conditions: T means True, the condition is satisfied. F means false, the condition is not satisfied. “-“ (hyphen) means the value of the condition is irrelevant for the action outcome. “N/A” means the condition is infeasible for the given rule.

For actions: X means that the action should occur. Blank means it should not occur.

Question 13

A full decision table has enough columns for every combination of conditions. Testing the full table can be time consuming since the number of rules grows exponentially with the number of conditions. How can we simplify it?

Answer 13

• Delete columns containing infeasible combinations.
• Merge columns where conditions don’t affect the outcome into a single column, where it makes sense to do so.
• Use a minimized decision table or a risk-based approach to determine test coverage.

Question 14

What is Decision Table Testing?

Answer 14

Coverage items are the columns containing feasible combinations of conditions. To achieve 100% coverage with this technique, test cases must exercise all these columns. Coverage is measured as the number of exercised columns, divided by the total number of feasible columns, and is expressed as a percentage.

Question 15

What are the benefits of decision table testing?

Answer 15

It provides a systematic approach to identify all combinations of conditions, some of which might otherwise be overlooked.
It helps to find gaps or contradictions in the requirements.

Question 16

What is a State Transition diagram?

Answer 16

A State transition diagram models the behavior of a system by showing its possible states and valid state transitions.

A transition is initiated by an event, which may have a guard condition. Transitions occur instantly and sometimes result in the software taking action. The common syntax to describe a transition is “event [guard condition] / action”.

Question 17

What is a State Table?

Answer 17

A state table is a model equivalent to a state transition diagram. Its rows represent states, and its columns represent events, together with any guard conditions that exist. Table entries (cells) represent transitions and contain the target state as well as the resulting actions, if defined.

In contrast to a state transition diagram, a state table shows invalid transitions as empty cells.

Question 18

How might a test case based on a state transition diagram or table work?

Answer 18

A test case based on a state transition diagram or state table is usually represented as a sequence of events, which results in a sequence of state changes (and sometimes actions). One test case usually will cover several transitions between states.

Question 19

Describe All States Coverage.

Answer 19

In this coverage criteria, coverage items are the states themselves. To achieve 100% coverage, test cases must ensure that all states are visited. Coverage is measured as the number of visited states divided by the total number of states.

Question 20

Describe valid transitions coverage (also called 0-switch coverage).

Answer 20

Coverage items are single valid transitions. To achieve 100% coverage, test cases must exercise all valid transitions. Coverage is measured as the number of exercised transitions divided by the total number of valid transitions.

Question 21

Describe All Transitions Coverage.

Answer 21

In this criteria, coverage items are all transitions shown in a state table. To achieve 100% coverage, test cases must exercise all valid transitions and attempt to exercise all invalid transitions.

Testing only one invalid transition in a single test case helps to avoid fault masking (a situation where one defect prevents the detection of another.

Coverage is measured as the number of valid and invalid transitions tested, divided by the total number of transitions.

Question 22

Rank the different types of state transition coverage from strongest to weakest.

Answer 22

All Transitions Coverage, if achieved, guarantees both valid transitions coverage and all states coverage, and is the strongest, but also the most difficult.

Valid Transitions Coverage, if achieved, guarantees all states coverage, and is the second strongest. It is the most widely used criterion for transition coverage.

All State coverage is weaker than either of the above because it can typically be achieved without exercising all valid transitions.