C2 Flashcards
black box testing
- test design against a specification, no implementation needed
- test values are derived from the specification or type information
- for each test value the expected result is derived from the specification (do not use the source code!)
input domain of a program
contains all possible inputs to that program
testing is about choosing finite sets of values from the input domain
input parameters
define the scope of the input domain (method parameters, data read from file, global variables, user input)
Domain for each input parameter is partitioned into regions of essentially equivalent values, at least one value is chosen from each region
domain partitions
a partition of D defines a set of blocks such that:
1. blocks must be pairwise disjoint (no overlap)
2. together the blocks cover the complete domain
for testing, choose a value from each block
5 steps to modeling the input domain
- identify testable functions
- find all the parameters
- model the input domain
- apply a test criterion to choose combinations of values
- refine combinations of blocks into test inputs (choose appropriate values from each block)
characteristics (examples)
- array (null, zero-length, arbitrary length)
- order of the input file (sorted, reverse sorted, abitrary)
- input device (DVD, CD, computer)
2 approaches to modeling the input domain
- interface-based: develop characteristics directly from individual input parameters (simplest application, can be partially automated)
- functionality-based approach: develops characteristics from a behavioral view of the program under test (harder to develop, may result in better tests)
interface-based approach
- mechanically consider each parameter in isolation
- relies mostly on syntax
- no use of domain and semantic information
- ignores relationships among parameters
functionality-based approach
- identify characteristics that correspond to the intended functionality (can incorporate domain and semantic knowledge and use relationships among parameters)
- modeling can be based on requirements but not on implementation
choosing combinations of values
- all combinations: all combinations from all characteristics (3 by 3 => 27 tests)
- each choice: one value from each block for each characteristic must be used in at least one test case (3 by 3 =?> 3 tests)
- pair-wise: a pair of values from two blocks from two characteristics must be combined with a value from every block for each other characteristic (3 by 3 => 9 tests)
- n-wise: a value from each block for each group of n characteristics must be combined (same as AC if n = nr of characteristics)
- (multiple) base choice: A (or multiple) base choice block is chosen for each characteristic, and a base test is formed by using the base choice for each characteristic. Subsequent tests are chosen by holding all but one base choice constant and using each non-base choice in each other characteristic (used if certain values are important, domain knowledge used)
constraints among characteristics
some combinations of blocks are infeasible => constraints among blocks
2 types:
- a block from one characteristic cannot be combined with a specific block from another
- a block from one characteristic can ONLY BE combined with a specific block form another characteristic
input space partitioning analysis and design
analysis:
- find all implicit and explicit parameters
- find the value range for each parameter
design:
- select suitable equivalence partitions
- choose coverage criteria
- select test values for coverage criteria and calculate expected result: choose values (possibly randomly) in the middle of partition, not at boundary
- if needed add test cases to cover all partitions of the return value/output parameters
input space partitioning strengths/weaknesses
strengths:
- fairly easy to apply, good basic level of black-box testing
- applicable to all levels of testing – unit, class, integration, system, etc.
- based only on the input space of the program, not on its implementation
weaknesses:
- correctness at the boundary is not tested
- interface based approach does not test combinations of inputs
boundary value analysis: analysis and design
analysis:
- identify boundary values for each partition
design:
- identify normal and error boundary values
- choose coverage criteria for normal boundary value
- select normal test values for coverage criteria and error test values for EC coverage
- eliminate redundant test cases
- calculate expected result
boundary value analysis stengths/weaknesses
strengths:
- test values are immediately given by the boundary values
- focus on area where faults are more likely
weaknesses:
- doubles the number of test cases for each partition
- combinations of values are not tested
