Black Box Testing Flashcards
Basic principles of black box testing
- Test-case design based on definition of spec (reqs) of software, instead of structure (source code)
- Complete coverage can be applied to black box testing (BBT)
- Rigorous specs have various benefits & help BBT (e.g. categorise inputs & derive expected outputs)
- Specs help test-case generation & test oracles
- Most faults caused by interactions of max 2 factors
- Want to reduce combos to manageable size while still providing effective fault detection power to test suite
What is the purpose of equivalence class partitioning?
Have sense of complete testing and avoid test redundancy
What are equivalence classes?
Partitions of input set in which input data has same effect on program (same output)
What is the purpose of disjoint classes?
Avoids redundancy
What is the process of WECT?
Weak equivalence class partioning
- Choosing 1 var value form each equiv class such that all classes covered - not all cartesian combos.
- Num test cases = max(|A|,|B|,|C|)
- Num WECT test cases = max partition size (D)
What is the process of SECT?
Strong equivalence class partioning
- Based on cartesian product of partition subsets (test all interactions of all equiv classes) - 1 item per partition - all cartesian combos
- Num test cases = |A| x |B| x |C|
- Num SECT test cases = partition size (D) x partition size (M) x partition size (Y)
What is a cartesian combination?
Cross join
What is the process of equivalence class partioning?
- ID input domain (of method/function params)
- Start from reqs
- Draw equiv classes diagram for each param
- Combine equiv classes, using WECT or SECT & spec expected output per test case
What is pair-wise and n-way testing?
- 2 way testing
- test cases = max x 2nd max
- n-way testing is generalisation of pair-wise testing
What is BVA?
Boundary-value analysis
- Behaviour of program not always similar for all input values of equiv class, esp at boundaries between classes e.g. off-by-one error
- Set values for input var at min, just above min, nominal (typical) value, just below max & at max
- Usual strategy for all input vars is holding values of all but 1 var at nominal values, let 1 var assume extreme value
- Function with n vars requires 4n + 1 test cases
What are the benefits of BVA?
- Works well with vars representing bounded physical quantities
- Extendable to robustness testing
What is WCT?
Worst-case testing
- Boundary-value assumes failures usually originate from 1 fault
- Cartesian product of {min, min+, nom, max-, max}
- 5n test cases (n number vars)
- Good strategy when physical vars have numerous interactions & failure costly
- Robust worst cases testing has 7n test cases
What is robustness testing?
- Robustness is degree to which system operates correctly in presence of exceptional inputs/stressful environmental conditions
- Goal of robustness testing to develop test cases & environments where robustness assessed
- Also look at behaviour of system when var extremes exceeded with value slightly above max & slightly less than min
- Function with n vars requires 6n + 1 test cases
What are the benenfits of MBT?
- Improved test-case design practices
- Detecting real fault during regression testing
- Measuring code coverage (client & server side)
- Made work of test engineers more interesting & organised
- Easier to see big picture with model in front of them & being directly executable
Fault Taxonomy
- Missing/incorrect transition by implementation of valid state based on correct input (event)
- Missing/incorrect output (action) by implementation based on correct input & transition
- Sneak path (extra transition) - implementation accepts illegal/unspecific by state input
- Illegal input failure - implementation fails to handle illegal message correctly (incorrect output, state corrupted)
- Corrupt state - based on correct input, implementation output invalid state
