Black Box Testing Flashcards
Basic principles of black box testing
- Test-case design based on definition of spec (reqs) of software, instead of structure (source code)
- Complete coverage can be applied to black box testing (BBT)
- Rigorous specs have various benefits & help BBT (e.g. categorise inputs & derive expected outputs)
- Specs help test-case generation & test oracles
- Most faults caused by interactions of max 2 factors
- Want to reduce combos to manageable size while still providing effective fault detection power to test suite
What is the purpose of equivalence class partitioning?
Have sense of complete testing and avoid test redundancy
What are equivalence classes?
Partitions of input set in which input data has same effect on program (same output)
What is the purpose of disjoint classes?
Avoids redundancy
What is the process of WECT?
Weak equivalence class partioning
- Choosing 1 var value form each equiv class such that all classes covered - not all cartesian combos.
- Num test cases = max(|A|,|B|,|C|)
- Num WECT test cases = max partition size (D)
What is the process of SECT?
Strong equivalence class partioning
- Based on cartesian product of partition subsets (test all interactions of all equiv classes) - 1 item per partition - all cartesian combos
- Num test cases = |A| x |B| x |C|
- Num SECT test cases = partition size (D) x partition size (M) x partition size (Y)
What is a cartesian combination?
Cross join
What is the process of equivalence class partioning?
- ID input domain (of method/function params)
- Start from reqs
- Draw equiv classes diagram for each param
- Combine equiv classes, using WECT or SECT & spec expected output per test case
What is pair-wise and n-way testing?
- 2 way testing
- test cases = max x 2nd max
- n-way testing is generalisation of pair-wise testing
What is BVA?
Boundary-value analysis
- Behaviour of program not always similar for all input values of equiv class, esp at boundaries between classes e.g. off-by-one error
- Set values for input var at min, just above min, nominal (typical) value, just below max & at max
- Usual strategy for all input vars is holding values of all but 1 var at nominal values, let 1 var assume extreme value
- Function with n vars requires 4n + 1 test cases
What are the benefits of BVA?
- Works well with vars representing bounded physical quantities
- Extendable to robustness testing
What is WCT?
Worst-case testing
- Boundary-value assumes failures usually originate from 1 fault
- Cartesian product of {min, min+, nom, max-, max}
- 5n test cases (n number vars)
- Good strategy when physical vars have numerous interactions & failure costly
- Robust worst cases testing has 7n test cases
What is robustness testing?
- Robustness is degree to which system operates correctly in presence of exceptional inputs/stressful environmental conditions
- Goal of robustness testing to develop test cases & environments where robustness assessed
- Also look at behaviour of system when var extremes exceeded with value slightly above max & slightly less than min
- Function with n vars requires 6n + 1 test cases
What are the benenfits of MBT?
- Improved test-case design practices
- Detecting real fault during regression testing
- Measuring code coverage (client & server side)
- Made work of test engineers more interesting & organised
- Easier to see big picture with model in front of them & being directly executable
Fault Taxonomy
- Missing/incorrect transition by implementation of valid state based on correct input (event)
- Missing/incorrect output (action) by implementation based on correct input & transition
- Sneak path (extra transition) - implementation accepts illegal/unspecific by state input
- Illegal input failure - implementation fails to handle illegal message correctly (incorrect output, state corrupted)
- Corrupt state - based on correct input, implementation output invalid state
Transition coverage for MBT
Needs/requires every transition in state model to be taken min 1x
Full predicate coverage for MBT
Requires each clause in each predicate (composite condition) of transitions to be tested true/false min 1x
Transition-pair coverage for MBT
Test for invalid sequences of transitions. For each pair of adjacent transitions (states), test suite should contain test case traversing pair of transitions in sequence min 1x
Complete sequence coverage for MBT
All poss sequences of transitions on state chart diagram tested, usually v expensive
What are the benefits of decision tables?
- Helps express test reqs in directly usable form
- Easy to understand & review by peers
- Support systematic derivation of test cases
- Support automated/manual gen of test cases
- Particular response (output) selected for testing by evaluating many related conditions
- Ideal for describing situations in which num of combos of actions taken under varying condition sets
What is the condition section in decision tables?
Section listing conditions & combos of them
- Express relationships among decision vars
What is the action/output section in decision tables?
Section lists responses to be produced when corresponding combos of conditions true
- Actions independent of input order & order in which conditions evaluated
Decision tables
- Kind of a type of MBT
- 1 test case from each column of corresponding truth table
Cause-effect graphs
Type of MBT
- Graphical technique helping derive decision tables
- Aimed at supporting interaction with domain experts & reverse engineering of specs for testing purposes
- ID causes (conditions on inputs, stimuli) & effects (outputs, changes in system state)
- Causes must be stated in boolean manner
- Can be used to explicitly specify constraints (environmental/external) on causes & effects
- Can help select more signif subset of input-output combos & build smaller decision tables
What is the structure of cause-effect graphs?
Node drawn for each cause & effect
- Placed on opposite side of sheet
- Edge from cause to effect indicates cause necessary condition for effect
- If 1 effect has 2+ causes, logical relationship of causes annotated by symbols for and (^) & or (v) between lines
- Cause whose negation necessary shown by not (~)
- 1 cause may be necessary for many effects, 1 effect may have many necessary causes
- Intermediate nodes may be used to simplify graph & its construction
Model-based testing
- MBT is application of model-based design for systematic & automated test-case design & optionally also for automated execution of test cases
- First introduced by IBM in 1970
- Can use GraphWalker
- Nodes/vertices are states (pages) of system to verify
- Edges/transitions are events like valid login & click
- Design test models (GUI flow-diagrams of given software under test (SUT))
- More semantic features on models e.g. guard conditions & actions on events/edges are behind scenes
- For large SUTs, test models must be modularised using shared nodes like function calls
- Test models can have diff forms e.g. state charts, BPMN (Business process model & notation)
- State charts are variation in which outputs can be incorporated based on state & input
- Action: If edge taken, action command executed. Written after / on edge
-Guard Condition: Edge transition occurs only if guard condition true. Written in square brackets on edge
What are the steps of category-partition testing?
- System divided into indiv functions that can be independently tested
- IDs params of each function & for each param, distinct categories
- Environmental characteristics under which function operates (of system state) may be considered
- Categories are major properties/characteristics for each
param- Further subdivided into choices in same way as equiv partitioning applied (poss values)
- Broader than Equivalence-Class Partitioning
- Integrates equiv class testing, boundary analysis, & few refinements
- Constraints between choices identified (how occurrence of 1 choice affects existence of other)
- Test frames/specs generated, defined as allowable combos of choices in categories & converted into test data
- All Combinations (AC): Typical. 1 value for every choice of every param used with 1 value of every poss choice of every other category
- Each choice (EC): Weaker. 1 value from each
choice for each category used in min 1 test case
