Lecture 10 Flashcards
Test Planning (TP)
The process of effectively organizing the testing activities for all the components associated with a solution.
[TP] Test Plan
Documents the strategy that will be used to verify and ensure that a product or system meets its design specs and other requirements. (Acceptance test plan, Unit test plan, Integration test plan)
[TP] Master Test Plan
A single high-level test plan for a project/product that unifies all other test plans.
[TP Test Plan] Document
- Plan ID
- Test Items
- Risk Issues
- Features to be tested/not tested
- Test Approach
- Pass/Fail Criteria
- Suspension Criteria
- Test Deliverables
- Environmental Requirements
- Staffing/Training Needs
- Schedule of Test
- Planning for Risks
- Approvals
[TP Test Plan] Approach/Strategy (Strategy)
- Special tools to be used
- Metrics to collect
- Configuration Combos
- Levels of Regression Testing
[TP Test Plan Strategies] Risk Based Testing (RBT)
Testing the functionality which has the highest impact (freq. of use) and rate of failure, due to lack of time, $, etc.
[TP Strategy RBT] Details
- Starts early in project to ID risks to system quality
- Uses info acquired to guide test planning/prep/execution
- Idea: Organize testing to reduce product risk of a deployed system
- Testing to achieve product outcome that balances risks with quality, features, budget, and schedule (QUALITY TRIANGLE)
[TP Strategy RBT] Test Strategy
1: Business analysis to understand reqs/process
2: Risk analysis (based on biz knowledge + solution): Higher risk => more testing
[TP] Test Completion (Pass/Fail) Criteria
- @ Unit Test Level: more focused on test cases themselves (of all % of test cases completed, allowable % of defects, code coverage tests)
- @ Master Test Plan Level: focused on child test plans (ex: all lower test plans complete, % of test plans w/out defects)
- In General: What’s the % and severity of defects?
[TP] Purposes
- Communicate with project team
(Org. Test policies/motivations)
(Test scope, objective, critical areas)
(Proj/Prod risks, resource considerations and constraints) - Manage Change
(Plans revise throughout projects lifetime - risk based change management helps manage change)
Test Execution (TE) verification or validation?
Executing code and comparing actual with expected results (AKA VALIDATION)
[TE] Guidelines
- TE occurs in a test/qa environment
- occurs in 2+ cycles (all tests executed in each cycle to ID/report/resolve critical defects, and most high defects)
- Test Readiness Review determines if test item can move into Test Execution phase (more common in waterfall, but appears in agile too sometimes)
[TE] Process
1) Select a subset of tests for each cycle based on an overall risk management strategy
2) Continuously execute tests, report defects, capture test status
3) Resolve blocking issues as they arise
4) Report status, adjust assignments, and reconsider plans/priorities daily
5) Report test cycle finding & status
Defect Tracking (DT)
Tracking the logged defects in a product from beginning to closure, and making new versions of the product that fix the defects.
[DT] Process Flow
DIAGRAM
Reporting Defects
- Reproduced at least once before reporting
- Objective
- Detailed
- Specific
Defect Severity <4>
- CRITICAL: no workaround
- MAJOR: non-obvious workaround
- MINOR: easy workaround
- TRIVIAL: workaround not necessary
Defect Priority (DP)
- URGENT (P1): must fix in next build
- HIGH (P2): must fix in upcoming builds, should be in next release
- MEDIUM (P3): may be fixed in or after next release
- LOW (P4): may/may not be fixed at all
[DP] Determination
- biz. need
- severity/impact
- probability/visibility
- available resources/time
Defect Density [DD]
- The number of confirmed defects in a software/component during a defined period (month/qtr/yr, phase of SDLC, entire SDLC) of dev/ops, divided by the size of the software/component.
- DD is a common metric in master test plans
[DD] Formula
DD = # of defects / size
- defects must be agreed upon and confirmed, not duplicates, and current (not dropped)
- size can be determined by functional points, # of lines in source code, etc
Test Summary Reporting
Summarizes the results of the levels of the testing activities and provides evaluations based on these results.
- May be used by any organization using the master test plan
- Includes any info uncovered by the tests accomplished, any assessments of quality of testing effort, quality of software system under test, statistics derived from defect reports
- Indicates whether software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders
(ex: reqs/usr stories implemented/fulfilled?)
