Test Planning, Strategies and the Delights of Monitoring Flashcards
Define Planning
- Scheduling of Activities (steps and in what order)
- Allocating resources
- Devising unambiguous milestones.
Define Monitoring
Judging progress against a defined plan (how are we doing?)
What must a good plan have?
visibility! The ability to monitor each step and make judgements
What is a Quality Process
Set of activities and responsibilities.
Focused on ensuring adequate dependability.
Concerned with project schedule or product usability.
What is the key principle of Quality planning
Cost of detecting and repairing fault increases as time passes between committing the errors and detecting resultant faults.
Quality plan includes matched set of validation and verification to detect faults quickly after the error is committed.
Quality planning leads to a quality end product
If our intermediate artifacts (early design, prototypes, test plans) are of poor quality, our end product will likely be of poorer quality.
How can we verify intermediate steps?
- Internal consistency checks (compliance with rules for that artifact type).
- External consistency checks (consistency with related artifacts, conformance to prior specification).
- Generation of correctness conjectures ( Lay groundwork of external consistency checks of other work products. Motivate refinement of current product)
Difference between a strategy and a plan
Strategy describes organization and structure over several projects. A plan is a standard structure prescribed in the strategy.
Strategy evolves slowly with organization and policy changes. Plan evolves quickly to project needs.
What do you consider when making a test strategy?
- You take past experiences in consideration.
- Use body of explicit knowledge instead of individual experiences. This is amenable to improvement, less vulnerable to organization change (losing people)
- Helps you avoid common recurring errors, increases development efficiency, maintain consistent process.
How to determine which strategy is good for a Organization?
- Consider structure and size
- Consider the process of the strategy
- Consider the domain where the strategy is good.
What are the elements of a test strategy
- Common quality requirements (unambiguous definitions and measures.)
- Activities prescribed by overall process.
- Guidelines for project staffing and assignment of roles.
Define a test strategy
Addresses risk and presents a process to reduce those.
Composed of test factors (risks/issues that need to be addressed) and Test phase (the phase of development life cycle where testing occurs.)
What are some test factors?
Correctness, file integrity, authorization, audit trail, access control, compliance, reliability, coupling, performance… etc
How to build a test strategy?
- select and rank test factors: specific to the system being developed.
- Identify system development phases: Obtained from the development methodology (waterfall, agile)
- Identify business risks with system: brainstorm including developers, users, customers and test personel.
- Place risks in test factor/test phase: risk team determine test phase in which each risk needs to be addressed by test team and which test factor risk is associated with.
What are the main elements of a test plan
- Items to be verified.
- Activities and resources
- approaches to be followed
- Criteria for evaluating results.
Define Quality Goals
Expressed as properties satisfied by the product
- Include metrics monitored during the project
Ex: “before entering acceptance testing, product must pass comprehensive systems testing”