5.3 Test Monitoring, Control, and Completion

Question 1

What is test monitoring concerned with?

Answer 1

Test monitoring is concerned with gathering information about testing, which can be used to assess progress and measure whether test exit criteria and associated tasks are satisfied.

Question 2

How does test control use the information from test monitoring?

Answer 2

Test control provides, in the form of control directives, guidance and corrective actions to achieve the most effective and efficient testing. Examples include:

• reprioritizing tests when an identified risk becomes an issue
• re-evaluating whether a test item meets entry criteria or exit criteria following a rework
• adjusting the test schedule to address changes in the delivery of the test environment
• adding new resources when and where needed

Question 3

What is involved in test completion, and when does it occur?

Answer 3

Test completion collects data from completed test activities to consolidate experience, testware, and any other relevant information.

Test completion activities occur at project milestones such as when a test level is completed, an agile iteration is finished, a test project is completed or cancelled, a software system is released, or a maintenance release is completed.

Question 5

What value do test metrics provide?

Answer 5

Test metrics are gathered to show progress against the planned schedule and budget, the current quality of the test object, and the effectiveness of test activities with respect to objectives or an iteration goal.

Question 6

What are some common types of test metrics?

Answer 6

• Project progress metrics (task completion, resource usage, test effort)
• Test progress metrics (test case implementation, test environment preparation, number of test cases run/not run and passed/failed, test execution time
• Product quality metrics (availability, response time, mean time to failure)
• Defect metrics (numbers/priorities of defects found/fixed, defect density, defect detection percentage)
• Risk metrics (residual risk level)
• Coverage metrics (requirements coverage, code coverage)
• Cost metrics (cost of testing, organizational cost of quality)

Question 7

What information should test reports included in order to be complete enough to be useful?

Answer 7

• applicable test period
• test progress, including notable deviations
• impediments for testing and their workarounds (if any)
• test metrics
• new and changed risks within this period
• testing planned for the next period

Question 8

When is a test completion report generated?

Answer 8

During test completion, when a project, test level, or test type is complete and, ideally, when its exit criteria have been met. This report uses test progress reports and other data.

Question 9

What should a test completion report include?

Answer 9

• test summary
• testing and product quality evaluations based on the test plan
• deviations from the test plan
• testing impediments and workarounds
• test metrics based on test progress reports
• unmitigated risks, defects not fixed
• lessons learned relevant to the testing

Question 10

How can test status be communicated?

Answer 10

The best means varies depending on the strategies, standards, and needs of the team and project. Options include:
- verbal communication
- dashboards
- electronic channels (email, chat)
- online documentation
- formal test reports