chapter 5 Managing the test activities Flashcards

1
Q

The process of recognizing, recording, classifying, investigating, resolving and disposing of defects

A

Defect management

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q
  • Documentation of the occurrence, nature, and status of a defect. Synonyms: bug report
A

Defect report

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

The set of conditions for officially starting a defined task. Reference: Gilb and Graham See also: exit criteria

A

Entry criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

The set of conditions for officially completing a defined task.

Synonyms?

A

Exit criteria

Synonyms: test completion criteria, completion criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q
  • A risk impacting the quality of a product. See also: risk
A

Product risk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

A risk that impacts project success. See also: risk

A

Project risk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

A factor that could result in future negative consequences.

A

Risk

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

The overall process of risk identification and risk assessment

A

Risk analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

The process to examine identified risks and determine the risk level

A

Risk assessment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

The overall process of risk mitigation and risk monitoring

A

Risk control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q
  • The process of finding, recognizing and describing risks.
A

Risk identification

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q
  • The measure of a risk defined by risk impact and risk likelihood.
A

Risk level

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q
  • The process for handling risks
A

Risk management

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q
  • The process through which decisions are reached and protective measures are implemented for reducing or maintaining risks to specified levels
A

Risk mitigation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q
  • The activity that checks and reports the status of known risks to stakeholders
A

Risk monitoring

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q
  • Testing in which the management, selection, prioritization, and use of testing activities and resources are based on corresponding risk types and risk levels.
A

Risk-based testing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q
  • The manner of implementing testing tasks
A

Test approach

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q
  • A type of test report produced at completion milestones that provides an evaluation of the corresponding test items against exit criteria.
A

Test completion report

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

The activity that develops and applies corrective actions to get a test project on track when it deviates from what was planned. See also: test management

A

Test control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

The activity that checks the status of testing activities, identifies any variances from planned or expected, and reports status to stakeholders. See also: test management

A

Test monitoring

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Documentation describing the test objectives to be achieved and the means and the schedule for achieving them, organized to coordinate testing activities. Reference: After ISO 29119-1 See also: master test plan, level test plan, test scope

A

Test plan

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q
  • The activity of establishing or updating a test plan.
A

Test planning

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q
  • A type of periodic test report that includes the progress of test activities against a baseline, risks, and alternatives requiring a decision. Synonyms: test status report
A

Test progress report

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q
  • A graphical model representing the relationship of the amount of testing per level, with more at the bottom than at the top
A

Test pyramid

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

A classification model of test types/test levels in four quadrants, relating them to two dimensions of test objectives: supporting the product team versus critiquing the product, and technology-facing versus business-facing

A

Testing quadrants

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

A test plan describes the objectives, resources and processes for a test project. A test plan:

A

Documents the means and schedule for achieving test objectives

Helps to ensure that the performed test activities will meet the established criteria

Serves as a means of communication with team members and other stakeholders

Demonstrates that testing will adhere to the existing test policy and test strategy (or explains why the testing will deviate from them)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

the efforts needed to achieve the test project objectives.

A

Test Plan

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

In iterative SDLCs, typically two kinds of planning occur:

A

release planning and iteration planning

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Test planning guides the testers’ thinking and forces the testers to confront the future challenges related to

A

risks, schedules, people, tools, costs, effort, etc.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

The typical content of a test plan includes:

A

Context of testing (e.g., scope, test objectives, constraints, test basis)

Assumptions and constraints of the test project

Stakeholders (e.g., roles, responsibilities, relevance to testing, hiring and training needs)

Communication (e.g., forms and frequency of communication, documentation templates)

Risk register (e.g., product risks, project risks)

Test approach (e.g., test levels, test types, test techniques, test deliverables, entry criteria and exit criteria, independence of testing, metrics to be collected, test data requirements, test environment requirements, deviations from the organizational test policy and test strategy)

Budget and schedule

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

looks ahead to the release of a product, defines and re-defines the product backlog, and may involve refining larger user stories into a set of smaller user stories.

It also serves as the basis for the test approach and test plan across all iterations.

A

release planning

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Testers involved in release planning participate in

A

writing testable user stories and acceptance criteria (see section 4.5),

participate in project and quality risk analyses (see section 5.2),

estimate test effort associated with user stories (see section 5.1.4),

determine the test approach,

and plan the testing for the release.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

looks ahead to the end of a single iteration and is concerned with the iteration backlog.

A

iteration planning

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Testers involved in iteration planning:

A

participate in the detailed risk analysis of user stories,

determine the testability of user stories,

break down user stories into tasks (particularly testing tasks),

estimate test effort for all testing tasks,

identify and refine functional and non-functional aspects of the test object.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

define the preconditions for undertaking a given activity.

A

Entry criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

If entry criteria are not met, it is likely that the activity will prove to be

A

more difficult, time-consuming, costly, and riskier.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

define what must be achieved in order to declare an activity completed.

A

Exit criteria

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

Entry criteria and exit criteria should be

A

defined for each test level, and will differ based on the test objectives.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

Typical entry criteria include:

A

availability of resources (e.g., people, tools, environments, test data, budget, time),

availability of testware (e.g., test basis, testable requirements, user stories, test cases),

and initial quality level of a test object (e.g., all smoke tests have passed).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

Typical exit criteria include:

A

measures of thoroughness (e.g., achieved level of coverage, number of unresolved defects, defect density, number of failed test cases),

and completion criteria (e.g., planned tests have been executed, static testing has been performed, all defects found are reported, all regression tests are automated).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

Running out of time or budget can also be viewed as valid

A

exit criteria

Even without other exit criteria being satisfied, it can be acceptable to end testing under such circumstances, if the stakeholders have reviewed and accepted the risk to go live without further testing.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

In Agile software development, exit criteria are often called

A

Definition of Done

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

defining the team’s objective metrics for a releasable item

A

Definition of Done

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

Entry criteria that a user story must fulfill to start the development and/or testing activities are called

A

Definition of Ready

45
Q

involves predicting the amount of test-related work needed to meet the objectives of a test project.

A

Test effort estimation

It is important to make it clear to the stakeholders that the estimate is based on a number of assumptions and is always subject to estimation error.

46
Q

Estimation for small tasks is usually more accurate than for the large ones. Therefore, when estimating a large task, it can

A

be decomposed into a set of smaller tasks which then in turn can be estimated.

47
Q

estimation techniques

A

Estimation based on ratios

Extrapolation.

Wideband Delphi.

Three-point estimation

48
Q

-In this metrics-based technique, figures are collected from previous projects within the organization, which makes it possible to derive “standard” ratios for similar projects.

A

Estimation based on ratios

49
Q

-The ratios of an organization’s own projects (e.g., taken from historical data) are generally the best source to use in the estimation process.

A

Estimation based on ratios

-These standard ratios can then be used to estimate the test effort for the new project. -For example, if in the previous project the development-to-test effort ratio was 3:2, and in the current project the development effort is expected to be 600 person-days, the test effort can be estimated to be 400 person-days.

50
Q

In this metrics-based technique, measurements are made as early as possible in the current project to gather the data

A

Extrapolation

Having enough observations, the effort required for the remaining work can be approximated by extrapolating this data (usually by applying a mathematical model). This method is very suitable in iterative SDLCs. For example, the team may extrapolate the test effort in the forthcoming iteration as the averaged effort from the last three iterations.

51
Q

-Data source: Data is gathered from past projects within the organization.

A

Estimation based on ratios

52
Q

-Data source: Data is gathered from the current project’s early stages or iterations.

A

Extrapolation

53
Q

In this iterative, expert-based technique, experts make experience-based estimations. Each expert, in isolation, estimates the effort.

A

Wideband Delphi

-The results are collected and if there are deviations that are out of range of the agreed upon boundaries, the experts discuss their current estimates. Each expert is then asked to make a new estimation based on that feedback, again in isolation. This process is repeated until a consensus is reached. Planning Poker is a variant of Wideband Delphi, commonly used in Agile software development. In Planning Poker, estimates are usually made using cards with numbers that represent the effort size.

54
Q

a consensus based estimation

A

Wideband Delphi

55
Q

In this expert-based technique, three estimations are made by the experts: the most optimistic estimation (a), the most likely estimation (m) and the most pessimistic estimation (b). The final estimate (E) is their weighted arithmetic mean.

A

Three-point estimation.

56
Q

In the most popular version of this technique, the estimate is calculated as E = (a + 4m + b) / 6. The advantage of this technique is that it allows the experts to calculate the measurement error: SD = (b – a) / 6. For example, if the estimates (in personhours) are: a=6, m=9 and b=18, then the final estimation is 10±2 person-hours (i.e., between 8 and 12 person-hours), because E = (6 + 49 + 18) / 6 = 10 and SD = (18 – 6) / 6 = 2.

A

Three-point estimation.

57
Q

test case prioritization: Once the test cases and test procedures are specified and assembled into test suites,

A

these test suites can be arranged in a test execution schedule that defines the order in which they are to be run

58
Q

When prioritizing test cases, different factors can be taken into account. The most commonly used test case prioritization strategies are as follows:

A

Risk-based prioritization

Coverage-based prioritization,

Requirements-based prioritization

Ideally, test cases would be ordered to run based on their priority levels, using, for example, one of the above-mentioned prioritization strategies. However, this practice may not work if the test cases or the features being tested have dependencies. If a test case with a higher priority is dependent on a test case with a lower priority, the lower priority test case must be executed first.

59
Q

where the order of test execution is based on the results of risk analysis (see section 5.2.3). Test cases covering the most important risks are executed first.

A

Risk-based prioritization,

60
Q

, where the order of test execution is based on coverage (e.g., statement coverage). Test cases achieving the highest coverage are executed first. In another variant, called additional coverage prioritization, the test case achieving the highest coverage is executed first; each subsequent test case is the one that achieves the highest additional coverage.

A

Coverage-based prioritization

61
Q

where the order of test execution is based on the priorities of the requirements traced back to the corresponding test cases. Requirement priorities are defined by stakeholders. Test cases related to the most important requirements are executed first.

A

Requirements-based prioritization,

62
Q

The order of test execution must also take into account the availability of resources. For example:

A

the required test tools, test environments or people that may only be available for a specific time window.

63
Q

is a model showing that different tests may have different granularity.

A

The test pyramid

64
Q

supports the team in test automation and in test effort allocation by showing that different goals are supported by different levels of test automation

A

The test pyramid model

65
Q

The pyramid layers represent groups of tests.

The higher the layer, the lower the test granularity (small parts ), test isolation and test execution time.

A

Test pyramid

66
Q

testing pyramid: The number and naming of the layers may differ. For example, the original test pyramid model (Cohn 2009) defines three layers:

A

“unit tests”, “service tests” and “UI tests”.

Another popular model defines unit (component) tests, integration (component integration) tests, and end-to-end tests. Other test levels (see section 2.2.1) can also be used.

67
Q

group the test levels with the appropriate test types, activities, test techniques and work products in the Agile software development.

A

testing quadrants

68
Q

This model also provides a way to differentiate and describe the types of tests to all stakeholders, including developers, testers, and business representatives.

A

testing quadrants

69
Q

Quadrant Q1

A

(technology facing, support the team). This quadrant contains component and component integration tests. These tests should be automated and included in the CI process.

70
Q

Quadrant Q2

A

(business facing, support the team). This quadrant contains functional tests, examples, user story tests, user experience prototypes, API testing, and simulations. These tests check the acceptance criteria and can be manual or automated.

71
Q

Quadrant Q3

A

(business facing, critique the product). This quadrant contains exploratory testing, usability testing, user acceptance testing. These tests are user-oriented and often manual.

72
Q

Quadrant Q4

A

(technology facing, critique the product). This quadrant contains smoke tests and non-functional tests (except usability tests). These tests are often automated.

73
Q

allows the organizations to increase the likelihood of achieving objectives, improve the quality of their products and increase the stakeholders’ confidence and trust.

A

Risk management

74
Q

The test approach, in which test activities are selected, prioritized, and managed based on risk analysis and risk control,

A

risk-based testing

75
Q

The main risk management activities are:

A

Risk analysis (consisting of risk identification and risk assessment; see section 5.2.3)


Risk control (consisting of risk mitigation and risk monitoring; see section 5.2.4)

76
Q

is a potential event, hazard, threat, or situation whose occurrence causes an adverse effect.

A

Risk

77
Q

A risk can be characterized by two factors:

A

Risk likelihood – the probability of the risk occurrence (greater than zero and less than one)


Risk impact (harm) – the consequences of this occurrence


These two factors express the risk level, which is a measure for the risk. The higher the risk level, the more important is its treatment.

78
Q

In software testing one is generally concerned with two types of risks:

A

project risks and product risks.

79
Q

Project risksare related to the management and control of the project.

Project risks include:

A

Organizational issues (e.g., delays in work products deliveries, inaccurate estimates, cost-cutting)

People issues (e.g., insufficient skills, conflicts, communication problems, shortage of staff)

Technical issues (e.g., scope creep, poor tool support)

Supplier issues (e.g., third-party delivery failure, bankruptcy of the supporting company)

80
Q

It happens when new features, requirements, or tasks are added to a project after it has already begun, often without proper review or adjustment to the project plan.

A
  • scope creep:
81
Q

when they occur, may have an impact on the project schedule, budget or scope, which affects the project’s ability to achieve its objectives.

A

Project risks,

82
Q

are related to the product quality characteristics (e.g., described in the ISO 25010 quality model).

A

Product risks

83
Q

Examples of product risks include:

A

missing or wrong functionality,

incorrect calculations,

runtime errors,

poor architecture,

inefficient algorithms,

inadequate response time,

poor user experience,

security vulnerabilities.

84
Q

Product risks, when they occur, may result in various negative consequences, including:

A

User dissatisfaction

Loss of revenue, trust, reputation

Damage to third parties

High maintenance costs, overload of the helpdesk

Criminal penalties

In extreme cases, physical damage, injuries or even death

85
Q

From a testing perspective, the goal of this risk is to provide an awareness of product risk in order to focus the testing effort in a way that minimizes the residual level (remaining level) of product risk. Ideally, this risk analysis begins early in the SDLC.

A

product risk analysis

86
Q

Product risk analysis consists of

A

risk identification and risk assessment.

87
Q

is about generating a comprehensive list of risks. Stakeholders can identify risks by using various techniques and tools, e.g., brainstorming, workshops, interviews, or cause-effect diagrams.

A

Risk identification

88
Q

Risk assessment involves:

A

categorization of identified risks,

determining their risk likelihood,

risk impact and level,

prioritizing,

proposing ways to handle them.

Categorization helps in assigning mitigation actions, because usually risks falling into the same category can be mitigated using a similar approach. (Mitigate is to make less severe)

89
Q

can use a quantitative or qualitative approach, or a mix of them.

A

-Risk assessment

90
Q

the risk level is calculated as the multiplication of risk likelihood and risk impact.

A

-In the quantitative approach

(risk assessment)

91
Q

the risk level can be determined using a risk matrix.

A
  • In the qualitative approach

(risk assessment)

92
Q

Product risk analysis may influence the thoroughness and scope of testing. Its results are used to:

A

Determine the scope of testing to be carried out

Determine the particular test levels and propose test types to be performed

Determine the test techniques to be employed and the coverage to be achieved

Estimate the test effort required for each task

Prioritize testing in an attempt to find the critical defects as early as possible

Determine whether any activities in addition to testing could be employed to reduce risk

93
Q

comprises all measures that are taken in response to identified and assessed product risks.

A

Product risk control

94
Q

Product risk control consists of

A

risk mitigation and risk monitoring.

95
Q

involves implementing the actions proposed in risk assessment to reduce the risk level.

A

Risk mitigation

96
Q

The aim of this risk is to ensure that the mitigation actions are effective, to obtain further information to improve risk assessment, and to identify emerging risks.

A

risk monitoring

97
Q

With respect to product risk control, once a risk has been analyzed, several response options to risk are possible (Examples):

A

risk mitigation by testing,
risk acceptance,
risk transfer, or
contingency plan

98
Q

Actions that can be taken to mitigate the product risks by testing are as follows:

A

Select the testers with the right level of experience and skills, suitable for a given risk type

Apply an appropriate level of independence of testing

Conduct reviews and perform static analysis

Apply the appropriate test techniques and coverage levels

Apply the appropriate test types addressing the affected quality characteristics

Perform dynamic testing, including regression testing


99
Q

is a variant of Wideband Delphi, commonly used in Agile software development.

A

Planning Poker

In Planning Poker, estimates are usually made using cards with numbers that represent the effort size.

100
Q

three-point estimation, the estimate is calculated as

A

E = (a + 4m + b) / 6

three estimations are made by the experts: the most optimistic estimation (a), the most likely estimation (m) and the most pessimistic estimation (b). The final estimate (E) is their weighted arithmetic mean.

101
Q

three-point estimation, the measurement error is calculated as

A

SD = (b – a) / 6

three estimations are made by the experts: the most optimistic estimation (a), the most likely estimation (m) and the most pessimistic estimation (b). The final estimate (E) is their weighted arithmetic mean.

102
Q

is a tool used to assess and prioritize risks by evaluating the likelihood of an event occurring and the potential impact or severity if the event occurs.

A

A risk matrix

103
Q

Risk analysis consists of

A

risk identification and risk assessment

104
Q

Risk control consists of

A

risk mitigation and risk monitoring

105
Q

The degree to which test conditions can be established for a component or system, and tests can be performed to determine whether those test conditions have been met.

A

testability

106
Q

Context of testing in The typical content of a test plan includes:

A

(e.g., scope, test objectives, constraints, test basis)

107
Q

Stakeholders in The typical content of a test plan includes:

A

(e.g., roles, responsibilities, relevance to testing, hiring and training needs)

108
Q

Risk register in The typical content of a test plan includes:

A

(e.g., product risks, project risks)

109
Q

Test approach in The typical content of a test plan includes:

A

(e.g., test levels, test types, test techniques, test deliverables, entry criteria and exit criteria, independence of testing, metrics to be collected, test data requirements, test environment requirements, deviations from the organizational test policy and test strategy)