Module 1 - The Test Analysts Tasks in the Test Process Flashcards

1
Q

Introduction (1.1)

In the ISTQB® Foundation Level syllabus, the test process is described as including the following activities:

A

Test planning
Test monitoring and control
Test analysis
Test design
Test implementation
Test execution
Test completion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Introduction (1.1)

Determining the appropriate tests, designing and implementing them and then executing them are the primary areas of concentration for the Test Analyst. While it is important to understand the other steps in the test process, the majority of the Test Analyst’s work usually focuses on the following activities:

A

Test analysis
Test design
Test implementation
Test execution

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Intro to the SDLC (1.2)

The overall SDLC should be considered when…

The moment of involvement for the Test Analyst is…

The amount of involvement, time required, information available and expectations can be quite…

The Test Analyst must be aware of…

A

Defining a test strategy.

Different for the various SDLCs;

Varied as well.

The types of information to supply to other related organizational roles such as:

  • Requirements engineering and management - requirements reviews feedback
  • Project management - schedule input
  • Configuration and change management – results of build verification testing, version control information
  • Software development - notifications of defects found
  • Software maintenance - reports on defects, defect removal efficiency, and confirmation testing
  • Technical support - accurate documentation for workarounds and known issues
  • Production of technical documentation (e.g., database design specifications, test environment documentation) - input to these documents as well as technical review of the documents
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Intro to the SDLC (1.2)

Test activities must be aligned with the…

A

the chosen SDLC whose nature may be sequential, iterative, incremental, or a hybrid of these.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Intro to the SDLC (1.2)

In a Sequential V-Model, System test planning occurs…

This will influence…

A

concurrently with project planning, and test monitoring and control continues until test completion.

The schedule inputs provided by the Test Analyst for project management purposes.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Intro to the SDLC (1.2)

In a Sequential V-Model, System test analysis and design aligns with…

A

Documents such as the system requirements specification, system and architectural (high-level) design specification, and component (lowlevel) design specification.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Intro to the SDLC (1.2)

In a Sequential V-Model, Implementation of the system test environment might start during…

A

System design, though the bulk of it typically would occur concurrently with coding and component testing, with work on
system test implementation activities stretching often until just days before the start of system test execution.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Intro to the SDLC (1.2)

In a Sequential V-Model, System test execution begins when…

System test execution continues until…

A

The entry criteria are met or, if necessary, waived, which typically means that at least component testing and often also component integration testing have met their exit criteria.

The system test exit criteria are met

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Intro to the SDLC (1.2)

In a Sequential V-Model, System test completion activities occur…

A

After the system test exit criteria are met.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Intro to the SDLC (1.2)

Iterative and incremental models may…

For example, an iterative model may…

Test analysis, design, implementation, and execution may…

A

Not follow the same order of activities and may exclude some activities.

Utilize a reduced set of test activities for each iteration.

Be conducted for each iteration, whereas high-level planning is done at the beginning of the project, and completion tasks are done at the end.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Intro to the SDLC (1.2)

In Agile software development, it is common to use…

There may not be a well-defined…

There is less…

A

A less formalized process and a much closer working relationship with project stakeholders that allows changes to occur more easily within the project.

Test Analyst role.

Comprehensive test documentation, and communication is shorter and more frequent.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Intro to the SDLC (1.2)

Agile software development involves…

This starts from the…

Reviews may not be…

Involvement is expected to…

A

Testing from the outset.

Initiation of the product development as the developers perform their initial architecture and design work.

Formalized but are continuous as the software evolves.

Be throughout the project and Test Analyst tasks are expected to be done by the team.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Intro to the SDLC (1.2)

Iterative and incremental models range from…

In such hybrid models, Test Analysts should…

A

Agile software development, where there is an expectation for change as the customer requirements evolve, to hybrid models, e.g., iterative/incremental development combined with a V-model approach.

Be involved in the planning and design aspects of the sequential activities, and then move to a more interactive role during the iterative/incremental activities.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Intro to the SDLC (1.2)

Whatever the SDLC being used, Test Analysts need to…

Test Analysts provide an…

A

understand the expectations for involvement as well as the timing of that involvement.

Effective contribution to software quality by adjusting their activities and their moment of involvement to the specific SDLC rather than sticking to a pre-defined role model.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Test Analysis (1.3)

During test planning, the scope of the testing project is defined. During test analysis, Test Analysts use this scope definition to:

A
  • Analyze the test basis
  • Identify defects of various types in the test basis
  • Identify and prioritize test conditions and features to be tested
  • Capture bi-directional traceability between each element of the test basis and the associated test conditions
  • Perform tasks associated with risk-based testing (see Chapter 2)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Test Analysis (1.3)

In order for Test Analysts to proceed effectively with test analysis, the following entry criteria should be met:

A
  • There is a body of knowledge (e.g., requirements, user stories) describing the test object that can serve as its test basis (see [ISTQB_FL_SYL] Sections 1.4.2 and 2.2 or a list of other possible sources of test basis).
  • This test basis has passed review with reasonable results and has been updated as needed after the review. Note that if high-level test cases are to be defined (see Section 1.4.1), the test basis may not yet need to be fully defined. In Agile software development, this review cycle will be iterative as the user stories are refined at the beginning of each iteration.
  • There is an approved budget and schedule available to accomplish the remaining testing tasks for this test object.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Test Analysis (1.3)

Test conditions are typically identified by…

In some situations, where documentation may be old or non-existent, the test conditions may be identified by…

In Agile software development…

A

Analysis of the test basis in conjunction with the test objectives (as defined in test planning).

Discussion with relevant stakeholders (e.g., in workshops or during iteration planning).

The acceptance criteria, which are defined as part of user stories, are often used as the basis for the test design.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Test Analysis (1.3)

While test conditions are usually specific to the item being tested, there are some standard considerations for the Test Analyst.

1: It is usually advisable to define test conditions at…

Initially, high-level conditions are identified to…

Subsequently, more detailed conditions are identified as…

Using this type of hierarchical approach to defining test conditions can help to…

This approach also allows a Test Analyst to…

2: If product risks have been defined, then the

A

Differing levels of detail.

Define general targets for testing, such as “functionality of screen x”.

The basis of specific test cases, such as “screen x rejects an account number that is one digit short of the correct length”.

Ensure the coverage is sufficient for the high-level items.

Start working on defining high-level test conditions for user stories that have not yet been refined.

Test conditions that will be necessary to address each product risk should be identified and traced back to that risk item.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Test Analysis (1.3)

The application of test techniques (as identified within the test strategy and/or the test plan) can be helpful in test analysis activities and may be used to support the following objectives:

A
  • Identifying test conditions
  • Reducing the likelihood of omitting important test conditions
  • Defining more precise and accurate test conditions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Test Analysis (1.3)

After the test conditions have been identified and refined:

A

Review of these conditions with the stakeholders can be conducted to ensure the requirements are clearly understood and that testing is aligned with the goals of the project.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Test Analysis (1.3)

At the conclusion of the test analysis activities for a given area (e.g., a specific function), the Test Analyst should know…

A

What specific tests must be designed for that area.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Test Design (1.4)

Still adhering to the scope determined during test planning, the test process continues as the Test Analyst designs the tests which will be implemented and executed. Test design includes the following activities:

A
  • Determining in which test areas low-level or high-level test cases are appropriate
  • Determining the test technique(s) that will enable the necessary coverage to be achieved. The techniques that may be used are established during test planning.
  • Using test techniques to design test cases and sets of test cases that cover the identified test conditions
  • Identifying necessary test data to support test conditions and test cases
  • Designing the test environment and identifying any required infrastructure including tools
  • Capturing bi-directional traceability (e.g., between the test basis, test conditions and test cases)
23
Q

Test Design (1.4)

Prioritization criteria identified during risk analysis and test planning should be…

A

Applied throughout the process, from analysis and design to implementation and execution.

24
Q

Test Design (1.4)

Depending on the types of tests being designed, one of the entry criteria for test design may be:

A

The availability of tools that will be used during the design work.

25
Q

Test Design (1.4)

During test design, the Test Analyst must consider at least the following:

  • Some test items are better addressed by defining only the…
  • The pass/fail criteria should be…
  • Tests should be designed to be understandable by…
  • Tests must also be understandable for other…
  • Tests should cover all types of interaction with the test object and should not be restricted to the…
  • Tests should be designed to test the interfaces between the…
  • Test design effort must be prioritized and balanced to align with…
A

Test conditions rather than going further into the definition of test scripts, which give the sequence of instructions required to execute a test. In this case, the test conditions should be defined to be used as a guide for unscripted testing.

Clearly identified.

Other testers, not just the author. If the author is not the person who executes the test, other testers will need to read and understand previously specified tests in order to understand the test objectives and the relative importance of the test.

Stakeholders such as developers (who may review the tests) and auditors (who may have to approve the tests).

Interactions of people through the user-visible interface. They may also include, for example, interaction with other systems and technical or physical events. (see [IREB_CPRE] for further details).

Various test items, as well as the behaviors of the items themselves.

The risk levels and business value.

26
Q

Test Design - Low and High Level Test Cases (1.4.1)

One of the jobs of the Test Analyst is to determine…

A

The best design level of test cases for a given situation. Low-level and high-level test cases are covered in [ISTQB_FL_SYL].

27
Q

Test Design - Low and High Level Test Cases (1.4.1)

Advantages of low level test cases:

  • Inexperienced testing staff can rely on…
  • Tests may be rerun by…
  • Non-obvious defects in the…
  • The level of detail enables an…
  • Time spent on automated test case implementation can be…
A

Detailed information provided within the project. Lowlevel test cases provide all the specific information and procedures needed for the tester to execute the test case (including any data requirements) and to verify the actual results.

Different testers and should achieve the same test results.

Test basis can be revealed.

Independent verification of the tests, such as audits, if required.

Reduced.

28
Q

Test Design - Low and High Level Test Cases (1.4.1)

Disadvantages of low level test cases:

  • They may require a significant amount of …
  • They tend to limit…
  • They require that the test basis be…
  • Their traceability to test conditions may take…
A

Effort, both for creation and maintenance.

Tester ingenuity during execution.

Well defined.

More effort than with high-level test cases.

29
Q

Test Design - Low and High Level Test Cases (1.4.1)

Advantages of high level test cases:

  • They give guidelines for what should be tested, and allow the Test Analyst to…
  • They may provide better…
  • They can be defined…
  • They make use of the Test Analyst’s…
  • They can be defined when…
  • They are better suited for…
A

Vary the actual data or even the procedure that is followed when executing the test.

Risk coverage than low-level test cases because they will vary somewhat each time they are executed.

Early in the requirements process.

Experience with both testing and the test object when the test is executed.

No detailed and formal documentation is required.

Reuse in different test cycles when different test data can be used.

30
Q

Test Design - Low and High Level Test Cases (1.4.1)

Disadvantages of high level test cases:

  • They are less…
  • More experienced testing staff may be…
  • When automating on the basis of high-level test cases, the lack of…
A

Reproducible, making verification difficult. This is because they lack the detailed description found in low-level test cases.

Needed to execute them

Details may result in validating the wrong actual results or missing items that should be validated.

31
Q

Test Design - Low and High Level Test Cases (1.4.1)

High-level test cases may be used to…

In this case, the test case creation is done…

A

Develop low-level test cases when the requirements become more defined and stable.

Sequentially, flowing from high-level to low-level with only the low-level test cases being used for execution.

32
Q

Test Design - Design of Test Cases (1.4.2)

Test cases are designed by the…

Test cases should be…

A

stepwise elaboration and refinement of the identified test conditions
using test techniques (see Chapter 3).

Repeatable, verifiable and traceable back to the test basis (e.g., requirements).

33
Q

Test Design - Design of Test Cases (1.4.2)

Test design includes the identification of the following:

A
  • Objective (i.e., the observable, measurable objective of test execution)
  • Preconditions, such as either project or localized test environment requirements and the plans for their delivery, state of the system prior to test execution, etc.
  • Test data requirements (both input data for the test case as well as data that must exist in the system for the test case to be executed)
  • Expected results with explicit pass/fail criteria
  • Postconditions, such as affected data, state of the system after test execution, triggers for subsequent processing, etc.
34
Q

Test Design - Design of Test Cases (1.4.2)

A particular challenge with test case design can be…

Computing this manually is often…

In identifying the expected result, testers are concerned not only with…

If the test basis is clearly defined…

However, test basis documentation might be…

In such cases, a Test Analyst must have…

Also, even when the test basis is well specified, complex interactions of complex stimuli and responses can make…

In Agile software development, the test oracle might be…

Test case execution without any way to determine…

A

The definition of the expected result of a test.

Tedious and error-prone; if possible, it might be preferable to find or create an automated test oracle.

Outputs on the screen, but also with data and environmental postconditions.

Identifying the correct result, theoretically, should be simple.

vague, contradictory, lacking coverage of key areas, or missing entirely.

Subject matter expertise or have access to it.

The definition of the expected results difficult; therefore, a test oracle is essential.

The product owner.

Correctness of actual results might have a very low added value or benefit, often generating invalid test reports or false confidence in the system.

35
Q

Test Design - Design of Test Cases (1.4.2)

The activities described above may be applied to…

When analyzing and designing tests, it is important to remember…

This helps to determine the…

A

All test levels, though the test basis will vary.

The target level for the test as well as the objective of the test.

Level of detail required as well as any tools that may be needed (e.g., drivers and stubs at the component test level).

36
Q

Test Design - Design of Test Cases (1.4.2)

During the development of test conditions and test cases, some amount of documentation is typically created, resulting in test work products. In practice the extent to which test work products are documented varies considerably. This can be affected by any of the following:

A
  • Project risks (what must/must not be documented)
  • The added value which the documentation brings to the project
  • Standards to be followed and/or regulations to be met
  • SDLC or approach used (e.g., an Agile approach aims for “just enough” documentation)
  • The requirement for traceability from the test basis through test analysis and design
37
Q

Test Design - Design of Test Cases (1.4.2)

Depending on the scope of the testing, test analysis and design address…

A

The quality characteristics for the test object(s). The ISO 25010 standard [ISO25010] provides a useful reference. When testing hardware/software systems, additional characteristics may apply.

38
Q

Test Design - Design of Test Cases (1.4.2)

The activities of test analysis and test design may be enhanced by:

In fact, conducting the test analysis and test design are often…

Test analysis and test design based on the requirements specification is…

Reading the requirements to use them for creating tests requires…

This activity often uncovers…

Similarly, test work products such as…

A

intertwining them with reviews and static analysis.

A form of static testing because problems may be found in the test basis documents during this activity.

An excellent way to prepare for a requirements review meeting.

Understanding the requirement and being able to determine a way to assess fulfillment of the requirement.

Missing requirements, requirements that are not clear, are untestable, or do not have defined acceptance criteria.

Test cases, risk analyses, and test plans can be subjected to reviews.

39
Q

Test Design - Design of Test Cases (1.4.2)

If testing requires infrastructure that is not readily available, the Test Analyst should…

Should these requirements not be…

It must be remembered that…

For example the infrastructure requirements may include…

A

define the detailed test infrastructure requirements during test design.

Completed in time, test implementation will be at risk of overrun due to unexpected time and effort.

Test infrastructure includes more than test objects and testware.

Rooms, equipment, personnel, software, tools, peripherals, communications equipment, user authorizations, and all other items required to run the tests.

40
Q

Test Design - Design of Test Cases (1.4.2)

The exit criteria for test analysis and test design…

It is important that the exit criteria are…

A

Will vary depending on the project parameters, but all items discussed in these two sections should be considered for inclusion in the defined exit criteria.

Measurable and that all the information required for the subsequent steps has been provided and all necessary preparation has been performed.

41
Q

Test Implementation (1.5)

Test implementation prepares the testware needed for test execution based on test analysis and design. It includes the following activities:

A
  • Developing test procedures, and, potentially, creating automated test scripts
  • Organizing test procedures and automated test scripts (if there are any) into test suites to be executed in a specific test run
  • Consulting the Test Manager in prioritizing the test cases and test suites to be executed
  • Creating a test execution schedule, including resource allocation, to enable test execution to begin (see [ISTQB_FL_SYL] Section 5.2.4)
  • Finalizing preparation of test data and test environments
  • Updating the traceability between the test basis and testware such as test conditions, test cases, test procedures, test scripts and test suites.
42
Q

Test Implementation (1.5)

During test implementation, Test Analysts…

Defining the test procedures requires…

Test procedures document any…

A

identify an efficient execution order of test cases and create test procedures.

Carefully identifying constraints and dependencies that might influence the test execution sequence.

Initial preconditions (e.g., loading of test data from a data repository) and any activities following execution (e.g., resetting the system status).

43
Q

Test Implementation (1.5)

Test Analysts identify test procedures and automated test scripts that can be…

This enables…

A

Grouped (e.g., they all relate to the testing of a particular high-level business process), and organize them into test suites.

Related test cases to be executed together.

44
Q

Test Implementation (1.5)

Test Analysts arrange test suites within a test execution schedule in a way that…

If a risk-based test strategy is being used…

There may be other factors that determine…

A

Results in efficient test execution.

The risk level will be the primary consideration in determining the execution order for the test cases.

Test case execution order such as the availability of the right people, equipment, data and the functionality to be tested.

45
Q

Test Implementation (1.5)

It is not unusual for code to be…

Particularly in iterative and incremental development models, it is…

A

released in sections and the test effort has to be coordinated with the sequence in which the software becomes available for testing.

Important for the Test Analyst to coordinate with the development team to ensure that the software will be released for testing in a testable order.

46
Q

Test Implementation (1.5)

The level of detail and associated complexity for work done during test implementation may…

In some cases…

A

be influenced by the detail of the test conditions and test cases.

Regulatory rules apply, and test work products should provide evidence of compliance to applicable standards such as the United States standard DO-178C (in Europe, ED 12C). [RTCA DO-178C/ED-12C].

47
Q

Test Implementation (1.5)

Test data is needed for most testing, and in some cases these sets of data can be quite large. During implementation Test Analysts…

This data must be…

Test Analysts may also…

A

Create input and environment data to load into databases and other such repositories.

“fit for purpose” to enable detection of defects.

Create data to be used with data-driven and keyword-driven testing (see Section 6.2) as well as for manual testing.

48
Q

Test Implementation (1.5)

Test implementation is also concerned with the test environment(s). During this activity the environment(s) should be…

A “fit for purpose” test environment is essential, i.e., the test environment should be…

Test environment changes may be…

If environment changes do occur during execution, it is important to…

A

fully set up and verified prior to test execution.

Capable of enabling the exposure of the defects present during controlled testing, operate normally when failures are not occurring, and adequately replicate, if required, the production or end-user environment for higher test levels.

Necessary during test execution depending on unanticipated changes, test results or other considerations.

Assess the impact of the changes to tests that have already been run.

49
Q

Test Implementation (1.5)

During test implementation, Test Analysts should check that those responsible for the creation and maintenance of the test environment are…

This includes…

In addition, Test Analysts must…

A

known and available, and that all the testware and test support tools and associated processes are ready for use.

Configuration management, defect management, and test logging and management.

Verify the procedures that gather data for evaluating current status against exit criteria and test results reporting.

50
Q

Test Implementation (1.5)

It is wise to use a balanced approach to test implementation as determined during test planning. For example…

In this case…

A

risk-based analytical test strategies are often blended with reactive test strategies.

Some percentage of the test implementation effort is allocated to testing which does not follow predetermined scripts (unscripted).

51
Q

Test Implementation (1.5)

Unscripted testing should not be…

Rather, it should be…

Over the years, testers have developed…

Test analysis, test design, and test implementation still occur, but…

A

Random or aimless as this can be unpredictable in duration and coverage, and give a low yield in defects.

Conducted in time boxed sessions, each given initial direction by a test charter, but with the freedom to depart from the charter’s prescriptions if potentially more productive test opportunities are discovered in the course of the session.

a variety of experience-based test techniques, such as attacks [Whittaker03], error guessing [Myers11], and exploratory testing [Whittaker09].

They take place primarily during test execution.

52
Q

Test Implementation (1.5)

When following reactive test strategies (unscripted, experience based, error guessing), the results of each test influence…

While these strategies are lightweight and often effective at finding defects, there are some drawbacks, including the following:

A

The analysis, design, and implementation of the subsequent tests.

  • Expertise from the Test Analyst is required
  • Duration can be difficult to predict
  • Coverage can be difficult to track
  • Repeatability can be lost without good documentation or tool support
53
Q

Test Execution (1.6)

Test execution is conducted according to the test execution schedule and includes the following tasks:

A
  • Executing manual tests, including exploratory testing
  • Executing automated tests
  • Comparing actual results with expected results
  • Analyzing anomalies to establish their likely causes
  • Reporting defects based on the failures observed
  • Logging the actual results of test execution
  • Updating the traceability between the test basis and testware to consider test results
  • Executing regression tests
54
Q

Test Execution (1.6)

The following are typical additional test execution tasks which may be performed by the Test Analyst:

A
  • Recognizing defect clusters which may indicate the need for more testing of a particular part of the test object
  • Making suggestions for future exploratory testing sessions based on the findings from exploratory testing
  • Identifying new risks from information obtained when performing test execution tasks
  • Making suggestions for improving any of the work products from the test implementation activity (e.g., improvements to test procedures)