The Test Analyst's Tasks in the Test Process Flashcards
What is the Test Implementation Phase about?
It is the Preparation of the testware needed for test execution based on Test Analysis and Test Design
List the activities of the test process
Test Planning
Test Monitoring & Control
Test Analysis
Test Design
Test Implementation
Test Execution
Test Completion
List the activities of the test process relevant to the TA
Test Design
Test Analysis
Test Implementation
Test Execution
True or False
The moment of involvement for the TA is different for various SDLCs
True
Name some types of information a TA typically supplies to other roles within the organization
- requirements reviews feedback
- schedule input for Project Management
- version control information and build verification testing results to Config and Change Mngmnt
- Defect notifications
- Defect Reporting and confirmation testing
- Documentation on workarounds and known issues for Technical support
- adding to technical documentation with specifications; test env documentation and/ or reviewing of these documents
What are typical additional tasks which may be performed by the TA during Test Execution?
- recognition of defect clusters
- suggestions for future exploratory testing sessions based
on findings from exploratory testing - the info acquired when performing test execution tasks can be used to identify new risks
- suggestions for improving any of the work products from the test implementation activity e.g. test procedures
True or False
Tests must also be understandable for other stakeholders
true
developers, auditors
list Test implementation activities
- development of test procedures and or creation of automated test scripts
- organization of test procedures and automated test scripts into test suites to be executed in a specific test run
- consultation of Test Manager in prioritizing test cases and test suites for execution
- creation of test execution schedule, incl resource allocation
- finalization of test data and test environments
updating traceability between test basis and testware (test conditions, test cases, procedures, scripts and suites)
name 4 experience-based test techniques
error guessing
exploratory testing
Checklist based testing
Defect based testing
what about test analyis, -design and -implementation when choosing for exploratory testing?
they still take place but during the test execution
name 4 drawbacks of unscripted testing (reactive test strategy)
- expertise is required
- test duration is difficult to predict
- coverage is difficult to define/track
- reproducibility can be difficult without good documentation or tool support
list 3 tasks of risk sessions
risk identification
risk assessment
risk mitigation
what is risk assessment?
the categorization of risks and determining their risk level (likelihood x impact)
how do the contributions of the TA and the TTA differ in risk assessment?
TTA: finding and understanding likelihood of risk to happen
TA: understanding impact on business
how can a TA know how to classify impact ?
following the guidelines from the Test Manager
what can the TA do to mitigate defined risks?
reviewing software work products (requirements; designs; user documentation)
have test cases with clear pass/fail criteria
implementation of risk mitigation activities as identified in the test strategy and test plan
re-evaluate known risks based on additional incoming information so that likelihood and or impact can be adjusted
use the info from test execution to identify new possible risks
explain what is meant by “depth-first” and “breadth-first” when prioritizing tests based on risk
depth-first
all of the highest risk tests are run before any lower risk tests
tests are run in strict risk order
breadth-first
a sample of tests across all the identified risk areas are selected, using risk level to weight the selection. this ensures coverage of every risk at least once
list the different types of test techniques
black-box
experience-based
list black-box test techniques
equivalence partitioning
boundary value analysis
decision table
state transition
use case
classification tree
pairwise
classification tree diagram
what is equivalence partitioning?
a technique used to reduce the number of test cases required in order to test the handling of inputs, outputs, internal values and time-related values
Partitioning is used to create equivalence partitions
of sets of values that are assumed to be processed in the same manner
testing 1 representative value from a partition assumes coverage for all other items in the same partition
which other black box technique is best to combine with Equivalance Partitioning?
Boundary value analysis, including the edges of the partitions into your test values
how do you measure test coverage when using the equivalence partitioning test technique?
By dividing the No of partitions tested using a value from that partition by the total No of partitions
what is pairwise testing?
the testing of combinations of input parameters each having several possible values.
what is a parameter-value pair?
the combination of a specific parameter (variable or factor) with a specific value of that parameter e.g.
parameter = color
value = red
how do you measure test coverage when using the Pair-wise test technique?
the 100% pairwise coverage requires every pair of values of any pair of parameters to be included in at least one combination
which other black box technique is best to combine with Pairwise testing?
Classification tree design
visualizing combinations of parameter-value pairs
equivalance partitioning, dividing the parameter-value pairs in partitions first to reduce the No of parameter values.
what is use case testing?
emulation of intended use of the system as specified by the use case.
a use case describes interactions b’ween actor and system.
which testing phases is use case testing generally used for?
system test
acceptance test
integration test ->if behavior of components/systems is specified by a use case
performance test -> use case portray realistic usage of the system
list some experience-based test techniques
error guessing
checklist based testing
exploratory testing
defect-based test techniques
what is the difference between two-value boundary and three-value boundary testing?
two -value boundary = the value on the boundary + the value just outside the boundary
three-value boudnary = the value before, on and just outside the boundary
list the product quality characteristics to be covered ty the TA as provided by the ISO 25010
functional suitability
usability
portability
compatibility
list per characteristic the sub-characteristics relevant to the TA following ISO 25010
functional suitability:
-f. correctness
-f. appropriateness
-f. completeness
usability
- appropriateness recognizability
- learnability
- operability
- user interface aesthetics
- user error protection
- accessibility
portability
- adaptability
- installability
- replaceability
compatibility
- interoperability
Why does quality characteristic testing require particular attention to
- the SDLC timing;
- tools required,
- software- and documentation availability and
- technical expertise?
an adequate planning
ramp up time
test execution time
may not be present without a strategy to deal with each individual characteristic
why should the TA be aware also of the other characteristics covered by a TTA?
- understanding of overlapping areas that may affect the testing of the characteristics the TA is responsible for.
E.g. failing performance leads to failed usability test if its too slow for the user to use.
and the other way round
E.g. interoperability issues means a compontent is not ready for portability testing
list the sub characteristics of usability
appropriateness recognizability
learnability
operability
user interface aesthetics
user error protection
accessibility
(aauulo)
list the sub characteristics of functional suitability
functional appropriateness
functional completeness
functional correctness
(cac)
list the sub characteristics of portability
adaptability
installability
replaceability
(ari ira ria)
list the sub characteristic of compatibility
interoperability
What test techniques are appropriate to test the functional completeness; functional correctness and, functional appropriateness?
a wide variety of test techniques
to test the specific functionality + regression testing for all unchanged functionality
which test conditions are required to verify the functional or non functional quality characteristics the TA is responsible for?
the test conditions can be decided based on the given set of requirements
What is the role of the TA in portability testing?
What is the role of the TA in interoperability testing?
Specifying tests for interoperability requires that combinations of the intended target environments are
identified, configured and available to the test team. These environments are then tested using a
selection of functional suitability test cases which exercise the various data exchange points present in
the environment
Analyst must understand these interactions and be able to create the conditions that will exercise the
various interactions
What are the typical defects to be targeted for the functional completeness-, correctness and, appropriateness?
when should the functional completeness-, correctness and, appropriateness be tested in the SDLC?
Functional suitability tests vary
according to the test level in which they are conducted and can also be influenced by the SDLC. For
example, a functional suitability test conducted during integration testing will test the functional suitability
of interfacing components which implement a single defined function. At the system test level, functional
suitability tests include testing the functional suitability of the system as a whole. For systems of systems,
functional suitability testing will focus primarily on end-to-end testing across the integrated system
Functional correctness testing can be conducted at any test level
Functional appropriateness testing is usually conducted during system
testing, but may also be conducted during the later stages of integration testing
Measuring
functional completeness may vary according to the particular test level and/or the SDLC used. For
example, functional completeness for Agile software development may be based on implemented user
stories and features. Functional completeness for system integration testing may focus on the coverage
of high-level business processes.
Which approaches are suitable to verify and validate the implementation of the usability requirements i.c.w. fulfilling the users’ expectations?
Testing
Reviews
Survey and questionnaires
what is interoperability testing?
verifying the exchange of information between two or more systems or components
it relates to how different components and software systems interact with each other.
what are the necessary test conditions for interoperability testing?
combinations of the intended target environments are identified, configured and available to the test team.
what should interoperability testing cover to ensure the data exchange works properly??
the testing should cover all the intended target environments including variations in hardware, software, middleware, operation system etcetera.
what can be used as a measure of interoperability?
Number of changes and effort required to implement and test changes
what can be said about software with good interoperability characteristics?
that the software can be integrated with a number of other systems
without requiring major changes
or
significant impact on non-functional behavior.
when does interoperability testing come into the picture?
during component integration and system integration testing
system integration testing: to determine how well the fully developed system interacts with other systems
what typical defects can be found during interoperability testing?
incorrect data exchange between interacting components
which black box test techniques are applicable during interoperability testing?
equivalence partitioning
boundary value analysis
decision tables
state transition diagrams
use cases
pairwise testing
What is portability testing?
it verifies the degree to which a software component or system can be transferred into its intended environment
- as a new installation
or - from an existing environment
who’s responsibility is it to identify risks and to design tests for the portability characteristics (installability; replaceability; adaptability)
shared between the TA and TTA
list test objectives for installability testing (portability testing)
- validation of different configurations
- verification functionality is available and working correctly upon under set configurations
- functional correctness of (de) installation procedures
- usability of de) installation procedure (clear instructions and error messages)
what is checked with adaptability testing?
whether a given application can be adapted effectively and efficiently to function correctly in all intended target environments: hardware; software; middleware; operating system; cloud; etc)
what needs the TA to analyze before being able to set up adaptability tests?
- identification of the intended environments the software should function on: such as mobile operation system versions ; browser versions
wat is replaceability testing?
the ability of software components or versions within a system to be exchanged for others
when is replaceability testing performed?
in parallel with functional integration tests
-> where more than 1 alternative component is available for integration into the complete system
how is coverage measured for boundary value?
Coverage is determined by taking the number of boundary conditions that are tested and dividing that
by the number of identified boundary conditions (either using the two-value or three-value method). The
coverage is stated as a percentage.
Similar to equivalence classes, in the case of multiple parameters, the Test Analyst should choose a
simple or combinatorial coverage type, depending on the risk.
how is coverage measured for equivalence partitioning?
No of partitions from which a test value has been used in a test case / total no of partitions
what is 0-switch coverage?
No of state transitions made for state transition test.
0-switch means from 1 state to another state, meaning 1 transition.
visit every state and traverse every transition at least once = 100% 0-switch coverage
what is 1-switch coverage?
No of state transitions made for state transition test.
1-switch means from 1 state to another state, then to another state, meaning 2 transitions.
e
explain N-switch coverage
the no of switches covered of length N=1, as a % of the total no of switches of that length.
e.g. 100% 1-switch coverage = every valid sequence of 2 successive transitions have been tested once.
100% 0-switch coverage = every valid transition of 1 have been tested e.g. from S1 to S2
what is round trip coverage?
it regards the state transition test: that all possible loops from one state back to the same state should be covered.
100% round trip coverage = when all loops from any state back to the same state have been tested for all states at which loops begin and end. this loop cannot contain more than one occurrence of any particular state except the initial/final one
S1-S1
S2-S2
S3-S3
etcetera
what are the benefits of experience based test techniques?
what are drawbacks of experience based test techniques as compared to black box and defect based test techniques?
reproducability
what are the principles of experience based test techniques?
how do defect based test techniques differ from black box test techniques?
what are defect based test techniques?
what types of defects are likely to be found using Equivalence Partitioning?
defects in the handling of various data values
what types of defects are likely to be found using Boundary Value Analysis?
displacement or omission of boundaries
defects regarding the handling of the boundary values
what types of defects are likely to be found using Decision Table Testing?
incorrect logic related processing based on particular combinations of conditions resulting in unexpected results.
unspecified results
omissions or contradictions of actions being not defined/specified
what types of defects are likely to be found using State Transition Testing?
incorrect event types or values
incorrect action types or values
incorrect initial state
inability to reach some exit state(s)
inability to enter required states
extra (unnecessary) states
inability to execute some valid transition(s) correctly
ability to execute invalid transitions
wrong guard conditions
omissions and or contradictions in the specifications
what types of defects are likely to be found using Pairwise testing?
defects related to the combined values of two parameters
what types of defects are likely to be found using Use Case testing?
mishandling of defined behaviors
missed alternative behaviors
incorrect processing of the conditions presented
poorly implemented or incorrect error messages
how is coverage measured for use case testing?
1 test case for basic behavior
additional test cases to cover each alternative and error handling behavior
how is coverage measured for pair wise testing?
include every pair of values of any pair of parameters in minimally 1 combination
parameter-value pair
e.g. color-red
how is coverage measured for the classification tree technique?
e.g. for minimum class coverage, ensure all values in a classification are at least tested once
how is coverage measured for the Decision Table Testing?
No of rules covered by test case / total no of feasible rules
what is meant with a collapsed decision table?
a technique to systematically reduce the no of combinations.
how is coverage measured for the Boundary Value Analysis?
No of boundary conditions tested / no of identified boundary conditions (2 or 3 value method)
how is coverage measured for Equivalence Partitioning?
No of partitions tested / no of identified partitions
chapter 6 automation tools
chapter 5 review
list disadvantages of experience based testing
less relevant for systems requiring detailed test documentation
reproducibility: less reproducibility possible
coverage: calculating coverage is limited
automation: tests less suited for subsequent automation
list advantages of experience based testing
system documentation lacking: then its a good alternative for more structured approaches
restricted testing time
other roles than testers but with domain knowledge can test
early feedback for developers
helps the team to get familiar with the software
operational failures analysis
diversity of test techniques
name some experience based test techniques
Error Guessing
checklist based testing
exploratory testing
defect based testing
list the characteristics of good exploratory testing
it is planned, interactive and creative
how can exploratory testing be managed?
by timeboxing - determining the time allowed for the testing upfront
using a test charter - designating the areas to be covered in a test session beforehand
debriefing session held by Test Manager to gather test results and determine follow up test charters
name some limitations/difficulties of exploratory testing
no coverage measure
reproducing test can be difficult
tracking test sessions in test management system can be difficult
why is a test charter helpful for exploratory testing?
the charter may identify where to focus the test effort, define what is in and out of scope for the test session and what resources should be committed to complete the planned tests .
chapter 6 automation tools
chapter 5 review
what is 1-wise test case?
1-wise says that all classes should be covered at least once, for 100% coverage
what is pairwise testing?
pairwise testing ensures that each parameter-value pair gets tested once against each parameter-value pair of each other parameter
it is, all pairs of parameter-value pairs for any two different parameters get tested.
hereby avoiding testing all combinations of parameter-value pairs
how to reach 100% pairwise coverage?
it requires every pair of values of any pair of parameters be included in at least once combination
when is a use case valid?
when in conveys realistic user transactions
what reduces the value of a use case?
if the use case does not reflect real user and organizational requirements
what is use case testing?
transactional, behavior-based tests emulating intended use of a component or system as specified by the use case.
use cases are defined in terms of interactions b’ween the actors and a component /or system.
what are defect based techniques?
A defect-based test technique is one in which the type of defect sought is used as the basis for test
design
what is round trip coverage?
state transition test technique.
it applies to the situations in which sequences of transitions form loops
when all loops from any state back to the same state have been tested for all states at which loops begin and end, then 100% round trip coverage is being achieved.
how is coverage measured for the state transition test technique?
coverage N means N+1 transitions are covered.
0 switch coverage: every state + every transition is covered e.g. from state A via transition 1 to state B
1 switch coverage = 2 transitions, from state A via transition 1 to state B via transition 2 to state C
how does a classification tree help a TA?
it helps identify parameters (classifications) and their equivalence partitions (classes)
for which black-box test techniques can a classification tree be of help/additional value?
pair wise testing
boundary value analysis
equivalence partitioning
all pairs coverage = each parameter value pair gets tested with every other parameter value pair.
how many testcases are needed when having 3 parameters each having 4 values?
case # Value 1 Value 2 Value 3
1 house wood city
2 house concrete suburb
3 house brick countryside
4 house mixed wilderness
5 semi-detached wood suburb
6 semi-detached concrete countryside
7 semi-detached brick wilderness
8 semi-detached mixed city
9 apartment wood countryside
10 apartment concrete wilderness
11 apartment brick city
12 apartment mixed suburb
13 cottage wood wilderness
14 cottage concrete city
15 cottage brick suburb
16 cottage mixed countryside
elk huis type wordt gecombineerd met elk mogelijk materiaal type.
elk materiaal type wordt ook weer gecombineerd met elke locatie type. hiermee zijn alle parameter-value pairs afgedekt.
all pairs coverage = each parameter value pair gets tested with every other parameter value pair.
and how many testcases are needed when having 3 parameters (language, browser, OSA) each having 3 values?
TC Language Browser OS
1 English Br1 OpS1
2 English Br2 OpS2
3 English Br3 OpS3
4 French Br1 OpS3
5 French Br2 OpS1
6 French Br3 OpS2
7 Japanese Br1 OpS2
8 Japanese Br2 OpS3
9 Japanese Br3 OpS1
je ziet, de talen worden met elke browser type gecombineerd.
elk browsertype zelf wordt oook weer met elke OS type gecombineerd. hierdoor maar 9 test cases nodig. zijn dus niet alle denkbare opties maar wel genoeg voor zogeheten all pair coverage dwz dat alle parameter-value pairs afgedekt zijn.
how to calculate to minimum no of testcases needed for use case testing?
1 TEST CASE for the mainstream path
+ testcases for every exception path
when is functional suitability testing done/in which sdlc moment?
functional suitability test conducted during integration testing will test the functional suitability
of interfacing components which implement a single defined function.
At the system test level, functional
suitability tests include testing the functional suitability of the system as a whole.
For systems of systems,
functional suitability testing will focus primarily on end-to-end testing across the integrated systems.
list the stages of the SDLC in correct order
explain the sub quality characteristic functional correctness
of the software quality characteristic functional suitability
Functional correctness involves verifying the application’s adherence to the specified or implied
requirements and may also include computational accuracy
explain what the sub quality characteristic functional appropriateness focuses on
Functional appropriateness testing involves evaluating and validating the appropriateness of a set of
functions for its intended specified tasks.
when in the SDLC is functional appropriateness tested?
Functional appropriateness testing is usually conducted
during system testing, but may also be conducted during the later
stages of integration testing
when in the SDLC is functional correctness tested?
at any test level of the SDLC
when in the SDLC is functional completeness tested?
that depends following the particular test level and/or the SDLC being used.
E.g. in Agile software development, once user stories and features have been implemented.
E.g. for system integration testing the testing may focus on the coverage of high-level business processes
what is interoperability testing?
verification of the exchange of info between two or more systems or components
the exchange of info and the use of the info exchanged
what actions are necessary during the test implementation phase so that interoperability testing can be done in the execution phase?
combinations of the intended target environments are
identified
configured
available to the test team
the TA has created the relevant data that can be exchanged/used for the interoperability testing.
what characterizes software with good interoperability?
that the software can be integrated with a number of other systems without requiring major changes or significant impact on non-functional behavior.
what is interoperability about?
(sub characteristic of the software quality characteristic ‘Compatibility’
how different components and software systems interact with each other
what can be used as a measure of interoperability?
the No of changes and effort required to implement and test changes when integrating the software with other systems
when is interoperability testing done?
during component integration and system integration testing
which black box test techniques are relevant for interoperability testing?
EP, BVA, decision tables, State transition, Use case, pairwise
all techniques are applicable.
to do blz 43 syllabus, wanneer welke software quality sub characteristic testen in de sdlc? wat is het. wat doet het etcetera.
ch 5
ch 6
examen vragen betrekken in kaartjes
