1. Fundamentals of Testing Flashcards
provide a definition
Test Object
1.1. What is Testing?
A software artifact when its being tested
provide a definition
Some common problems resulting from incorrectly working software
1.1. What is Testing?
Loss of money, time or business reputation, and, in extreme cases, even injury or death
provide a definition
Software Testing
1.1. What is Testing?
A set of activities to discover defects and evaluate the quality of software artifacts
Answer a question
What assesses software quality and helps reducing the risk of software failure in operation?
1.1. What is Testing?
Software Testing
Answer a question
Does testing consists in only executing tests?
1.1. What is Testing?
No, software testing also includes other activities and must be aligned with the software development lifecycle
Provide a definition
Verification
1.1. What is Testing?
Checking whether the system meets specified requirements
Provide a defintion
Validation
1.1. What is Testing?
Checking whether the system meets users’ and other stakeholders’ needs in its operational environment.
Provide an example
Verification testing is successful, but validation testing has failed
1.1. What is Testing?
For example, there is a bug ticket describing business needs - an admin user wants to be able to use filtering on results on a certain page - and an exact issue - when filter is used then there appears an error and filtering is not being successful. A developer made a fix that fixed the error, however, filtering by some parameters still returns an empty results list, while it shoudn’t. Therefore, the fix meets the specified requirements (error is removed, filtering by some parameters works), however, it does not meet the users’ needs (filtering by some parameters is not possible)
Provide a definition
Dynamic Testing
1.1. What is Testing?
Involves the execution of software. Uses different types of test techniques and test approaches to derive test cases. Triggers failures that are caused by defects in the software
Provide a defintion
Static Testing
1.1. What is Testing?
Does not involve execution of software. Includes reviews and static analysis. Directly finds defects in the test object
Answer a question
What are the activites involved in test management?
1.1. What is Testing?
Planning, Estimation, Monitoring, Control
Answer a question
What is a role of tools in testing?
1.1. What is Testing?
Testers use tools, but it is important to remember that testing is largely an intellectual activity, requiring the testers to have specialized knowledge, use analytical skills and apply critical thinking and systems thinking.
Answer a question
What standard provides info on software testing concepts?
1.1. What is Testing?
ISO/IEC/IEEE 29119-1
Provide a list
Typical test objectives
9
1.1. What is Testing?
- Evaluating work products such as requirements, user stories, designs, and code
- Triggering failures and finding defects
- Ensuring required coverage of a test object
- Reducing the level of risk of inadequate software quality
- Verifying whether specified requirements have been fulfilled
- Verifying that a test object complies with contractual, legal, and regulatory requirements
- Providing information to stakeholders to allow them to make informed decisions
- Building confidence in the quality of the test object
- Validating whether the test object is complete and works as expected by the stakeholders
Provide a definition
Debugging
1.1. What is Testing?
Is concerned with finding causes of this failure (defects), analyzing these causes, and eliminating them
Provide a list
Typical debugging process
1.1. What is Testing?
- Reproduction of a failure
- Diagnosis (finding the root cause)
- Fixing the cause
Provide a defintion
Confirmation testing
1.1. What is Testing?
Checking whether fixes resolved a problem
Answer a question
What is a reason for performing a regression testing after a confirmation testing?
1.1. What is Testing?
To check whether the fixes are causing failures in other parts of the test object
Explain a difference
Testing and Debugging
1.1. What is Testing?
When static testing identifies a defect, debugging is concerned with removing it
Answer a question
Does a static testing requires debugging?
1.1. What is Testing?
There is no need for reproduction or diagnosis, since static testing directly finds defects, and cannot cause failures
Answer a question
What does testing as a form of quality control helps to acheive?
1.2. Why is Testing Necessary?
Testing, as a form of quality control, helps in achieving the agreed upon goals within the set scope, time, quality, and budget constraints.
Answer a question
How does testing contributes to the quality of test objects?
1.2. Why is Testing Necessary?
The defects detected by testing can then be removed (by
debugging – a non-testing activity), so testing indirectly contributes to higher quality test objects.
Answer a question
How is testing considered cost-effective in quality assurance?
1.2. Why is Testing Necessary?
Testing provides a cost-effective means of detecting defects.
Answer a question
What role does testing play in the broader context of project management and decision-making throughout the Software Development Life Cycle?
1.2. Why is Testing Necessary?
Testing provides a means of directly evaluating the quality of a test object at various stages in the SDLC.
These measures are used as part of a larger project management activity, contributing to decisions to
move to the next stage of the SDLC, such as the release decision
Answer a question
What is the significance of testing in representing user needs during the development lifecycle?
1.2. Why is Testing Necessary?
Testing provides users with indirect representation on the development project. Testers ensure that their
understanding of users’ needs are considered throughout the development lifecycle. The alternative is to
involve a representative set of users as part of the development project, which is not usually possible due
to the high costs and lack of availability of suitable users.
Answer a question
Why might testing be essential in software development beyond the primary goal of ensuring product quality?
1.2. Why is Testing Necessary?
Testing may also be required to meet contractual or legal requirements, or to comply with regulatory
standards.
Answer a question
Does “testing” and “quality assurance” mean the same?
1.2. Why is Testing Necessary?
Testing and QA
are not the same.
Fiish the sentence
Testing is a form of…
1.2. Why is Testing Necessary?
Quality Control
Provide a definition
Quality Control
1.2. Why is Testing Necessary?
QC is a product-oriented, corrective approach that focuses on those activities supporting the achievement
of appropriate levels of quality.
Provide a list
Major forms of Quality Control
1.2. Why is Testing Necessary?
- testing
- formal methods ((model checking and proof of correctness)
- simulation
- prototyping
Answer a question
Describe testing as a form of a Quality Control
1.2. Why is Testing Necessary?
QA is a process-oriented, preventive approach that focuses on the implementation and improvement of
processes. QA applies to both the development and testing processes, and is the responsibility of everyone
on a project.
Answer a question
What is a working assumption of testing as a form of QC?
1.2. Why is Testing Necessary?
It works on the basis that if a good process is followed correctly, then it will generate a good
product.
Answer a question
How test results are used?
1.2. Why is Testing Necessary?
Test results are used by QA and QC. In QC they are used to fix defects, while in QA they provide
feedback on how well the development and test processes are performing.
Provide a definition
Error
1.2. Why is Testing Necessary?
In the context of software testing and development, an “error” refers to a mistake made by a human, typically the programmer or developer, in the source code or the design of the software. It is an incorrect or unintended action or decision during the programming phase that leads to a fault or bug in the software.
Provide a definition
Error
1.2. Why is Testing Necessary?
In the context of software testing and development, an “error” refers to a mistake made by a human, typically the programmer or developer, in the source code or the design of the software.
Answer a question
For what reasons do humans make errors?
1.2. Why is Testing Necessary?
Humans make errors for various reasons, such as time pressure, complexity of work products,
processes, infrastructure or interactions, or simply because they are tired or lack adequate training.
Provide a definition
Defect
1.2. Why is Testing Necessary?
Defect or bug is the manifestation of that error in the software development artifacts, such as documentation, such as a requirements specification or a test script, in source
code, or in a supporting artifact such as a build file.
Provide a definition
Failure
1.2. Why is Testing Necessary?
If a defect in code is executed, the
system may fail to do what it should do, or do something it shouldn’t, causing a failure. Some defects will
always result in a failure if executed, while others will only result in a failure in specific circumstances, and
some may never result in a failure.
Provide an answer
What are some other causes for the failures
1.2. Why is Testing Necessary?
Failures can also be caused by environmental
conditions, such as when radiation or electromagnetic field cause defects in firmware
Provide a definition
Root cause
1.2. Why is Testing Necessary?
A root cause is a fundamental reason for the occurrence of a problem (e.g., a situation that leads to an
error).
Provide a definition
Root cause analysis
1.2. Why is Testing Necessary?
Root causes are identified through root cause analysis, which is typically performed when a failure
occurs or a defect is identified. It is believed that further similar failures or defects can be prevented or
their frequency reduced by addressing the root cause, such as by removing it.
Provide an answer
How many testing principles is there
1.3. Testing principles
7
Provide a definition
First testing principle
1.3. Testing principles
Testing shows the presence, not the absence of defects.
Provide a definition
Second testing principle
1.3. Testing principles
Exhaustive testing is impossible
Provide a definition
Thrid testing principle
1.3. Testing principles
Early testing saves time and money
Provide a definition
Fourth testing principle
1.3. Testing principles
Defects cluster together
Provide a definition
Fifth testing principle
1.3. Testing principles
Tests wear out
Provide a definition
Sixth testing principle
1.3. Testing principles
Testing is context dependent
Provide a definition
Seventh testing principle
1.3. Testing principles
Absence-of-defects fallacy. It is a fallacy (i.e., a misconception) to expect that software verification
will ensure the success of a system.
Provide a deifintion
Test Process
1.4.Test Activities, Testware and Test Roles
The test process, though context-dependent, typically includes a set of standard activities essential for achieving test objectives. This structured approach is crucial for effective testing across various scenarios.
Provide a deifintion
Test Process
1.4.Test Activities, Testware and Test Roles
The test process, though context-dependent, typically includes a set of standard activities essential for achieving test objectives. This structured approach is crucial for effective testing across various scenarios.
Provide a list
Test Activities and Tasks included in Test Process
1.4.Test Activities, Testware and Test Roles
- Test planning
- Test monitoring and control
- Test analysis
- Test design
- Test implementation
- Test execution
- Test completion
Provide a Definition
Test planning
1.4.Test Activities, Testware and Test Roles
Test planning consists of defining the test objectives and then selecting an approach that best achieves
the objectives within the constraints imposed by the overall context.
Provide a Definition
Test monitoring and control
1.4.Test Activities, Testware and Test Roles
Test monitoring involves the ongoing checking of all test activities and the
comparison of actual progress against the plan. Test control involves taking the actions necessary to
meet the objectives of testing.
Provide a Definition
Test analysis
1.4.Test Activities, Testware and Test Roles
Test analysis includes analyzing the test basis to identify testable features and to define and prioritize
associated test conditions, together with the related risks and risk levels. The test basis
and the test objects are also evaluated to identify defects they may contain and to assess their testability.
Test analysis is often supported by the use of test techniques. Test analysis answers the
question “what to test?” in terms of measurable coverage criteria.
Provide a Definition
Test design
1.4.Test Activities, Testware and Test Roles
Test design includes elaborating the test conditions into test cases and other testware (e.g., test
charters). This activity often involves the identification of coverage items, which serve as a guide to
specify test case inputs. Test techniques (see chapter 4) can be used to support this activity. Test design also includes defining the test data requirements, designing the test environment and identifying any
other required infrastructure and tools. Test design answers the question “how to test?”.
Provide a Definition
Test implementation
1.4.Test Activities, Testware and Test Roles
includes creating or acquiring the testware necessary for test execution (e.g., test
data). Test cases can be organized into test procedures and are often assembled into test suites. Manual
and automated test scripts are created. Test procedures are prioritized and arranged within a test
execution schedule for efficient test execution. The test environment is built and
verified to be set up correctly.
Provide a Definition
Test execution
1.4.Test Activities, Testware and Test Roles
Test execution includes running the tests in accordance with the test execution schedule (test runs).
Test execution may be manual or automated. Test execution can take many forms, including continuous
testing or pair testing sessions. Actual test results are compared with the expected results. The test
results are logged. Anomalies are analyzed to identify their likely causes. This analysis allows us to report
the anomalies based on the failures observed
Provide a Definition
Test completion
1.4.Test Activities, Testware and Test Roles
Test completion activities usually occur at project milestones (e.g., release, end of iteration, test level
completion) for any unresolved defects, change requests or product backlog items created. Any testware
that may be useful in the future is identified and archived or handed over to the appropriate teams. The
test environment is shut down to an agreed state. The test activities are analyzed to identify lessons
learned and improvements for future iterations, releases, or projects. A test completion
report is created and communicated to the stakeholders.
Answer a question
Who funds testing?
1.4.Test Activities, Testware and Test Roles
Testing is funded by stakeholders
Answer a question
What is the final goal of testing?
1.4.Test Activities, Testware and Test Roles
its final goal is to help fulfill
the stakeholders’ business needs.
Provide a list
Contextual factors upon which testing depends
1.4.Test Activities, Testware and Test Roles
- Stakeholders
- Team members
- Business domain
- Technical factors
- Project constraints
- Organizational factors
- Software development lifecycle
- Tools
Describe a testing contextual factor
Stakeholders
1.4.Test Activities, Testware and Test Roles
needs, expectations, requirements, willingness to cooperate, etc
Describe a testing contextual factor
Team members
1.4.Test Activities, Testware and Test Roles
skills, knowledge, level of experience, availability, training needs, etc.
Describe a testing contextual factor
Business domain
1.4.Test Activities, Testware and Test Roles
criticality of the test object, identified risks, market needs, specific legal
regulations, etc
Describe a testing contextual factor
Technical factors
1.4.Test Activities, Testware and Test Roles
type of software, product architecture, technology used, etc
Describe a testing contextual factor
Project constraints
1.4.Test Activities, Testware and Test Roles
scope, time, budget, resources, etc.
Describe a testing contextual factor
Organizational factors
1.4.Test Activities, Testware and Test Roles
organizational structure, existing policies, practices used, etc
Describe a testing contextual factor
Software development lifecycle
1.4.Test Activities, Testware and Test Roles
engineering practices, development methods, etc
Describe a testing contextual factor
Tools
1.4.Test Activities, Testware and Test Roles
availability, usability, compliance, etc
Provide a list
Test-related issues influenced by the contextual factors
1.4.Test Activities, Testware and Test Roles
test strategy, test techniques
used, degree of test automation, required level of coverage, level of detail of test documentation,
reporting
Provide a definition
Testware
1.4.Test Activities, Testware and Test Roles
Testware is created as output work products from the test activities and tasks. Its a work product.
Answer a question
What is a task of configuration management
1.4.Test Activities, Testware and Test Roles
Proper configuration management (see section 5.4) ensures consistency and integrity of work products
Provide a list
Testing work products
1.4.Test Activities, Testware and Test Roles
- Test planning work products
- Test monitoring and control work products
- Test analysis work products
- Test design work products
- Test implementation work products
- Test execution work products
- Test completion work products
Describe a testing work product
Test planning work products
1.4.Test Activities, Testware and Test Roles
include: test plan, test schedule, risk register, and entry and exit
criteria. Risk register is a list of risks together with risk likelihood, risk impact and
information about risk mitigation. Test schedule, risk register and entry and exit
criteria are often a part of the test plan.
Describe a testing work product
Test monitoring and control work products
1.4.Test Activities, Testware and Test Roles
test progress reports, documentation of control directives and risk information
Describe a testing work product
Test analysis work products
1.4.Test Activities, Testware and Test Roles
include: (prioritized) test conditions (e.g., acceptance criteria), and defect reports regarding defects in the test basis (if not fixed directly).
Describe a testing work product
Test design work products
1.4.Test Activities, Testware and Test Roles
include: (prioritized) test cases, test charters, coverage items, test
data requirements and test environment requirements.
Describe a testing work product
Test implementation work products
1.4.Test Activities, Testware and Test Roles
include: test procedures, automated test scripts, test
suites, test data, test execution schedule, and test environment elements. Examples of test
environment elements include: stubs, drivers, simulators, and service virtualizations.
Describe a testing work product
Test execution work products
1.4.Test Activities, Testware and Test Roles
include: test logs, and defect reports
Describe a testing work product
Test completion work products
1.4.Test Activities, Testware and Test Roles
include: test completion report (see section 5.3.2), action items
for improvement of subsequent projects or iterations, documented lessons learned, and change
requests (e.g., as product backlog items).
Answer a question
What is a purpose for maintaining Traceability between the Test Basis and Testware
1.4.Test Activities, Testware and Test Roles
In order to implement effective test monitoring and control, it is important to establish and maintain
traceability throughout the test process between the test basis elements, testware associated with these
elements (e.g., test conditions, risks, test cases), test results, and detected defects.
Answer a question
What is a relationship between traceability and coverage evaluation?
1.4.Test Activities, Testware and Test Roles
Accurate traceability supports coverage evaluation, so it is very useful if measurable coverage criteria are
defined in the test basis.
Answer a question
How coverage evaluation can be used for test monitoring and control?
1.4.Test Activities, Testware and Test Roles
The coverage criteria can function as key performance indicators to drive the
activities that show to what extent the test objectives have been achieved
Provide an example on
How coverage evaluation can be used for test monitoring and control.
1.4.Test Activities, Testware and Test Roles
- Traceability of test cases to requirements can verify that the requirements are covered by test
cases. - Traceability of test results to risks can be used to evaluate the level of residual risk in a test
object.
Provide a list
Benefits of having good traceability
1.4.Test Activities, Testware and Test Roles
- In addition to evaluating coverage, good traceability makes it possible to determine the impact of
changes, facilitates test audits, and helps meet IT governance criteria. - Good traceability also makes test
progress and completion reports more easily understandable by including the status of test basis
elements. - This can also assist in communicating the technical aspects of testing to stakeholders in an
understandable manner
Provide a list
What is the role of traceability in evaluating the overall success and effectiveness of a project?
1.4.Test Activities, Testware and Test Roles
Traceability provides information to assess product quality, process capability,
and project progress against business goals.
Answer a question
Describe a test management role
1.4.Test Activities, Testware and Test Roles
The test management role takes overall responsibility for the test process, test team and leadership of the
test activities. The test management role is mainly focused on the activities of test planning, test
monitoring and control and test completion. The way in which the test management role is carried out
varies depending on the context. For example, in Agile software development, some of the test
management tasks may be handled by the Agile team. Tasks that span multiple teams or the entire
organization may be performed by test managers outside of the development team.
Answer a question
Describe a testing role
1.4.Test Activities, Testware and Test Roles
The testing role takes overall responsibility for the engineering (technical) aspect of testing. The testing
role is mainly focused on the activities of test analysis, test design, test implementation and test
execution
Answer a question
Can the same individuals assume different roles at various stages of a project or process?
1.4.Test Activities, Testware and Test Roles
Different people may take on these roles at different times. For example, the test management role can
be performed by a team leader, by a test manager, by a development manager, etc. It is also possible for
one person to take on the roles of testing and test management at the same time.
