Chapter 1 - Fundamentals of Testing

Question 1

What is testing?

Answer 1

It is a process withinn the software development lifecycle that evaluates the quality of components or a system and their related work products (test objects). It assesses software quality and helps reduce risk of failure.

Testing is NOT just executing tests!

Question 2

What is a Test Object?

Answer 2

It is the specific work product that is to be tested.

Question 3

What can be a Test Object?

Answer 3

It can be the product (code - component testing or the system - System Testing)
or
it can be intermediate work products such as requirements specifications, architectural specifications, test cases, etc.

Before any Test it has to be clear what the Test Object is.

Question 4

What is Static Testing?

Answer 4

Happens before we execute or test any code. It includes reviews and static analysis.

Question 5

What is Dynamic Testing?

Answer 5

Executing code to analyze its behaviour.

It uses different types of tests techniques and test approaches to derive test cases.

Question 6

What is Validation?

Answer 6

Confirmation through examination that the work product meets the needs of users and other stakeholders

Checks for functionality

“Was the correct product developed?”

Question 7

What is Verification?

Answer 7

Checking if part of the code or the system align with the specified requirements.

“Was the product developed correctly”

Question 8

What is a Test Objective?

Answer 8

It is the purpose of the test.

Can vary depending on the current requirements, which testing phase we are in, and other factors.

It is important to clarify the objective before initiating other testing activities.

Example:
Is the focus set on speed of delivery? Are we focusing finding defects? Are we focusing on Validation? Are we focusing on Compliance? etc.

Question 9

What do we understand as Quality in the realm of testing?

Answer 9

It is the degree to which a work product satisfied the specified and implied needs of the stakeholders

Question 10

What is Coverage and what is it used for?

Answer 10

To have measurable objectives Coverage is used.

It shows the degree to which specified coverage items have been exercised in a specific test suite.

In software testing, coverage refers to the extent to which the software’s functionality, code, or requirements are tested. It measures how much of the application is exercised during testing, helping ensure that critical parts of the system are verified.

The result is measured in percentage.

Question 11

What is a Coverage Item?

Answer 11

A coverage item is a specific work element or aspect of the software that is being measured for coverage. Depending on the type of coverage, a coverage item can be code, code functions, code branches, requirements, execution paths, test cases

Examples:
Code Coverage
Definition:
Measures how much of the source code is executed when tests run.

Suppose a function contains 10 lines of code, but tests only execute 8 of them.

Coverage = (8/10) × 100% = 80% statement coverage.

or

Requirement Coverage
Definition:
Ensures that all documented requirements have test cases covering them.

Coverage Items:
Software requirements or user stories.

A banking application has 10 requirements, but test cases exist for only 8.

Coverage = (8/10) × 100% = 80% requirement coverage.

Question 12

What is a coverate item according to ISTQB definition?

Answer 12

an attribute or combination of attributes derived from one or more test conditions by using a test technique

when talking about coverage items they need to be specified

Question 13

Is Debugging testing?

Answer 13

No!
Testing can only identify failures. Debugging is then performed by programmers. Then the fixes are tested again.

Question 14

What is Regression testing?

Answer 14

A type of change-related testing to detect whether defects have been introduced or uncovered in unchanged areas of the software.

This is done after defects in other areas of the product have been fixed. Fixes in some areas can lead to new errors in previously functioning areas.

Question 15

Typical test obejctives are…
Evaluating…
Causing…
Ensuring…
Reducing…
Verivying…
Verifying…
Providing…
Building…
Validating…

Answer 15

• Evaluating work products (requirements, user stories, designs, code, etc)
• Causing failures, finding defects
• Ensuring required coverage of test object
• Reducing risk level of inadequate software quality
• Verifying if specified requirements have been fulfilled
• Verifying compliance of test objects with laws, contracts, regulations
• Providing information to stakeholders to make informed decisions
• Building confidence in quality of test object
• Validating completness of test object and if it works as needed

Question 16

What is Confirmation Testing?

Answer 16

Checks if the fixes (debugging) have resolved the problem. Preferably done by the same tester who performed the initial test.

Question 17

Failure VS Defect

Answer 17

Failures are triggered during testing and are caused by defects in the test object.

The tester triggers failures, the programmer finds the defect and fixes it.

Question 18

Why is testing necessary?

Answer 18

Helps achieve agreed upon test objectives within a set scope, time, quality and budget.

Helps identify defects in software.

Question 19

How does testing contribute to success?

Answer 19

• provides cost effective means of detecting defects early on
• provides means of evaluating quality of test objects at various stages in the SDLC & informs further decision making
• considers the stakeholder’s needs throughout the SDLC and provides indirect representation for users
• ensures that contract requirements, legal requirements, and regulatory compliance is met

Question 20

What is the difference between Testing (Quality Control) and QA (Quality Assurance)

Answer 20

Testing: product oriented form of quality control, corrective approach, activity focus lies on achieving quality

QA: process oriented, preventitive approach, focus lies on defect prevention, implementing and improving processes that will generate good products

Question 21

Are test results used by QA or testing or both?

Answer 21

Both!
- Testing uses them to fix defects
- QA use them to evaluate how well the development and test process is performing

Question 22

Give example for Error, Defect, Failure

Answer 22

Humans make errors (mistakes) which produce defects (faults, bugs) which may result in failures.

Question 23

Where can defects be found?

Answer 23

documentation, requirements, test scripts, source code, supporting work products

Question 24

What happens if defects are not caught early in the SDLC?

Answer 24

They often lead to defective work products later in the lifecycle.

Question 25

Do defects always result in failures?

Answer 25

No! - some may result in failures - some may result in failures only in specific circumstances - some may never result in a failire

Question 26

Can errors and defects be caused by external factors?

Answer 26

Yes! They can be caused by environmental conditions such as radiation or electromagnetic fields.

Question 27

What is a root cause and how are they treated/found?

Answer 27

The fundamental reason for the occurance of a problem. Found via root cause analysis & typically performed when failure occurs or defect is found. Addressing the root cause can prevent further similar defects or failures.

Question 28

Define Defect

Answer 28

An imperfection or deficiency in a work product where it does not meet its requirements or specifications.

Question 29

Define Failure

Answer 29

An event in which a component or a system does not performed a required function within specified limits

Question 30

Define Error

Answer 30

human action that produces an incorrect result an error in one work product that leads to a defect can lead to an error which leads to a defect in a related work product. Example: an error in the requirements can lead to an error in the programming

Question 31

What are false positives and false negatives and what produces them?

Answer 31

They happen through errors in testing: False positive: unexpected behaviour is falsly reported as a defect. Found a defect that is none. False negative: existing defects are not found. False negatives are much worse than false positives.

Question 32

Define Root Cause

Answer 32

A source of a defect that if removed will decrease or remove the occurance of said defect type

Question 33

What are testing principles?

Answer 33

General guidelines applicable to all testing.

Question 34

What are the 7 testing principles?

Answer 34

1. Shows the presence not the absences of defects 2. exhaustive testing is impossible 3. Early testing saves time and money 4. Defects cluster togeter 5. Tests wear out 6. Testing is context dependent 7. Absence-of-defects fallacy

Question 35

Explain the following testing principle: 1. Testing shows the presence not the absence of defects

Answer 35

Testing can never prove that there are no defects left. However, testing reduces the probability of defects remaining undiscovered in test objects.

Question 36

Explain the following testing principle: 2. Exhaustive testing is impossible

Answer 36

Testing everything is not feasible. Test efforts are focused through test techniques, test case prioritization and risk-based testing.

Question 37

Explain the following testing principle: 3. Early testing saves time and money

Answer 37

Defects that are removed early in the SDLC will not cause supsequent defects on derived work products. The cost of finding failure is reduced by finding them early.

Question 38

Explain the following testing principle: 4. Defects cluster together

Answer 38

The Pareto principle. A small number of components usually contain most defects. Predicting and finding such clusters is important input for risk-based testing.

Question 39

Explain the following testing principle: 5. Tests wear out

Answer 39

Apart from regression testing, existing tests and test data need to be modified over time since repeating the same tests many times will become increasingly ineffective at detecting defects.

Question 40

Explain the following testing principle: 6. Testing is context dependent

Answer 40

Testing is always done differently depending on the context.

Question 41

Explain the following testing principle: 7. Abscence-of-defects fallacy

Answer 41

Just because verification has made sure the software works as expected, does not mean the sofware will meet stakeholder needs or be competitive. Validation is also required.

Question 42

Name 7 Test activities

Answer 42

Test planning Test monitoring and controling Test analysis Test design Test implementation Test execution Test completion

Question 43

What are test activities and what are they good for?

Answer 43

a commen set of activities without which testing is less likely to achieve test objectives. sets of activities form a test process which is tailored to the situation at hand. How and when these activities occur is usually decided during test planning.

Question 44

Describe Test Planning

Answer 44

defining the test objectives and selecting best approach to achieve objectives within imposed constraints.

Question 45

Describe Test Montoring and Test Control

Answer 45

Monitoring: checks all test activities and compares progress against plan Control: taking actions necessary to meet test objectives

Question 46

Describe Test Analysis

Answer 46

Answers the question "What to test" in terms of measurable coverage criteria. - analyse test basis to identify testable features - test conditions are defined and prioritized - identify possible defect occurances and assess testability

Question 47

Describe Test Design

Answer 47

Answers the question "How to test" - identification of coverage items - elaborating test conditions into test cases and other testware - defining test data requirements - designing test environment - identifying necessary infrastructure and tools

Question 48

Describe Test Implementation

Answer 48

Creating or acquiring the testware necessary for test execution - test cases are organized into test procedures and often assembled into test suits - test scrips are created - procedures are prioritized and arranged within a schedule for efficiency - test environment ist build and checked

Question 49

Describe Test Execution

Answer 49

Running the tests according to schedule - manual or automated and many other forms of testing - results are compared with expectations - anomalies are analized and causes identified - anomalies are observed and failures reported

Question 50

Describe Test Completion

Answer 50

Test activities are analyzed to identify lessons learned and improvements for future iterations are made - orccurs at project milestones - unresolved defects, change requests and product backlog are noted - useful testware for the future is identified and archived - test environment is shut down - test completion report is created

Question 51

What is "Test basis"

Answer 51

The body of knowledge (source documents or information) used as the basis for test analysis (identifying what needs to be tested) and test design (creating the actual test cases). Can include different types of artifacts (requirements docs, design docs, code, risk analysis, regulations, etc)

Question 52

What are 8 contextual factors that will shape how testing is carried out?

Answer 52

1. Stakeholders (needs, requirements, etc) 2. Team members (skills, knowledge, etc) 3. Business Domain (criticality, risks, market needs, regulations, etc.) 4. Technical Factor (used software, etc) 5. Project constraints (Scope, time, resources etc) 6. Organizational factors (practices, policies, structure) 7. SDLC (dev methods, engineering practices) 8. Tools (availability, usability, etc)

Question 53

What is "Testware"?

Answer 53

In German: Testsoftware, Programme, Anwendungen, Reports created as output work products from test activities. Few Examples: Test Planning - test plan, schedule, entry criteria, exit criteria Test Monitoring - progress reports Analysis - prioritized test conditions Design - test cases, charters, coverage items Implementation - test procedures, scripts, suits, manuals Execution - logs, defect reports Completion - change requests, completion reports

Question 54

What is "traceability" and what are its advantages?

Answer 54

The ability to establish explicit relationships between related work products or items within work products. Traceability means that during the whole test process we maintain traceable connectivity between our test basis elements, our testware, our results and our defects. It supports coverage evaluation if coverage criteria are defined and can act as a performance indicator. Makes it possible to determine impact of changes, facilitates audits and helps meet IT governance criteria. Example: Traceability between test cases to requirements can verify that requirements are covered by test cases.

Question 55

What is the main focus of the test management role?

Answer 55

responsibility for test process, test team, leadership, leading test activities, planning, monitoring

Question 56

What is the main focus of the testing role?

Answer 56

responsibility for engineering the tests, executing them. Focus on test analysis, design, implementation and execution

Question 57

What is the "Whole Team Approach" and what are its benefits?

Answer 57

Any team member with the necessary knowledge and skills can perform any task and everyone is responsible for quality. Improves team dynamic, communication, and collaboration through colocation. Synergistic use of individual skills within team benefits project and close interaction ensures desired quality levels.

Question 58

When is "Whole Team Approach" not desired?

Answer 58

Context dependant. For instance highly critical safety situations might require tester independence

Question 59

What is Independence of Testing?

Answer 59

Degree of independence is measured by how far removed the tester is from the author of the test object. Ex. Developers can test their own product (low level of independence); an external company can test the product (very high level of independence)

Question 60

What are the pros and cons to independence of testing?

Answer 60

Pro: - different approach and mindset towards finding defects. - Finding different types of defects on different levels of independence (dev will find other defects as external company). - Can verify, challenge or disprove assumptions made by stakeholders. Cons: - isolation from dev. team - lack in collaboration and communication - possible adverse relationship (bringer of bad news)

Question 61

What are "Test Conditions"?

Answer 61

A test condition is something that can be tested—a feature, a rule, a requirement, or a situation that might need checking. Need to be well defined and possibly prioritized. Serves as guide to create test cases. Can be any item or event of a component or system that could be verified by test cases

Question 62

What is a "Master Test Plan"?

Answer 62

A test plan that is used to coordinate multiple test levels or test types.

Question 63

What is a "Test Progress Report"?

Answer 63

A type of periodic test report that includes the progress of test activities against a baseline, risks, and alternatives requiring a decision.

Question 64

What does a "Test Case" entail?

Answer 64

A set of preconditions, inputs, actions, expected results and postconditions, developed based on test conditions.

Question 65

What is the relation between test basis and test conditions?

Answer 65

Test basis = Where you get the information about what the system should do. Test condition = What you choose to test, based on that information. Example Cupcake App Test Basis: requirement - the user must be able to order between 1-12 cupcakes Test conditions: the app allows selecting 1-12 Cupcakes. the app does not allow 13 or 0 cupcakes. From this we then create test cases.