Intro

Question 1

BS 7925-1

Answer 1

Testing involves executing software with the intent to identify errors and ensure the application meets specified requirements.

Question 2

IEEE Definition

Answer 2

Testing evaluates systems or components, manually or automatically, to verify that they satisfy requirements or identify discrepancies between expected and actual outcomes.

Question 3

ISEB Syllabus

Answer 3

Testing measures software quality by identifying defects, encompassing both functional (what the software does) and non-functional (how it performs) aspects.

Question 4

Entities in the Testing Problem

Answer 4

• P (Implementation): Development of the system
• S (Specification): Requirements or goals defining the system’s correct behavior.
• O (Observation): Outputs and effects of the system under testing.
• T (Testing): Process of verifying that P aligns with S by examining O.

Question 5

Testing Axioms

Answer 5

1. Bugs cannot be fully eliminated
2. Exhaustive Testing is impossible
3. Testing is risk-based and context-dependent
4. Testing starts in SDLC
5. Finding Bugs often uncovers more

Question 6

Testing is risk-based and context-dependent

Answer 6

Strategies vary between critical systems like medical software and non-critical systems.

Question 7

Testing starts in SDLC

Answer 7

Catch bugs early to minimize cost and effort to fix them later.

Question 8

Testing Principles

Answer 8

• Traceability: Every test should correspond to a requirement
• Risk Prioritization: Test high impact areas to maximize impact.
• Pareto Principle: 80% of issues stem from 20% of the code.
• Diversity of Techniques: A single method can’t uncover all types of bugs, use a mix of testing strategies (ex. White box, Black box)
• Bug Fix Timing: Fixing defects early reduces cost and complexity
• Pesticide Paradox: Regularly change and enhance tests to ensure continued effectiveness in finding bugs.

Question 9

Additional Principles

Answer 9

1. Sensitivity: A test is more effective if it fails consistently when an issue exists
2. Intentions: Clearly define what each test aims to achieve to prevent ambiguity
3. Partition: Break down large problems into smaller, manageable sections
4. Feedback: Improve the testing process continuously based on past and present outcomes.

Question 10

Error

Answer 10

Human mistake that leads to an incorrect implementation

Question 11

Fault/Defect/Bug

Answer 11

Specific problem in the program (ex. Incorrect logic data)

Question 12

Failure

Answer 12

Visible manifestation of a fault during program execution

Question 13

Levels of Testing

Answer 13

1. Unit testing - Verifies individual components or modules in isolation
2. Integration testing - Ensures different components work together correctly
3. System testing - Evaluates the entire application as a whole against requirements

Question 14

The Testing Process

Answer 14

1. Test Planning
2. Test Design
3. Environment Setup
4. Execution
5. Problem Reporting
6. Exit Criteria

Question 15

Test Planning

Answer 15

Establish the entry/exit criteria, Test strategies, tools, schedules and resources and introduce problem tracking and reporting mechanisms.

Question 16

Test Design

Answer 16

Review system requirements, architecture, and testability. Define test conditions, required test data and specific test cases with preconditions and steps

Question 17

Environment Setup

Answer 17

Configure hardware, software, network and database environments needed for testing

Question 18

Execution

Answer 18

Run test cases, record outcomes (Pass/Fail/Not executed) and perform regression testing to make sure fixes don’t break existing functionality.

Question 19

Problem Reporting

Answer 19

Document with description, steps to reproduce, severity and priority, and potential new test ideas.

Question 20

Exit Criteria

Answer 20

Define conditions for ending testing; Coverage threshold met, no critical faults remaining, Testing within time and budget constraints.

Question 21

Test Case

Answer 21

Input, execution steps, and expected outcome

Question 22

Test Specification

Answer 22

Requirements satisfied by one or more test cases

Question 23

Test Suite

Answer 23

Collection of test cases

Question 24

Adequacy Criterion

Answer 24

Measure of how effectively a test suite meets testing requirements.

Question 25

Types of Coverage Analysis

Answer 25

• Statement (Every line of code executes)
• Branch (Every decision point’s branches are tested)
• Condition (Tests logical conditions for all T/F combinations)
• Path (Tests all potential execution paths)
• Data (Asses how input data affects execution)

Question 26

balance between effort and effectiveness

Answer 26

You may spend 40 hours testing or 80 hours, but if they produce 10 and 12 bugs respectively then it’s better to spend 40 hours.