Software Testing Flashcards
Why do we test software?
Validation Testing
Defect Testing
Validation Testing?
Demonstrate that software meets requirements
Test the main success scenario of each use case
Test the main use-case extensions
Defect Testing?
Find defects by finding inputs where the software behaviour is incorrect
Test use-case extensions that define error handling
Test how the system operates
Test the boundaries
What does testing identify?
Deficiencies in requirements, design and implementation
Why is Validation needed?
Software needs to be demonstrated to be suitable
When is Validation performed?
Throughout the entire development process, not an end-of-cycle activity
Why is Verification Testing needed?
Have the software for completeness, does it deliver on all its functional and non-functional requirements
inputs causing?
anomalous behaviour
outputs which reveal?
the presence of defects
Methods of verifying and validating software?
Inspection and Reviews
Testing
Inspection and Reviews?
Software process artifacts are being statically analyzed for the purpose of being verified and validated
Types of Testing?
Development Testing
Release Testing
User Testing
What is Development Testing?
Conducted by software developers as part of the development process
What is Release Testing?
Conducted by independent testing teams that verifies and validates that the integrate software is correct
What is User Testing?
Conducted by users or pseudo-users to validate that software is suitable
What is Software Inspection?
In-depth, errors do not cover
Does not need running code, not complete code
Economical because it does not require any additional test harness
Very effective, 60%-90% of errors
Debugging Code?
Development activity aimed at solving bugs/issues discovered through software testing
Integral part of software development
Debugging is a critical validation and inspection activity
Limitations of Software Inspection?
A manual process which prevents regression verification
Hard to inspect complex runtime conditions
Expensive process
Complex Runtime Conditions?
Unexpected interactions between components
Timing problems
Performance issues
Factors that influence the breadth and depth of validation and testing activities include?
Software Purpose
User Expectations
Market Environment
Unit Testing?
Development testing that verifies the functionality of objects and their methods
Types of Development Testing?
Unit Testing
Component (Interface) Testing
System Testing
In agile processes, testing is seen as the?
Driving force of the development process through Test Driven Development
What does Unit Testing do?
Verifies the functionality implemented in objects and their methods by testing all possible states, inputs and outputs
Units are tested in isolation
Automated using unit test framework
Unit Testing Frameworks?
JUnit
XCTest
Nunit, MSTest, VSTest, xUnit
Mocha, Unit.js
Anatomy of a Unit Test?
Implemented as a test class with test methods Test classes represent a cohesive concern of testing one unit
Test Methods are organized as follows:
Setup (Arrange) Part
Call (Act) Part
Assertion Part
Setup (Arrange) Part?
Initializes the variables to be used in the test and the list of inputs that will be used
Uses mocks and dependency injection to isolate the component to test
Call (Act) Part?
Invoke the method being tested the input and environment that has been setup
Verification (Assert) Part?
Use asserts to compare the result obtained with the expected results and determine the outcome of the test
Writing Effective Unit Tests?
Determining inputs to be used in unit tests
Partition testing
Guideline-based Testing
Know how much of the code is tested
Determining Inputs to be used in unit tests?
Choose inputs to reflect normal operation
Choose abnormal inputs to reflect error conditions
Partition Testing?
Identify groups of inputs with similar behaviour
Test input in the middle of the partition
Test inputs as the boundaries of the partition
Unit Testing Guidelines?
Single element or no element
Different sizes of collections
First, middle, and last element
Choose inputs that force all error messages
Component Testing?
Several units are integrated to create a component whose functionality is exposed via interfaces
Interface errors
Types of interfaces in Component Testing?
- Methods / Procedure based interface
- Messaged passing interfaces
- Shared memory interfaces
Interface Errors?
Interface Misuse
Timing Errors
Interface Misuse?
Wrong parameters, calling methods in the wrong order, incorrect state
Timing Errors?
Prevalent with shared memory and message passing interfaces
Component Testing Guidelines?
Test the boundaries of interaction with external components
Test arguments just like in a unit test
Testing calling methods in different order and in different states
Stress testing for speed, memory and number of messages
System Testing?
Components are integrated with each other and with external systems to form the system
Some errors only occur when the entire system is integrated and this is what system testing aims to verify it
Use-case based testing is used to verify the main success scenario
System Testing Guidelines?
Exhaustive testing not possible/practical
All system functions accessible through menus and UI elements must be tested
Combinations of functions must be determined and tested
Never wait until the end to system test a product
Software Testing Process? (Graph)
Design test cases
Prepare test data
Run program with test data
Compare results to test cases
Test-Driven Development?
The most enduring and generalizable practice in (Extreme Programming)
Can be practiced in both Agile and Plan-Based processes
Emphasizes the testability of he system
TDD Benefits?
Code coverage Regression testing Effective and constant refactoring Effective debugging Provides elements of requirements analysis and design that leverage only coding
TDD Limitations?
Not an alternative to requirements analysis
Not an alternative to software architecture and software design and modeling
Hard to use for component and system testing for large enough systems
Hard to use when system uses legacy code or large components
Release Testing?
Black-box testing using tests derived from the software requirements
Release testing is done by QA team. Must be objective and independent.
Validation testing to demonstrate system meets requirements.
Goal of Release Testing?
Ensure the system is ready to be delivered to customers and or users
- Functionality
- Performance
- Dependability
- Robustness
How to Conduct Release Testing?
Requirements-Based Testing
Scenario-Based Testing
Requirement-Based Testing?
Requirements should be testable
Consider each requirements and derive a set of tests for it
Demonstrates the system has properly implemented its requirements
Traceability records demonstrates that all requirements have been tested
Scenario-Based Testing?
Uses real-world usage scenarios when neither use-cases nor stories were used in requirements definition
Release testers run through the scenario using the system to validate the system responds accordingly
Performance Testing?
Validate that the system can process the intended load
Increase the load on the system until the system performance is unacceptable
Stress Testing
Stress Testing?
Test how the system behaves when it is subjected to demands outside of the design limits
Particularly useful to distributed systems
Important component of security testing
User Testing?
Performed by users or customers of the system
Performed in the user environment
Impossible to replicate the complexity of all user environments
Types of User Testing?
Alpha Testing
Beta Testing
Acceptance Testing
Alpha Testing?
Select set of users work with developers to test early releases of the software
Beta Testing?
Larger group of users work with the software aiming to discover defects
Acceptance Testing Process>
Define acceptance criteria Plan acceptance testing Derive acceptance tests Run acceptance tests Negotiate test results Accept or reject system
