Software Testing Flashcards
Why do we test software?
to ensure we’re building the right product and building it right
What are the two general categories of testing?
Validation Testing
Defect Testing
Why is Testing important?
Testing identifies deficiencies in requirements, design & implementation
What is the purposes of Validation Testing?
Demonstrate that software meets requirements
What does Validation Testing test?
- The main success scenario of each use-case
- The main use-case extensions
What is the purpose of defect testing?
Finding defects by finding inputs where the software behaviour is incorrect
What does Defect Testing test?
- Use-case extensions that define error handling
- How the system operates under stress
- The boundaries of functional & non-functional requirements
What did Edsger Dijkstra say about Testing?
Testing can only show the presence of errors, not their absence
True or False: Validation Testing is performed as an end-of-cycle activity
False, Validation Testing is performed throughout the entire development process
What are some of the questions that Validation Testing attempts to answer?
- Are we building the right product?
- Does the product do what the customer expects it to?
- Does the customer validate that the software solves the problem it’s intended to solve?
What are the three purposes of Verification Testing?
- Find defects by finding inputs where the software behaviour is incorrect
- Find defects by modifying & stressing the environment in which the software operates
- Have the software for completeness (does it deliver on all its functional & non-functional requirements?)
How do we verify & validate software?
- Inspection & Reviews
- Testing
What are the properties of Software Inspection?
- In-depth (errors don’t cover/mask other errors)
- Does not need running or complete code
- Economical (no additional Unit Tests)
- Everything can be inspected, not just code
- Can consider qualities (non-functional requirements) that are hard to test otherwise
How effective if Software Inspection?
60-90% of errors can be found via inspection
What are the three types of Testing?
- Development Testing
- Release Testing
- User Testing
What is Development Testing?
Conducted by developers as part of the development process
What is Release Testing?
conducted by independent testing teams that verifies & validates that the integrate software is correct
What is User Testing?
Conducted by users or pseudo-users to validate that software is suitable
What is the main purpose of debugging code?
to validate your assumptions
True or False: Debugging is not integral to software development and is a separate testing activity
False, Debugging is integral to software development and is not a separate testing activity
What are the limitations of Software Inspection?
- Typically a manual process which prevents regression verification
- Hard to inspect complex runtime conditions
- Considered an expensive process
Why is regression verification a limitation of Software Inspection?
Static code analysis can be performed automatically using tools to discover anomalies, compliance with standards, testability, code contracts
Why are runtime conditions a limitation for software inspection?
- Unexpected interactions between components
- Timing problems
- Performance issues
What is Unit Testing?
Development testing that verifies the functionality of objects & their methods
What is Component Testing?
Development testing tht verifies several units (e.g. objects) that are integrated to create a component
What is System Testing?
Development testing that verifies components that are integrated to create a component
What is Release Testing?
Validation Testing of a release to demonstrate system meets requirements & is performed by an independent QA team
What are the types of Development Testing?
- Unit Testing
- Component (Interface) Testing
- System Testing
Development Testing is primarily what?
A defect testing activity with the goal of finding as many bugs as possible
What is the purpose of Unit Testing?
Verifies the functionality implemented in objects & their methods by testing all possible states, inputs & outputs
True or False: Unit Testing is conducted in the system with highly coupled units
False, Units are tested in isolation with a loosely coupled relationship
How is Unit Testing automated?
Using Unit Test Frameworks
List some Unit Testing Frameworks
- JUnit (Java)
- XCTest (Swift)
- Nunit/MSTest/VSTest/xUnit (C#)
- Mocha/Unit.js (NodeJS)
Unit tests are typically implemented as what?
A Test Class with Test Methods
How are Test Methods organized?
- Setup/Arrange
- Call/Act
- Assertion
How can you write effective Unit Tests?
- Determining inputs to be used in Unit Tests
- Partition Testing
- Guideline-based Testing
What criteria can inputs be selected by for unit testing?
- choose inputs to reflect normal operation
- Choose abnormal inputs to reflect error conditions
What is Partition Testing?
- Identify groups of inputs with similar behaviour
- Test input in the middle of the partition
- Test inputs at the boundaries of the partition
What are the basic Unit Testing Guidelines?
- Test sequences with a single element or no element
- Use different sizes of collections for different tasks
- Tests the first, middle & last element of a sequence
- choose inputs that force all error messages
- Load testing to attempt to overflow buffers & test the limits of the application
- Force computation results that are too large or too small
- Repeat the same inputs multiple times
What are the types of Interfaces that are tested in Component Testing?
- Methods/Precedure based interfaces
- Message Passing Interfaces (Async)
- Shared Memory interfaces
What are they two types of Interface Errors in Component Testing?
- Interface Misuse
- Timing Errors
What are the Component Testing Guidelines?
- Test the boundaries of interaction with external components
- Test arguments just like in a unit-test
- Testing calling methods in different order & in different states
- Stress testing for speed, memory, & number of messages
Why is System Testing important?
Some errors only occur when the entire system is integrated & this is what System Testing aims to verify
What is the purpose of Use-Case based testing?
Verifying the main success scenario, extensions & error handling extensions
What are the System Testing Guidelines?
- Exhaustive testing is not possible/practical
- All system functions accessible through menus & UI elements must be tested
- Combinations of functions must be determined & tested
- Never wait until the end to system test a product
What is Test Driven Development?
the mot enduring & generalizable practice in XP (Extreme Programming)
True or False: Test Driven Development is practiced in either Agile or Plan-Based Processes, not both
False, TDD can be practices in both Agile & Plan-Based Processes
What does Test Driven Development emphasize?
The testability of the system
What are the Benefits of Test Driven Development?
- Code Coverage
- Regression Testing
- Support effective & constant refactoring
- Supports effective debugging
- Provides elements of requirements analysis & design that leverage only coding/programming knowledge
What are the Limitations of Test Driven Development?
- Not an alternative to requirements analysis
- Not an alternative to software architecture & software design modelling
- Hard to use for component & system testing for large enough systems
- Hard to use when system uses legacy code or large components
True or False: Release testing is done by the development team
False, Release testing is done by a QA team not the development team
What are the goals of Release Testing?
Ensure the system is ready to be delivered to customers &/or users
What are the two ways to conduct Release Testing?
- Requirements-Based Testing
- Scenario-Based Testing
What are the properties of Requirements-Based Release Testing?
- Requirements should be testable
- Consider each requirement & derive a set of tests for it
- Demonstrates the system has properly implemented its requirements
- Traceability records demonstrates that all requirements have been tested
What are the properties of Scenario-Based Testing?
- Uses real-world usage scenarios when neither use-cases nor stories were used in requirements definition
- Release testers run through the scenario using the system to validate the system responds accordingly
- A scenario encapsulates many requirements so this test integration of related functionality
What is Performance Testing?
Validating that the system can process the intended load (e.g. number of users, requests, size of data)
How is Performance Testing conducted?
By increasing the load on the system until the system performance is unacceptable
What is Stress Testing?
Test how the system behaves when it is subjected to demands outside of the design limits; also a type of Performance Testing
What are the requirements of Performance Testing?
Define the operational profile of the system; the typical load of a system needed to get accurate test of the operational performance of the system
What is User Testing?
Performed by users/customer of the system in the user’s environment
Why is User Testing Important?
Because it’s impossible to replicate the complexity of all user environments
What are the types of User Testing?
Alpha Testing
Beta Testing
Acceptance Testing
What is Alpha Testing?
Select set of users work with developers to test early releases of the software
What is Beta Testing?
Larger group of users work with the software aiming to discover defects (defect testing)
