System Testing Flashcards
Trigger Conditions
Inputs that will give rise to failures indicating the existence of faults that can be isolated and eliminated
When does Unit testing begin
Happens in parallel with construction. Each developer tests the units that he or she is implementing
Assemblages
Large sub-systems of combined unit tests that are tested together.
Alpha Testing
1st stage of system testing, developers test the entire product
Beta Testing
2nd state of system testing, involves actual users testing the product in either a simulated env. or the actual env.
Subsystem testing
unit and integration testing
System testing
Alpha and beta testing
Differences b/w system and subsystem testing
system involves an increment of the entire product
subsystem testing does not
System involves units or sub-systems that were constructed by several people
Units tests typically created and executed by the developer who constructed the unit
Who preforms sub-system tests?
People who have intimate knowledge of the engineering design and implementation of the product, include CLEAR and BLACK-box techniques
Who preforms system testing?
People without knowledge of product design. BLACK BOX
How are sub-system tests created?
From detailed requirements specs or detailed user storiess
How are system tests created?
From descriptions of interactions b/w users and the system (use cases or user stories about features). State of system is likely to change during a test
What do Unit testing and integration testing depend on?
The nature of the product: customize or build the product Differences matter these differences don’t matter in system tests
Finger pointing is more likely to occur in sub-system or system testing?
System testing because entire team vs. individual
Personnel Alpha Testing
Testers
Enviornment Alpha Testing
Controlled - to isolate and record failures
Purpose Alpha Testing
Validation (indirect) and Verification
Recording Alpha Testing
Extensive Logging
Personnel Beta Testing Acceptance and Installation
Users
Environment Beta Testing Acceptance
Controlled
Purpose Beta Testing Acceptance
Validation (Direct)
Recording Beta Testing Acceptance
Limited Logging
Environment Beta Testing Installation
Uncontrolled
Purpose Beta Testing Installation
Verification
Recording Beta Testing Installation
Limited Logging
Problems with Alpha Testing
Conflict of interest - not always want to identify failures want to prove it is fault free
Don’t want testing and development team to become adversairies
Functional alpha testing
attempts to validate and verify the features provided by the software product. Validation when the tests are created and verification when the tests are conducted
How is Functional alpha testing conducted (bottom-up or top-down)?
Bottom-up, most basic functionality tested first to make it easier to isolate failures
Operational Profiles
Contain information about the relative frequency of different use cases and the order in which use cases typically occur. Tests are then reflect these patterns of usage
Non-Functional Alpha Testing
Execution requirements (response time and memory usage)
Types of Non-Functional Alpha Testing (6)
Timing tests reliability tests availability test Stress Tests Recover Tests Configuration Tests
Timing Tests
Evaluate non-functional requirements related to the time required to perform a function
Benchmarks
Standardized timing tests
Reliability Tests
determine the probability that the product will fail in a given interval of time
Availability tests
Determine the probability that a product will be usable at a particular time
Stress Tests/Torture tests
Determine robustness (ability to operate under a wide range of conditions) and the safety (ability to minimize the damage resulting from a failure) of the system
Stress tests that exclusively focus on safety are called?
Recovery tests
Configuration tests
used to assess the product of different hardware or software platforms and in different operating enviornments
Usability Tests
Tests checking requirements about the user interface
Internationalization/localization tests
Usability tests that involve both translations and locale-specific information (currencies, number formats, and time and date info)
Accessibility testing
ensures that the user interface is appropriate for all people regardless of physical ability.
Beta Acceptance Testing
Done by clients to validate the product (satisfies clients’ needs and desires).
Order and frequency in which features are used is not controlled
But all features are tested
Tests functional and non-functional execution
Beta Installation Testing
Used to verify product, to determine that it operates properly in the user’s environment. Not expert testers so failures they report may be hard to understand (minimize dependence on installation testing)
Traditional Process
Some requirements will be mapped to unit test and others will be mapped to system tests (few mapped to integration tests)
Most important testing in traditional process?
Unit testing (occur earlier hence more important) Integration doesn’t begin until fairly late in process
Stubs
Placeholder code that stands in for units that have not yet been constructed (for traditional interface)
When does alpha testing occur in traditional process
Near the end, because don’t need working version until end
Test Plan
Document that plans the system testing in traditional process
How are tests made for traditional processes
From requirements specs
When are alpha and integration testing for Scrum?
Every sprint to show items in project backlog are complete
Build Verification Tests/smoke tests
Product is built and tested each day using automated tests, ensure that product remains at a given quality throughout its incremental development.
Need to smoke test code before each commit
Release candidates (RC)
Product versions containing all features planned for a release (beta testing for Beta testing but beta can still be done with each sprint depends on team)
Which process makes it harder to preform regression testing?
Scrum, because hard to isolate regression testing from the rest of testing process because without a formal requirements spec it is difficult to distinguish old features from new features (hard to have baseline for tests)
How to record GUI tests
record interactions
script that records actions (often easier to understand than recordings and easier to rerun than recordings)
Bug tracking systems
Tools that exist to help testers record, track and report test results
Engineering Design for System Testing
Design patterns that make it easier to conduct system tests. Command pattern encapsulates functionality in individual objects. (test independently of the events that trigger that functionality
Construction for System Testing
Make decisions during the construction phase that facilitate system testing. Some languages and libraries have built-in support for logging. Try catch make it easier to build a flexible test harness
