Lesson 7 - Software Testing (Ch 22)

Question 1

Software testing must be planned carefully to avoid _____ development time and resources.

Answer 1

wasting

Question 2

Testing begins “in the _____” and progresses “to the _____”.

Answer 2

small, large

Question 3

Initially _____ components are tested and debugged.

Answer 3

individual

Question 4

After the individual components have been tested and added to the system, _____ testing takes place.

Answer 4

integration

Question 5

Once the full software product is completed, _____ testing is performed.

Answer 5

system

Question 6

The __________ document should be reviewed like all other software engineering work products.

Answer 6

Test Specification

Question 7

Strategic Approach to Software Testing

• Many software errors are eliminated before testing begins by conducting effective technical _____

Answer 7

reviews

Question 8

Strategic Approach to Software Testing

• Different testing techniques are _____ at different points in time.

Answer 8

appropriate

Question 9

Strategic Approach to Software Testing

• The developer of the software conducts testing and may be assisted by _____ test groups for large projects.

Answer 9

independent

Question 10

Strategic Approach to Software Testing

• Testing and debugging are _____ activities.

Answer 10

different

Question 11

Strategic Approach to Software Testing

• Debugging must be _____ in any testing strategy.

Answer 11

accommodated

Question 12

Verification and Validation

• Make a distinction between verification (are we building the __________?) and validation (are we building the __________?)

Answer 12

product right, right product

Question 13

Verification and Validation

• Software testing is only one _____ of Software Quality Assurance (SQA)

Answer 13

element

Question 14

Verification and Validation

• Quality must be built in to the _____ process, you can’t use testing to add quality after the fact

Answer 14

development

Question 15

Organizing for Software Testing

• The role of the __________ is to remove the conflict of interest inherent when the builder is testing his or her own product.

Answer 15

Independent Test Group (ITG)

Question 16

Organizing for Software Testing

o The developer should do no testing at all
o Software is tossed “over the wall” to people to test it mercilessly
o Testers are not involved with the project until it is time for it to be tested

Answer 16

Misconceptions regarding the use of independent testing teams

Question 17

Organizing for Software Testing

• The developer and ITGC must work together throughout the software project to ensure that thorough _____ will be conducted

Answer 17

tests

Question 18

Software Testing Strategy

• Unit Testing – makes heavy use of testing techniques that exercise specific control paths to detect errors in each software component _______

Answer 18

individually

Question 19

Software Testing Strategy

• Integration Testing – focuses on issues associated with verification and program _____ as components begin interacting with one another

Answer 19

construction

Question 20

Software Testing Strategy

• Validation Testing – provides assurance that the software validation criteria (established during requirements analysis) meets all functional, behavioral, and performance _____

Answer 20

requirements

Question 21

Software Testing Strategy

• System Testing – verifies that all system elements mesh properly and that _____ system function and performance has been achieved

Answer 21

overall

Question 22

• Specify product requirements in a quantifiable manner before testing starts.
• Specify testing objectives explicitly.
• Identify categories of users for the software and develop a profile for each.
• Develop a test plan that emphasizes rapid cycle testing.
• Build robust software that is designed to test itself.
• Use effective formal reviews as a filter prior to testing.
• Conduct formal technical reviews to assess the test strategy and test cases.
• Develop a continuous improvement approach for the testing process.

Answer 22

Strategic Testing Issues

Question 23

• Module interfaces are tested for proper information flow.
• Local data are examined to ensure that integrity is maintained.
• Boundary conditions are tested.
• Basis (independent) path are tested.
• All error handling paths should be tested.
• Drivers and/or stubs need to be developed to test incomplete software.

Answer 23

Unit Testing

Question 24

• Sandwich testing uses top-down tests for upper levels of program structure coupled with bottom-up tests for subordinate levels
• Testers should strive to indentify critical modules having the following requirements
• Overall plan for integration of software and the specific tests are documented in a test specification

Answer 24

Integration Testing

Question 25

Integration Testing Strategies 1. Main control module used as a test driver and stubs are substitutes for components directly subordinate to it. 2. Subordinate stubs are replaced one at a time with real components (following the depth-first or breadth-first approach). 3. Tests are conducted as each component is integrated. 4. On completion of each set of tests and other stub is replaced with a real component. 5. Regression testing may be used to ensure that new errors not introduced.

Answer 25

Top-down integration testing

Question 26

Integration Testing Strategies 1. Low level components are combined into clusters that perform a specific software function. 2. A driver (control program) is written to coordinate test case input and output. 3. The cluster is tested. 4. Drivers are removed and clusters are combined moving upward in the program structure.

Answer 26

Bottom-up integration testing

Question 27

Integration Testing Strategies used to check for defects propagated to other modules by changes made to existing program 1. Representative sample of existing test cases is used to exercise all software functions. 2. Additional test cases focusing software functions likely to be affected by the change. 3. Tests cases that focus on the changed software components.

Answer 27

Regression testing

Question 28

Integration Testing Strategies 1. Software components already translated into code are integrated into a build. 2. A series of tests designed to expose errors that will keep the build from performing its functions are created. 3. The build is integrated with the other builds and the entire product is smoke tested daily (either top-down or bottom integration may be used).

Answer 28

Smoke testing

Question 29

* Interface integrity – internal and external module interfaces are tested as each module or cluster is added to the software * Functional validity – test to uncover functional defects in the software * Information content – test for errors in local or global data structures * Performance – verify specified performance bounds are tested

Answer 29

General Software Test Criteria

Question 30

* Unit Testing – components being tested are classes not modules * Integration Testing – as classes are integrated into the architecture regression tests are run to uncover communication and collaboration errors between objects * Systems Testing – the system as a whole is tested to uncover requirement errors

Answer 30

Object-Oriented Test Strategies

Question 31

* smallest testable unit is the encapsulated class or object * similar to system testing of conventional software * do not test operations in isolation from one another * driven by class operations and state behavior, not algorithmic detail and data flow across module interface

Answer 31

Object-Oriented Unit Testing

Question 32

* focuses on groups of classes that collaborate or communicate in some manner * integration of operations one at a time into classes is often meaningless * thread-based testing – testing all classes required to respond to one system input or event * use-based testing – begins by testing independent classes (classes that use very few server classes) first and the dependent classes that make use of them * cluster testing – groups of collaborating classes are tested for interaction errors * regression testing is important as each thread, cluster, or subsystem is added to the system

Answer 32

Object-Oriented Integration Testing

Question 33

1. WebApp content model is reviewed to uncover errors. 2. Interface model is reviewed to ensure all use-cases are accommodated. 3. Design model for WebApp is reviewed to uncover navigation errors. 4. User interface is tested to uncover presentation errors and/or navigation mechanics problems. 5. Selected functional components are unit tested. 6. Navigation throughout the architecture is tested. 7. WebApp is implemented in a variety of different environmental configurations and the compatibility of WebApp with each is assessed. 8. Security tests are conducted. 9. Performance tests are conducted. 10. WebApp is tested by a controlled and monitored group of end-users (looking for content errors, navigation errors, usability concerns, compatibility issues, reliability, and performance).

Answer 33

WebApp Testing Strategies

Question 34

* User experience testing – ensuring app meets stakeholder usability and accessibility expectations * Device compatibility testing – testing apps on multiple devices * Performance testing – testing non-functional app requirements * Connectivity testing – testing ability of app to connect reliably * Security testing – ensuring app meets stakeholder security expectations * Testing-in-the-wild – testing app on user devices in actual user environments * Certification testing – app meets the distribution standards

Answer 34

MobileApp Testing

Question 35

``` • Focuses on visible user actions and user recognizable outputs from the system • Validation tests are based on the use-case scenarios, the behavior model, and the event flow diagram created in the analysis model o Must ensure that each function or performance characteristic conforms to its specification. o Deviations (deficiencies) must be negotiated with the customer to establish a means for resolving the errors. • Configuration review or audit is used to ensure that all elements of the software configuration have been properly developed, cataloged, and documented to allow its support during its maintenance phase. ```

Answer 35

Validation Testing

Question 36

* Making sure the software works correctly for intended user in his or her normal work environment. * Alpha test – version of the complete software is tested by customer under the supervision of the developer at the developer’s site * Beta test – version of the complete software is tested by customer at his or her own site without the developer being present

Answer 36

Acceptance Testing

Question 37

* Series of tests whose purpose is to exercise a computer-based system * The focus of these system tests cases identify interfacing errors * Recovery testing – checks the system’s ability to recover from failures * Security testing – verifies that system protection mechanism prevent improper penetration or data alteration * Stress testing – program is checked to see how well it deals with abnormal resource demands (i.e. quantity, frequency, or volume) * Performance testing – designed to test the run-time performance of software, especially real-time software * Deployment (or configuration) testing – exercises the software in each of the environment in which it is to operate

Answer 37

System Testing

Question 38

* The symptom and the cause may be geographically remote (symptom may appear in one part of a program). * The symptom may disappear (temporarily) when another error is corrected. * The symptom may actually be caused by non-errors (e.g., round-off inaccuracies). * The symptom may be caused by human error that is not easily traced. * The symptom may be a result of timing problems, rather than processing problems. * It may be difficult to accurately reproduce input conditions (e.g., a real-time application in which input ordering is indeterminate). * The symptom may be intermittent. This is particularly common in embedded systems that couple hardware and software inextricably. * The symptom may be due to causes that are distributed across a number of tasks running on different processors.

Answer 38

Bug Casuses

Question 39

* Debugging (removal of a defect) occurs as a consequence of successful testing. * Some people are better at debugging than others. * Common approaches (may be partially automated with debugging tools): * Brute force – memory dumps and run-time traces are examined for clues to error causes * Backtracking – source code is examined by looking backwards from symptom to potential causes of errors * Cause elimination – uses binary partitioning to reduce the number of locations potential where errors can exist)

Answer 39

Debugging Strategies

Question 40

* Is the cause of the bug reproduced in another part of the program? * What “next bug” might be introduced by the fix that is being proposed? * What could have been done to prevent this bug in the first place?

Answer 40

Bug Removal Considerations