csci 387 quiz 4 Flashcards
program testing is intended to show that
a program does what it is intended to do and to discover program defects before it is put into use
when you test software, you execute a program using
artificial data
program testing goals
to demonstrate to the developer and the customer that the software meets its requirements (validation); to discover situations in which the behavior of the software is incorrect, undesirable, or does not conform to its specification (defect)
for custom software, there must be at least ? test(s) for every requirement in the requirements document
one
for generic software products, there must be tests for ?
all of the system features, plus combinations of these features, that will be incorporated in the product release
defect testing is concerned with
rooting out undesirable system behavior such as crashes, unwanted interactions with other systems, incorrect computations, and data corruption
verification
are we building the product right (the software should conform to its specification)
validation
are we building the right product (the software should do what the user really requires)
aim of verification and validation is to
establish confidence that the system is ‘fit for purpose’
verification and validation confidence depends on
system’s purpose, user expectations, and marketing environment
static verification
software inspections concerned with analysis of the static system representation to discover problems (may be supplemented by tool-based document and code analysis)
dynamic verification
software testing concerned with exercising and observing product behavior (the system is executed with test data and its operational behavior is observed)
software inspections involve
people examining the source representation to discover anomalies/defects
inspections do not require ?, so they may be used ? implementation
execution of a system; before
during testing, ? can mask/hide other errors
errors
incomplete versions of a system can be inspected (with/without) additional costs
without
an inspection can also consider broader quality attributes of a program, such as ?
compliance with standards, portability, maintainability
inspections and testing are ? verification techniques
complementary and not opposing
both ? and ? should be used during the V&V process
inspections and testing
inspections can check conformance with ? but not with ?
a specification; a customer’s real requirements
inspections cannot check ? characteristics such as ?
non-functional; performance, usability, etc.
stages of testing
development, release, user
development testing
the system is tested during development to discover bugs and defects
release testing
a separate testing team tests a complete version of the system before it is released to users
user testing
users or potential users of a system test the system in their own environment
development testing includes
unit testing, component testing, and system testing
unit testing
individual program units or object classes are testing
unit testing should focus on testing
the functionality of objects or methods
component testing
several individual units are integrated to create composite components
component testing should focus on
testing component interfaces
system testing
some or all of the components in a system are integrated and the system is tested as a whole
system testing should focus on
testing component interactions
unit testing is a ? testing process
defect
units in unit testing may be
individual functions or methods within an object, object classes with several attributes and methods, composite components with defined interfaces used to access their functionality
object class testing: complete test coverage of a class involves
testing all operation associated with an object; setting and interrogating all object attributes; exercising the object in all possible states
? makes it more difficult to design object class tests as the information to be tested is not localized
inheritance
using a state model, identify ? to be tested and the ? to cause these transitions
sequences of state transitions; event sequences
whenever possible, unit testing should be ? so that tests are run and checked without manual intervention
automated
in automated unit testing, you make use of a ? to write and run your program tests
test automation framework (such as JUnit)
unit testing frameworks provide generic test classes that you ? to create specific test cases. they can then run all of the tests that you have implemented and report on ?
extend; the success of the tests
automated test components
setup, call, assertion parts
setup part
where you initialize the system with the test case, namely the inputs and expected outputs
call part
where you call the object or method to be tested
assertion part
where you compare the result of the call with the expected results (true = successful)
two types of unit test cases
- reflect normal operation of a program/show that the component works as expected
- use abnormal inputs to check that these are properly processed and do not crash the component
testing strategies
partition, guideline-based
partition testing is where you
identify groups of inputs that have common characteristics and should be processed in the same way
guideline-based testing is where you
use testing guidelines to choose test cases (guidelines reflect previous experience of the kinds of errors that programmers often make when developing components)
equivalence partition
the domain where the program behaves in an equivalent way for each class member
boundary value analysis/range checking
test boundaries between equivalence partitions; test values at both valid and invalid boundaries
test software with sequences which have only a ? value
single
use sequences of (different/same) sizes in different tests
different
derive tests so that the ? elements of the sequence are accessed
first, middle, last
test with sequences of length ?
zero
choose inputs that force the system to
generate all error messages
design inputs that cause input buffers to
overflow
force computation results to be
too large or too small
software components are often composite components that are made up of
several interacting objects
you access the functionality of the interacting objects through the defined
component interface
testing composite components should focus on showing
that the component interface behaves according to its specification (you can assume unit tests have been completed)
interface types
parameter, shared memory, procedural interfaces, message passing interfaces
parameter interfaces
data passed from one method or procedure to another
shared memory interfaces
block of memory is shared between procedures or functions
procedural interfaces
sub-system encapsulates a set of procedures to be called by other sub-systems
message passing interfaces
sub-systems request services from other sub-systems
interface misuse
a calling component calls another component and makes an error in its use of its interface (e.g., parameters in the wrong order)
interface misunderstanding
a calling component embed assumptions about the behavior of the called components which are incorrect (e.g., binary search in an unsorted array)
timing errors with interface
the called and calling component operate at different speeds and out-of-date information is accessed
for the interface, design tests so that parameters to a called procedure are at the ? ends of their ranges
extreme
for the interface, always test pointer parameters with ? pointers
null
system testing during development involves
integrating components to create a version of the system and then testing the integrated system
the focus in system testing is
testing the interactions between components
system testing tests the ? behavior of a system
emergent
during system testing, ? and ? may be integrated with the newly developed components
reusable components that have been separately developed and off-the-shelf-systems
the use-cases developed to identify system interactions can be used as
a basis for system testing
testing policies
define the required system test coverage
examples of testing policies
all system functions that are accessed through menus should be tested; combinations of functions that are accessed through the same menu should be tested; etc.
test coverage
measures the degree to which the specification or code of a software program has been exercised by tests
code coverage
measures the degree to which the source code of a program has been tested
code coverage criteria include
equivalence, boundary, control-flow, and state-based testing
statement coverage
each statement executed at least once by some test case
edge coverage
every edge (branch) of the control flow is traversed at least once by some test case
condition coverage
every condition takes true and false outcomes at least once in some test case
path coverage
finds the number of distinct paths through the program to be traversed at least once
control flow testing using the ? of a program to ?
control structure; develop the test cases for the program
the control flow graph of the program represents
the control structure of a program
the control flow graph G=(N, E) of a program consists of
a set of nodes N and a set of edges E
each node on the control flow graph represents ?; there are ? types of nodes; there is a unique ? and a unique ?
a set of program statements; five; entry, exit
in the control flow graph, there is an edge from node a to node b if
the control may flow from the last statement in a to the first statement in b
decision node
contains a conditional statement that creates two or more control branches (if or switch)
merge node
represents a program point where multiple control branches merge
statement node
contains a sequence of statements; control must enter from the first statement and exit from the last
state-based testing
defines a set of abstract states that a software unit can take and tests the unit’s behavior by comparing its actual states to the expected states
the state of an object is defined as
a constraint on the values of object’s attributes
ensure state coverage conditions by
covering all identified states at least once, covering all valid transitions at least once, and triggering all invalid transitions at least once
test-driven development is an approach to program development in which you
interleave testing and code development; tests are written before code and ‘passing’ the tests is the critical driver of development
TDD
start by identifying the increment of functionality that is required; write a test for this and implement; run this test; once all tests run successfully, move on
benefits of test-driven development
code coverage, regression testing, simplified debugging, system documentation
regression testing
testing the system to check that changes have not ‘broken’ previously working code; all tests are rerun every time a change is made to the program
release testing
the process of testing a particular release of a system that is intended for use outside of the development team
the primary goal of the release testing process is to
convince the supplier of the system that it is good enough for use
release testing is usually a ? testing process where tests are only derived from the system specification
black-box
requirements-based testing involves
examining each requirement and developing a test or tests for it
performance testing
usually involves planning a series of tests where the load is steadily increased until the system performance becomes acceptable
stress testing is a form of performance testing where the system is
deliberately overloaded to test its failure behavior
user or customer testing is a stage in the testing process in which
users or customers provide input and advice on system testing
user testing is essential because
the user’s working environment may have a major effect on the reliability, performance, usability, and robustness of a system that cannot be replicated in a testing environment
types of user testing
alpha, beta, acceptance
alpha testing
users of the software work with the development team to test the software at the developer’s site
beta testing
a release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers
acceptance testing
customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment; primarily for custom systems
in agile methods, the user/customer is part of ? and is responsible for ?
the development team; making decisions on the acceptability of the system
in agile methods, there (is a / is no) separate testing process
is no
