Black box Flashcards
Software testing is an important technique for assessing the quality of a software product. In this chapter, we will explain the following: 1 • • • • • the basics of software testing, a verification and validation practice, throughout the entire software development lifecycle the two basic techniques of software testing, black-box testing and white-box testing six types of testing that involve both black- and white-box techniques. strategies for writing fewer test cases and still finding a
What is black box testing?
Testing in which the tester does not know or have access to source code itself. It is derived from the box being ‘black’ or ‘opaque’. Tester knows inputs and expected output and tests.
Black box testing (also called functional testing) is testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions.
What is white box testing?
Testing where the tester knows or has access to source code, this would be the software developer. This is in reference that the box is invisible e.g.: white.
White box testing (also called structural testing and glass box testing) is testing that takes into account the internal mechanism of a system or component.
Software testing is part of what two processes?
Verification and Validation.
What is an informal definition of verification?
Verification: Are we building the product right?
Through verification, we make sure the product behaves the way we want it to. For example, on the left in Figure 1, there was a problem because the specification said that players should collect $200 if they land on or pass Go. Apparently a programmer implemented this requirement as if the player had to pass Go to collect. A test case in which the player landed on Go revealed this error.
Verification (the first V) is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase [11]. Verification activities include testing and reviews. For example, in the software for the Monopoly game, we can verify that
two players cannot own the same house.
What is an informal definition of validation?
Validation: Are we building the right product?
Through validation, we check to make sure that somewhere in the process a mistake hasn’t been made such that the product build is not what the customer asked for; validation always involves comparison against requirements. For example, on the right in Figure 1, the customer specified requirements for the Monopoly game – but the programmer delivered the game of Life. Maybe the programmer thought he or she “knew better” than the customer that the game of Life was more fun than Monopoly and wanted to “delight” the customer with something more fun than the specifications stated. This example may seem exaggerated – but as programmers we can miss the mark by that much if we don’t listen well enough or don’t pay attention to details – or if we second guess what the customer says and think we know better how to solve the customer’s problems.
What is software testing?
Software testing is the process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs) and to evaluate the features of the software item [9, 12].
Verification vs Validation.
Verification
Are we building the product right?
“I landed on “Go” but didn’t get my $200!”
Validation
Are we building the right product?
“I know this game has money and players and “Go” – but this is not the game I wanted.”
What are the 5 ways software may not be incorrect?
Mistake – a human action that produces an incorrect result
Fault [or Defect] – an incorrect step, process, or data definition in a program
Failure – the inability of a system or component to perform its required function within the specified performance requirement.
Error – the difference between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition
Specification – a document that specifies in a complete, precise, verifiable manner, the requirements, design, behavior, or other characteristic of a system or component, and often the procedures for determining whether these provisions have been satisfied.
Define ‘mistake’
a human action that produces an incorrect result.
Define ‘fault’ (aka ‘defect’)
an incorrect step, process, or data definition in a program.
Define ‘failure’
the inability of a system or component to perform its required function
within the specified performance requirement.
Define ‘error’
the difference between a computed, observed, or measured value or
condition and the true, specified, or theoretically correct value or condition.
Define ‘specification’
a document that specifies in a complete, precise, verifiable
manner, the requirements, design, behavior, or other characteristic of a system or component, and often the procedures for determining whether these provisions have been satisfied.
What is the order that errors, faults, and mistakes occur?
mistake > fault > failure > error
A mistake committed by a person becomes a fault (or defect) in a software artifact, such as the specification, design, or code.
This fault, unless caught, propagates as a defect in the executable code. When a defective piece of code is executed, the fault may become a visible anomaly (a variance from the specification or desired behavior) and a failure is observed
Otherwise, the fault remains latent. Testing can reveal failures, but it is the faults that must be found and removed [3]; finding a fault (the cause of a failure) can be time consuming and unpredictable. Error is a measure of just how incorrect the results are.
What do we try to cause during testing?
A purpose of testing is to cause failures in order to make faults visible [10] so that the faults can be fixed and not be delivered in the code that goes to customers. Another purpose of testing is to assess the overall quality level of the code. For example, a test team may determine a project with too many high-severity defects should be sent back to development for additional work to improve the quality before the testing effort should continue. Or, the management may have a policy that no product can ship if testing is continuing to reveal high-severity defects.
What are the two expense cases of testing?
To test, or not to test.
To minimize costs of testing, what is one goal of testing?
To minimize the costs associated with testing and with software failures, a goal of testing must be to uncover as many defects as possible with as little testing as possible
We want to write test cases that what likelyhood of uncovering how many faults?
In other words, we want to write test cases that have a high likelihood of uncovering the faults that are the most likely to be observed as a failure in normal use
All software testing is performed with what type of code?
Executable code, that is code that runs so we can perform tests against it.
What type of minimal code might need to be written to perform a test against?
Scaffolding code (or just scaffolding)
How is scaffolding defined?
Scaffolding is defined as computer programs and data files built to support software development and testing but not intended to be included in the final product [11].
What does scaffolding do?
Scaffolding code is code that simulates the functions of components that don’t exist yet and allow the program to execute [16]
What are the two components of scaffolding?
Scaffolding code involves the creation of stubs and test drivers.
What’s a stub?
Stubs are modules that simulate components that aren’t written yet, formally defined as a computer program statement substituting for the body of a software module that is or will be defined elsewhere [11]. For example, you might write a skeleton of a method with just the method signature and a hard-coded but valid return value.