QA in Construction Flashcards
3 things that programmers must accomplish
1) Syntactically correct (grammar)
2) Software must be consistant with standards sanctioned by the project and organization
3) Software must implement algorithms and data structures in accord with design specs
Code Analysis
Verification and validation that is automated
Code Review
Verification and Validation that is not automated
Dynamic analysis
analysis techniques while programs are running
Static analysis
Examine software when it is not running
Formal Reviews
Code inspections because each participant has a well-defined role ,the activity follows a well-defined process, and the activity is guided by a checklist.
Rate of code review should be about how many lines of code per hour?
200 lines for preparation and team inspection should not exceed 150 per hour
What should code inspection checklist have on it?
Specific defects that inspectors should look for during preparation
Ex of things on code checklist
- naming conventions
- are variable names confusing?
- is every variable and attribute correctly typed
- method returns correct value
- appropriate access modifiers
- nested if statements should be converted into switch statement
- all exceptions handled
Simplest type of static analysis
Syntax Checking
Syntax Checking
ensures that the software obeys grammatical rules of language it is written in. Most frequently performed by a language-sensitive editor.
Something to be aware of Syntax Checkers
Editors do not always correctly identify the root cause of the problem
Style Checking
Standards related to typography based on a style guide. (upper and lower case, naming, spaces, tabs, brackets)
First style checker was called?
Lint
Usage checkers
look for 3 things:
- suspicious or error prone constructs (uninitialized variables, use of division operator)
- non-portable constructs (may have range problem)
- memory allocation inconsistencies.
Difference between usage and idiom checkers
Idiom checkers are more varied
Formal Methods
refer to the class of static analysis tools that rely on mathematical models
Types of Formal Methods (3)
Model checking
Data Flow analysis
Symbolic evaluation
Model Checking
the process of automatically determining if a program or sub program satisfies certain requirements. The inputs to a model checker are the program or sub-program and formal specs of requirements. Requirements specified using logical expressions.
Data Flow Analysis
Process of enumerating the set of possible values calculated at various points in a program or sub-program using ideas from graph theory
Symbolic evaluation
Process of automatically tracing the execution of a program or sub-program using symbolic values rather than numeric values. Used to identify the values that will cause different statements to be executed.
Unit Testing
The testing of individual units or sub-programs in isolation. One value for each sub-program argument and one corresponded expected output. To determine faults in sub-program
False negative
the conclusion that there are no faults due to an incomplete test suit
False Positive
Incorrect expected value (conclusion that there is a fault when there isn’t)
Black Box Testing
testing the processing details of the component, system or product are presumed to be unknown. The person developing the tests only knows what can go into the tested unit and what should come out
Clear/White/open box
Person developing the tests knows exactly how the componenet, sub-system or product is built and how it should operate. Combo of two is generally required for effective testing
What does clear box testing focus on?
Coverage of tests.
How to measure coverage?
Control Flow Graph (CFG)
Describe control flow graph:
action node - represents a piece of code with one entry point and one exit point (statement represented as an action)
decision node - represents the start of a piece of code with one entry point and multiple exit points
Statement Coverage
The percentage of statements exercised when a set of tests cases is executed.
Branch Coverage
Percentage of branch directions taken when a set of tests cases is executed. Identifies the outbound edges from decision nodes that have and have not been traversed.
Path Coverage
The percentage of all execution paths taken when a set of test cases executes.
Is 100% path coverage achievable for large programs
NO
Test Driven Development
TTD A process in which tests are developed as the code is and for very small increments in functionality.
Are all tests supposed to be created before the program is written?
No, You want to have some written before (black box) and then others for (TTD) written after the code that focus on coverage (clear box).
Complete enumeration/exhaustive testing
Creates a test suite that contains all possible inputs. Usually a huge number b/c has to do every value 2^32 for integer and 2^64
Oracle
Another sub-program that is known to output the correct answers
Randomization
“brute force” approach which can be done in either a structured or unstructured fashion. Stat with the completely enumerated test suit and then select tests from that suite in some probabilistic fashion ( looking for possible error triggers).
Boundary Value Analysis
Idea that failure triggers tend to be found at the “boundaries” of the sets of inputs and outputs.
Other popular value-based heuristics for individual parameters
- number 0 for dividing by zero
- very large and very small numbers (underflow and overflow)
- characters or string version of digits and numbers
- equal value params
- different relative value params
- small and large arrays
- arrays w/ 1 or 0 elements
- unsorted arrays
- arrays with duplicates and no duplicates
Equivalence class
Partitioning test cases into sets in which each test case in the set is expected to trigger the same failure. Helps so don’t have to create more than one test case from each equivalence class.
Ideal test should do what?
Correct the fault and NOT introduce a new fault
Regression Testing
Running all tests over again after making any software change (even just editing comments)
Assertion
relates actual output to expected output
What unit testing tool do we use?
JUnit
Advantages of Unit Testing
- easy to isolate different tests
- easy to construct isolate tests
- reports generated are easy to read and highlight the failed tests
- can be supplemented with code coverage tools
Symptom
A characteristic of a failure that can be observed
Error
Something that a person does that gives rise to a fault
Trigger Condition
Condition that causes a product to fail
Symptom and Error in a program that divides by 0
Symptom = the program crashes
Fault- the omission of statements that checks the user-supplied value and display an error message
Trigger Condition - dividing by zero
Debugging process (5) steps
Stabilize Localize Correct Verify Globalize
Stabilize
Understand the symptom and trigger condition of test so failure can be reproduced
Localize
Locate the fault (form hypothesis and then tests if your hypothesis for the error was correct)
Correct
Fix Fault
Verify
Test fix, make sure not introduce new fault
Globalize
Look for and fix similar defects in other parts of the system
Debug Code
Includes both temporary output statements that can be used to monitor state information and temporary input statements that can be used to pause the execution of a component
Biggest disadvantages of debug code:
- chunkiness
- can obscure the code it is meant to debug
- can take a significant amount of time to add
- it can be difficult to know what is output
- can print a lot of stuff
Debugger
A tool that can be used to trace the execution of a code without having to modify it in any way
Breakpoint
Pauses the execution of a program. Can start executing a program but pause execution when particular statement is reached
Step into
If statement is a call to a sub0program, enter the sub-program and execute each of its statements one at a time
Step over
Ignores internal details
Step out
Get out of a sub-program, continuing execution of its code until it returns then pausing again after the return
Watches
monitor state information. Can monitor values contained in different variables to see how they change.
Profiling
The process of measuring the amount of time (or space) used by sub-programs and statements during a particular execution of a program.
What kind of analysis is Profiling?
Dynamic b/c occurs while the program is executing
Refactoring
changing code without altering its behavior. To improve structure
When should refactoring be done (3)?
- duplication of code
- lack of clarity (code hard to understand)
- code smell
Code smell
Comments that duplicate code, classes that do nothing but hold data, failure to hide info, tight coupling, low cohesion, bloated classes, lazy classes, long methods, reliance on switch statements instead of classes
Introduce explain variable
refactoring technique that adds temp variable to make code easier to understand
Inline temporary variable
refactoring technique get rid of temp variables that make code harder to understand
Extract sub-program
refactoring technique that splits a sub-program into two or more sub-programs
Inline sub-program
refactoring technique when sub-program is too short to warrant being a separate identity get rid of it
What kind of testing should unit and integration testing be (black or clear)?
Clear
Test-driven development (TDD)
A style of coding in which writing tests is integral to the coding process itself. Has many tests that check behavior in ways that do not depend on code (because they are written before the code) -> high quality tests
Essential factors of TDD
testing framework is assumed tests are written before code tests and code are written incrementally code is only written to pass tests Refactoring is expected
