Unit/Mutation Testing Flashcards
What do programmers write tests to do?
- Confirm an understanding of the specification
- Can ensure that all different aspects are verified
- Can test individual components that are outside of normal view
- Internal (private) classes /functions /methods that are not directly exposed or deep in the application
What are the two options for testing code?
- Code Trace
- A code Trace is a static analysis that is done by viewing the code and walking through the code by hand.
- Debugging to understand or manipulate operation. - Write coded tests to execute specific components of a program with particular data sets.
- Add code using print to screen or print to file to confirm operation.
- Add structured Unit Tests to the Project
What is a Unit Test?
A unit test is a test case using a single data set that tests a function/method or confirms the state of a component (class instance or object)
- A unit test is constructed as a method of a special testing class that is not part of the application, but is included in the project.
- Units tests may or may not ship with the source code of the product.
How do you confirm a software component it operational?
Software components can have their operation confirmed by using a number of data sets that reflect the operation of a component over a range of values.
- In general, most methods / functions will require more than a single unit test to confirm operation.
- The goal of unit testing is to provide tests that provide data coverage and code coverage
Define Data Coverage
- Testing that uses different data combinations of input data that will ensure that the range of acceptable and unacceptable will produce the correct results/
- 100% data coverage indicates that ALL combinations of data sets have been supplied to the component being tested **
- ** Generally not practical
AIM to achieve 100% PRACTICAL data coverage
- ** Generally not practical
Define Code Coverage
- Testing that will ensure that all code paths have been executed at least once during the set of test cases used.
- NOTE: Practical data coverage MUST yield 100% code coverage
What are the main gotcha’s to remember in code coverage tests?
- Check both sides of OR statements in an IF
- Check both upper and lower limit boundaries
What is the goal of unit testing?
- Confirmation of expected operation
- These are considered
normal data sets
- Use cases (supplied
during specification
development) are useful
to develop meaningful
test data sets - Confirmation of expected error conditions
- These are considered to
be non-normal datasets
- Out of range data sets.
invalid data should
result in continued
program operation
using a standard error
handling mechanism
Unit testing works on which principal?
The principal of applying assertions to method/function results
What are assertions?
An assertion is a way to confirm that a return value or object variable contains a value that is known to be correct.
eg. if (rectangle.getHeight() == height)System.out.println(“PASSED – Height set successfully”);elseSystem.out.println(“FAILED – Height set failed”);
How are Assertions used when developing unit tests?
Assertions are placed into unit tests to confirm or verify an expected result.
- If the assertion results in a true response value the test is said to pass
- A false return value from an assertion is considered a fail
What annotation is used to mark the method as a test method?
@Test before the method
Give an example of a more general assertion that could be used for testing a result
@Test
public void testCalculateAmount() {
double expectedAmount = 1279.61;
Mortgage instance = new Mortgage(0.06,25,200000);
instance.calculateAmount();
int result =
instance.getAmount();
// Confirm values are the same
assertEquals (expectedAmount, result, 1.0E-6);}
(the last param is the accuracy since testing doubles “0.0001% accuracy”)
What are the 2 @Test options?
Expected=exception
Timeout=value in ms
What is the expected exception option?
@Test (expected = ArithmeticException.Class)
- Used to manage cases where the pass case produces an exception
What is the timeout value option?
@Test (timeout = 10000)
- Used to manage cases where the function needs to provide a performance guarantee
What is the goal of Test Data Selection?
- Create test data sets that will produce examples of ALL expected results with a minimum number if test cases.
- Equivalent - duplicate test cases should be removed
What is equivalence partitioning?
- Test cases that are “generally” the same.
- A test case that will produce a result or code path that is identical to another test case that has already been executed. (note an if statement that imposes a limit must be created to test both sides)
- Allows the tester to reduce the number of test cases that will be required to prove desired / documented operation.
What are boundary conditions?
Boundary conditions are test cases that are at the edge of the acceptable data range.
- Can include data range limits.
- Can also include size limits.
- Can include performance limits.
What are boundary issues?
Many subtle errors will exist at the edges of ranges or at data size limits (array size limits)
- If a function has an integer input range of 10 to 20 inclusive for a parameter a full boundary condition test will include 10, 20, 9, and 21
What are the two types of data sets unit tests must include?
Normal Data Sets
- Typical use is to confirm operation and results of calculations
Non-Normal Data Sets
- Typical use is to confirm special cases and or expected error conditions
Name a source that could be used to generate a test data set?
The specification and/or use cases are an excellent source of the initial data sets that can be used to for test cases
What is a good test data set for VV and why?:
Test Case evaluation
Integer[] findMultiples (int[] values)
- Method will take an array of values and return an array of the values that more than once.
- The input array can be any array of integer values.
- values = {1,10,100,1000,-100,200,10,10,1000}
- return = {10,1000} or {1000,10} - Observations :
▪Data set contains a range of values (including negatives)
▪Multiple values are returned.
▪Some value are rejected.
▪The same negative and positive value is in the set (different numbers should be rejected)
▪The last value in the array is included in the set.
ACCEPTABLE DATA SET
What are some examples of non normal data sets for the previous method?
- Empty
▪{} no values provided
▪Could return an empty set or throw exception - A set with no recurring data ▪{1,2,3,4,5,6,7,8,9,10}
▪Should return an empty se
What is mutation testing and what does it involve?
Mutation testing:
- Evaluates the quality of software tests.
- Involved modifying a program’s source code in small ways.
- A test suite (set of test cases) that does not detect and reject the mutated code is considered defective.
Define Testing Automation
- Any mechanism that can be executed
- Report will be produces that summarizes the results of the test
- Junit within Netbeans is an example of on automated test mechanism
- All tests will run
- A simple report summary (bar chart) is generated showing pass cases and fail cases.
- Details of failures can be provided
What are the 4 main ways that test automation can aid the tester and the testing process?
Speed - Tests can be executed without user intervention.
Efficiency - Test can be run independently of other development activities (multi-task)
Accuracy/Precision - The test will be the same each time.
Relentless - Test tools do not get tired or bored
What is regression testing?
Running a series of previously successfully passed tests against a new version/revision of the application or component.
JUNIT provides an automated regression testing mechanism that can/should be used to ensure code modifications do not break existing functionality.
Name two types of regression tests and an example application in an automated environment
White-Box Unit Testing
▪ All updates/revisions and verified against a set of units tests
▪ Can be integrated into the development process that will prevent revisions be accepted by the source control system unless all unit tests pass
Black-Box Integrated Test
▪ Result capture of integrated application is compared against a previous capture – Tools like diff (unix)
▪ Considered to be a compare against a GOLD-STANDA
What are the three realities to keep in mind when using test automation?
- Software may be changed / improved
▪Some tests may be invalidated with new versions of the software - Time spent on Testing (writing tests) will take time away from development
- Invasive Testing may impact a real-running system.
▪ If an automated tool finds a bug, try to reproduce by hand to confirm.
What types of mutations do you need to change?
- Language independent
- Value Mutations
- Decision Mutations
- Statement Mutations
- Language dependent
What are value mutations?
Changing constants, loop bounds (adding/subtracting values)
What are decision mutations?
< will be changed to > or <=
What are statement mutations?
swapping, deleting, duplicating lines of code
What are some language dependent mutations?
- changing modifiers from protected to public
- changing between static / non static
- delete member initialization (int x = 0;)
- delete this.xxx
- changing argument orders
