Extra Lecture: How to write good tests Flashcards
What are the two main types of program testing?
- FUNCTIONAL tests (ensure that a program works as expected i.e. does what it is expected to do)
- NON FUNCTIONAL tests (ensures that a program runs well)
What is the functionality of a program?
The functionality of a program is the set of ACTIONS, OPERATIONS, and BEHAVIOUR of a program. It’s functionality is about what it DOES
Functional Testing: What are the three main risks in writing tests?
- Not testing a feature called out in the specification.
Thus there is a danger that your software will not then have that feature. - Not testing an unusual input for a feature.
Thus there is a danger that the software will normally work, but won’t work in that particular case - Misunderstanding a feature described in the specification.
Thus there is a danger that the software passess all tests but doesn’t do what was required.
How can we reduce the probability of these three risks?
- Making sure that the tests are derived from the specification, not from the code.
- Having different people writing the tests from those writing the code.
- If the same programmer writes the code and tests, the programmer should write the tests
before writing the code.
i.e. these techniques seperate the development of the code from the developement of the tests. This makes it more likely that the tests are independent and not derived from the code. Also encourages double checking.
What is black box testing?
During black box testing, the verification team only looks at the inputs and outputs.
The testers do not look at the code.
What is exhaustive testing?
An exhaustive test checks that the all possible inputs give the correct results. This is the ideal.
However, it is only possible for the simplest programs.
The alternative is to design a series of tests that checks every type of function described in the
specification at least once.
The test designer must develop a set of tests that exercises all of the functions.
It is important to understand that the number of tests doesn’t matter.
What matters is that the functionality is thoroughly tested.
What are edge case tests?
Edge case testing means testing both sides of the boundaries. e.g. for a user inpu where the requested values are between 1 to 100 inclusive. Must test with 0 and 1, as well as 100 and 101 to check that the expected outcome is achieved for all.
What is code coverage?
The code coverage is a report which indicates which code in the script has been executed and which
has not been executed. The code coverage is summed over all of the tests.
Code coverage is normally measured as a percentage.
There are 3 types
What are the three types of code coverage?
The three main types of code coverage are:
- LINE COVERAGE which measures whether each line is executed or not.
- BRANCH COVERAGE which measures whether each branch is executed or not.
- branch refers to a possible path the code can take due to control flow, like if, else, for, while, try, etc. *
- CONDITION COVERAGE which measures whether each combination of true/false values is tested or not.
Some key things to remember when testing
- Be careful, 100% code coverage does not prove that the program is fully correct.
Some bugs don’t show up with code coverage measurements.
However, code coverage is a useful metric.
If a feature has not been tested, you must assume that it does not work.
A common mistake is assuming that feature works. Every time that the code is changed, all of the tests should be re-run. Re-running all the tests is called a regression test.
