Hoofdstuk 5 - Testing

Question 1

What is testing, a testing technique and a testing strategy?

Answer 1

◦ Testing is the activity of executing a program with the intent of finding a defect
◦ Testing techniques are techniques with a high probability of finding an as yet undiscovered mistake. A criterion for this is that you need coverage of the code/requirements/model/…
◦ A testing strategy will tell you when you should perform which testing technique. You need confidence that you can safely proceed

Question 2

What is the difference between an error, a failure and a defect?

Answer 2

◦ A defect is a design or coding mistake that may cause abnormal behavior
◦ A failure is a deviation between the specification and the running system. It is a manifestation of a defect during system execution.
◦ An error is the input that causes a failure.

Question 3

What is a test case? A test stub? A test driver? A test fixture?

Answer 3

◦ A test case is a set of inputs and expected results that exercise a component with the purpose of causing failures. It is a predicate that answers “true” when the component answers with the expected results for the given input and “false” otherwise

    ◦ A test stub --> It is dummy code providing necessary input values and behavior

    ◦ A test driver --> A main function that executes a number of test cases

    ◦ A test fixture is a fixed state of software under test, baseline running tests. It covers all that is needed to set-up the appropriate test context.

Question 4

What are the differences and similarities between basis path testing, condition testing and loop testing?

Answer 4

◦ Basis path testing: all portions of control flow (=branches) at least once
◦ Condition testing: all possible values of compound conditions at least once
◦ Loop testing: all loops, iterated at least 0, once, and N times.

Question 5

How many tests should you write to achieve MC/DC coverage? And multiple condition coverage?

Answer 5

◦ MC/DC: n + 1 test cases for a decision with n conditions
◦ Multiple condition: 2n test cases for a decision with n conditions

Question 6

Where do you situate alpha/beta testing in the four quadrants model?

Answer 6

In the upper right quadrant. This is the “manual” quadrant situated between business facing and critique product.

Question 7

What are the differences and similarities between unit testing and regression testing?

Answer 7

◦ Similarities:
▪ Optimal Fault Localization
▪ Automate as much as possible
◦ Differences:
▪ Unit Testing – Exercise small component (“unit under test”)
▪ Regression Testing – Exercise complete system (“no regression”)

Question 8

How do you know when you tested enough?

Answer 8

◦ Cynical answers (sad but true)
▪ You’re never done: each run of the system is a new test
▪ You’re done when you are out of time/money
◦ Non-cynical answer:
▪ Test until you’ve reduced failure rate under risk threshold (statistical testing)

Question 9

What is alpha-testing and beta-testing? When is it used?

Answer 9

◦ Alpha testing:
▪ End-users are invited at the developers site
▪ testing is done in a controlled environment
◦ Beta testing:
▪ Software is released to selected customers
▪ Testing is done in “real world” setting, without developers present

Question 10

What is the difference between stress-testing and performance testing?

Answer 10

◦ Stress testing tests extreme conditions while performance testing tests run-time performance of system.
◦ Stress testing example: provide input twice as fast and check whether system fails
◦ Performance testing example: time/memory consumption → first do it, then do it right, then do it fast.