Property Based Testing

Question 1

What is property based testing?

Answer 1

Defining tests over invariant properties or specs. Sample tests from the input space

Question 2

What is the concept behind property based testing?

Answer 2

Use formal reasoning and program analysis to make each unit test cover more behavior

Question 3

A test scenario can be either _______ or ______

Answer 3

concrete (x=5), abstract (all x: x>0)

Question 4

For abstract test cases, we can generate ________ and consult the _______. This is the motivation of property based testing

Answer 4

test-cases, oracle

Question 5

What does property based testing process focus on?

Answer 5

Generating many tests for functional properties, Focussed on goals

Question 6

Describe traditional and property testing

Answer 6

Traditional testing: Example based (concrete scenarios)

Property testing: Based on generic properties (for all x, y, z: f(x, y, z) -> g(x, y, z))

Question 7

In random sampling, what is a generator?

Answer 7

A way to sample complex types for your properties. Some domains may require effort to represent

Question 8

Property based testing can help developers ________ their code because the process is so spec focussed

Answer 8

understand

Question 9

What are some common properties that we can test? Give an example for each

Answer 9

Symmetry (encode(decode(x)) == x)
Alternatives (different sorts)
Induction (car(cons(head,tail)) == head)
Idempotence (sort(sort(x)) == sort(x))
Invariants (sort(x).size() == x.size())

Question 10

What are good properties to check for a sorting function sort(x)?

Answer 10

Size does not change after sort
is_ordered(x)
All elements present in both lists, along with frequency
freq(x) == freq(sort(x))
How can we resolve ties?

Question 11

What are some benefits for property testing?

Answer 11

Tests have clear mathematical presentation
Testing process moves from examples to entire input space
Can decrease maintenance costs with same coverage
Failing tests have test case reduction applied