Lecture 15 - Python Testing and Debugging

Question 1

What is TDD? (3)

Answer 1

• instead of writing formal specifications of behavior, program behavior is specified as a series of test cases which must be passed
• the minimum necessary code is then written to make the program pass the test cases
• test cases are gradually added until the program has the desired functionality

Question 2

What’s an interesting characteristic of TDD?

Answer 2

Will often roll back the code and write the update code again instead
of spending a long time debugging a single piece of code

Question 3

What are the steps of the TDD cycle? (6)

Answer 3

Question 4

What is a code coverage analyzer?

Answer 4

A tool to tell you what parts of a program have actually been active

Question 5

What’s the idea of code coverage?

Answer 5

Infrequently-used statements in a program can hide bugs for a long time, so we want to figure out how to actually use everything

Question 6

What is a test double?

Answer 6

A generic term for any case where you replace a production object for testing purposes.

Question 7

Why do we use test doubles?

Answer 7

Sometimes it’s not possible to unit test certain code because of the unavailability of an external dependency (e.g. a database)

Question 8

For a test double, when can the substitution be made? (2)

Answer 8

• –link time (compile alternate version)
• –run time (only for the duration of the actual test)

Question 9

What are the types of test doubles?

Answer 9

Question 10

How are test doubles done at runtime for

• Java
• Python

Answer 10

Question 11

What is monkey patching?

Answer 11

“dynamic modifications of a class or module at runtime,
motivated by the intent to patch existing third-party code as
a workaround to a bug or feature which does not act as
desired”

Question 12

Applications of monkey patching (4)

Answer 12

–stub out a function for testing
–modify behavior of third-party code without forking that code
–apply a run-time patch
–distribute security fixes

Question 13

What are dangerous things to be aware of regarding monkey patching? (3)

Answer 13

• changes in the code being patched can break the patch
• multiple modules trying to monkey patch the same thing can conflict
• source code on disk no longer reflects the actual behavior of the program, making debugging very difficult

Question 14

Describe how to monkey patch in the case where you want to replace a function

Answer 14

Question 15

• Describe how to monkey patch in the case where you want to wrap a function (2)

Answer 15

• To add code that executes before and/or after an existing function:
  
  • Copy the original function pointer.
    
    • a closure is a convenient place to store the original function pointer if you don’t need to restore it
  • replace that pointer with a pointer to your monkey patch function

Question 16

Why would you wrap a function when monkey patching? (wrapping is a subcase of monkey patching)

Answer 16

To add code that executes before and/or after an existing function

Question 17

In the monkey patching case where you wrap a function, what happens when the patch function is invoked?

Answer 17

• –execute the “before” action(s)
• –call the original function and store the returned result
• –execute the “after” action(s)
• –return the stored result of the original function

Question 18

What happens if you forget to return the stored result of the original function, when wrapping a function for monkey patching

Answer 18

anything relying on the function’s return value breaks

Question 19

In the case of wrapping a function for monkey patching, how do you unwrap the function (aka undo the monkey patch)?

Answer 19

–store the copy of the original function pointer you made before applying the patch back into the function pointer

Question 20

What are 3 unit testing modules in python?

Answer 20

• unittest module
• doctest module
• mock module

Question 21

What are high level descriptions/characteristics for

• unittest (2)
• doctest (1)
• mock (1)

Answer 21

unittest:

• allows writing full unit tests
• tests do not need to be in single file; all tests are found and invoked by the main harness

doctest:

• allows writing simplified tests as comments

mock:

• allows insertion of replacement objects that monitor how they are being used and/or return specific values

Question 22

Describe how to use unittest

Answer 22

Question 23

Describe doctest (3)

Answer 23

Question 24

Describe the mock module (2)

Answer 24

• Allows parts of the system under test to be replaced by mock objects
• Provides classes Mock and MagicMock

Question 25

Describe the Mock and MagicMock classes (3)

What can you do with them?

Answer 25

1.

–callable, create attributes as new mocks when accessed
–accessing the same attribute will always return the same mock
–MagicMock implements Python’s internal “magic” methods

2.

–record the attribute accesses
–permit assertions to be made about what has been done with them

Question 26

• What’s the first step of code optimization?
• What do you use to help with this?

Answer 26

• Find out which parts of your code need to be faster
• A code profiler will tell you which functions account for the most CPU time

Question 27

Compare two code profilers in python (3)

Answer 27

profile and cProfile modules

• both have same API
• Profile is in pure Python (more overhead, allows extensions)
• cProfile is written in C and has much less overhead

Question 28

How do you use cprofile? (actual code)

Answer 28

Question 29

List python-specific optimizations (2)

Answer 29

• use comprehensions or map/reduce instead of for loops
  
  • reduces interpreter overhead
• ensure integers stay within +/- 231 to avoid slower inf-precision math

Question 31

What is part of the mock module? (1)

Answer 31

Mock and MagicMock classes

Question 32

When is setUp and tearDown run?

Question 33

How do you introduce a doctest?

Answer 33

In a doc string, use >>>

Question 34

How do you import doctest?

Answer 34

• Import doctest
• doctest.testmod()

Question 35

What are methods of mock class? (3)

Answer 35

• M.called
• m.call_count
• m.called_with_args

Question 36

What can you do with the mock class?

Answer 36

Among other things:

• Can assert that method was called
• What args a method was called with