Regression Testing

Question 1

The testing process should support and facilitate software ________

Answer 1

evolution

Question 2

What is regression testing?

Answer 2

Retesting software as it evolves to ensure previous functionality

Question 3

Regression tests allow you to confidently ____ _____

Answer 3

make changes

Question 4

Most testing is what type of testing?

Answer 4

regression testing

Question 5

What are 2 reasons previously working functionalities could fail?

Answer 5

Changing one component can unintentionally impact the other

New environments can introduce unexpected behaviour

Question 6

What are some common time intervals to run regression test suites?

Answer 6

On commit, nightly, weekly

Question 7

What are some bad qualities of a regression test suite?

Answer 7

Takes weeks to run

Many manual steps

Question 8

What are 3 common components for a regression test suite? What do they offer?

Answer 8

Tests for prev fixed bugs - identify bug prone components
Unit tests - Useful for refactoring
General system tests

Question 9

Regression tests are usually a ______ of tests generated for other purposes

Answer 9

subset

Question 10

What are smoke tests/sanities?

Answer 10

A subset of tests that validate the build process and core behaviors

Question 11

How can we limit regression test suites?

Answer 11

Don’t add redundant tests
Run only sanities, then run more thorough tests nightly
Perform change impact analysis

Question 12

What do you identify in change impact analysis?

Answer 12

How changes affect the rest of the software

Question 13

What are 3 ways to decide which tests to run on demand with change impact analysis?

Answer 13

Conservative: Run all tests
Cheap: Run tests with test requirements related to the lines changed
Middle Ground: Run those tests affected by how changes propagate through the software

Question 14

Why are cheap tests not adequate?

Answer 14

It may not catch unintentional changes due to dependencies

Question 15

What do we do when regression tests fail and the new and old versions should be equivalent (during refactors)?

Answer 15

Find the misbehaving code and correct it

Question 16

What are some reasons for regression test failures when the new and old versions are not equivalent (maybe extreme refactoring)?

Answer 16

The software has a bug to fix
Test inputs are stale and must be fixed
Expected behaviour has changed and must be fixed

Question 17

Identifying the cause of failure for regression tests and maintaining the tests in general is ______

Answer 17

costly

Question 18

How can we separate the reasons for regression test failures

Answer 18

Commit them separately

Question 19

What are scale issues with regression tests? How can we address these burdens?

Answer 19

Running the tests
Interpreting the results
Updating tests
Adding new tests
Focus on automated testing and testability