Regression Testing Flashcards
The testing process should support and facilitate software ________
evolution
What is regression testing?
Retesting software as it evolves to ensure previous functionality
Regression tests allow you to confidently ____ _____
make changes
Most testing is what type of testing?
regression testing
What are 2 reasons previously working functionalities could fail?
Changing one component can unintentionally impact the other
New environments can introduce unexpected behaviour
What are some common time intervals to run regression test suites?
On commit, nightly, weekly
What are some bad qualities of a regression test suite?
Takes weeks to run
Many manual steps
What are 3 common components for a regression test suite? What do they offer?
Tests for prev fixed bugs - identify bug prone components
Unit tests - Useful for refactoring
General system tests
Regression tests are usually a ______ of tests generated for other purposes
subset
What are smoke tests/sanities?
A subset of tests that validate the build process and core behaviors
How can we limit regression test suites?
Don’t add redundant tests
Run only sanities, then run more thorough tests nightly
Perform change impact analysis
What do you identify in change impact analysis?
How changes affect the rest of the software
What are 3 ways to decide which tests to run on demand with change impact analysis?
Conservative: Run all tests
Cheap: Run tests with test requirements related to the lines changed
Middle Ground: Run those tests affected by how changes propagate through the software
Why are cheap tests not adequate?
It may not catch unintentional changes due to dependencies
What do we do when regression tests fail and the new and old versions should be equivalent (during refactors)?
Find the misbehaving code and correct it
What are some reasons for regression test failures when the new and old versions are not equivalent (maybe extreme refactoring)?
The software has a bug to fix
Test inputs are stale and must be fixed
Expected behaviour has changed and must be fixed
Identifying the cause of failure for regression tests and maintaining the tests in general is ______
costly
How can we separate the reasons for regression test failures
Commit them separately
What are scale issues with regression tests? How can we address these burdens?
Running the tests Interpreting the results Updating tests Adding new tests Focus on automated testing and testability
