Chapter 8 - Testing Flashcards
test pyramid
pyramid partitions the tests according to their granularity.
Unit tests
check small parts of the code, usually a single class.
They form the base of the pyramid, meaning most tests fall into this category. Unit tests are simple, easier to implement, and fast to run.
integration tests or service tests
verify a system’s functionality or transaction
These tests involve multiple classes from different packages and may include external components like databases. They require more time to implement and are slower to run.
end-to-end tests, also referred to as user interface tests or system tests
simulate a user session on the system as authentically as possible.
They are more complex, time-consuming, and fewer in number. End-to-end tests also tend to be brittle, meaning minor alterations in the user interface might require changes in these tests.
Automated test percentage breakdown
A generic recommendation suggests that automated tests should be implemented in the following proportion: 70% as unit tests; 20% as integration tests; and 10% as end-to-end tests
defect (bug)
when a piece of code does not comply with its specification
failure
If defective code is executed and causes the program to produce an incorrect result or behavior
fixture
create the test context
This involves instantiating the objects we intend to test and, if necessary, initializing them
The program state verified by one or more test methods, including data, objects, etc. In the context of unit testing, the function of a fixture is to fix the state, i.e., set up the data and objects to be verified by the test.
Test Method
A method that implements a test and is annotated with the @Test annotation
Test Case
In JUnit, the term refers to a class containing test methods. The name originates from the first versions of JUnit, where the test methods were implemented in classes that inherited from a TestCase class.
Test Suite
A set of test cases typically executed together by the unit testing framework, which in our case is JUnit.
System Under Test (SUT)
The system being tested. It’s a generic term, also used in other types of tests, not limited to unit tests. The term production code is also sometimes used to refer to the SUT.
Test-Driven Development
Development style: you can write the tests first, before any production code. Initially, these tests will fail. Thus, you start with code that only compiles but whose tests fail. Then you implement the production code and test again. At this point, the tests should pass.
regressions
occurs when a modification in one part of the code—whether to fix a bug, implement a new feature, or perform a refactoring—inadvertently introduces a bug in another part.
FIRST
Fast
Independent
Repeatable
Self-checking
Timely
Fast
Developers must run unit tests frequently to receive feedback about bugs and regressions. Therefore, it’s important that these tests execute quickly, ideally in milliseconds.
Independent
The order of execution of unit tests should not matter. For any two tests T1 and T2, running T1 followed by T2 must produce the same result as running T2 and then T1. Indeed, T1 and T2 could also be executed concurrently. For the tests to be independent, T1 should not change any part of the global state that is later used by T2, and vice versa.
Repeatable
Unit tests should always provide the same result. This means that if a test T is called n times, the result should be the same in all n executions. Therefore, T should either pass in every execution or fail in every execution.
Self-checking
The result of unit tests should be easily verifiable. Developers should not have to open and analyze an output file or manually provide input data to interpret the test result. Instead, the results should be displayed in the IDE
Timely
Tests should be written as early as possible, ideally even before the code that needs to be tested.
Flaky Tests or Erratic Tests
Tests with non-deterministic results
Concurrency is one of the main causes of flaky behavior
Test Smells
represent sub-optimal implementation decisions in test code, which, in principle, should be avoided
Includes:
Obscure Test
Tests with Conditional Logic
Code Duplication
Obscure Test
a long, complex, and difficult-to-understand test
Tests with Conditional Logic
include code that may not be executed.
Code Duplication
occurs when there are repeated blocks of code in several test methods
Test coverage
a metric that helps determine the number of tests we need to write for a program.
It measures the percentage of statements in a program executed by the existing tests, calculated as:
Test coverage = (Number of statements executed by the tests) / (Total number of statements in the program)
function coverage
percentage of functions executed by the tests
function call coverage
among all the statements in a program that call functions, the percentage exercised by the tests
branch coverage
percentage of branches in a program executed by the tests; an if always generates two branches: one when the condition is true and another when it is false
C1 Coverage
C0 Coverage vs C1 Coverage
C0 Coverage: Statement coverages
C1 Coverage: branch coverages
Testability
describes how easy it is to test a program
mock (or stub)
we can create an object that emulates the real object, but only for testing purposes.
behavioral test
checks for events (e.g., method calls) that occur during the execution of the tests
Dummy Objects
passed as arguments to a method but they are not used in the method’s body. Their primary purpose is to bypass the language’s type system
Fake Objects
have a simpler implementation than a real object. For example, they can simulate a database in main memory.
test doubles
mocks and stubs
Dummy Objects
Fake Objects
service tests (or integration tests)
These tests aim to exercise a complete service, or a complete feature of the system, rather than testing a small unit of code
involve more classes, sometimes from distinct packages. They also test dependencies and real systems, such as databases and remote services.
system tests or interface tests (end-to-end tests)
These tests simulate the use of a system by a real user. They are the most resource-intensive automated tests, requiring more effort to implement and taking the longest to execute.
black-box technique
tests are written considering only the interface of the code under test.
functional tests
white-box technique
test implementation takes into account information about the code and its structure
structural tests
Equivalence Classes
recommends dividing the inputs of a program into sets of values that have the same likelihood of presenting a bug
Boundary Value Analysis
testing with the boundary values of each equivalence class and with the values that immediately precede or succeed such boundaries.
exhaustive testing
testing a program with all possible inputs
Random testing
the test data is chosen randomly
Acceptance Tests
tests carried out by the customers, using their own data. The results determine whether or not the customers will accept the implemented software.
Alpha tests
conducted with customers, but in a controlled environment, such as the developer’s workstation
beta tests
If the system passes the alpha tests, a test with a larger customer group can be conducted, no longer in a controlled environment
performance tests
to assess the system’s behavior under specific load conditions
Usability tests
used to evaluate the system’s user interface and often involve the observation of real users interacting with the system
Failure tests
simulate abnormal events in a system, such as the failure of specific services or even an entire data center.
