Unit testing Flashcards
Different types of tests
integration tests, web tests, load tests, unit tests and others.
Test Driven Development (TDD)
Test Driven Development (TDD) is when a unit test is written before the code it is meant to check. TDD is like creating an outline for a book before we write it. It is meant to help developers write simpler, more readable, and efficient code.
Do and Donts
Try not to introduce dependencies on infrastructure when writing unit tests. These make the tests slow and brittle, and should be reserved for integration tests. You can avoid these dependencies in your application by following the Explicit Dependencies Principle and using Dependency Injection. You can also keep your unit tests in a separate project from your integration tests. This ensures your unit test project doesn’t have references to or dependencies on infrastructure packages.
What are unit tests?
Having automated tests is a great way to ensure a software application does what its authors intend it to do. There are multiple types of tests for software applications. These include integration tests, web tests, load tests, and others. Unit tests test individual software components and methods. Unit tests should only test code within the developer’s control. They should not test infrastructure concerns. Infrastructure concerns include databases, file systems, and network resources.
.NET Core 2.0 and later supports .NET Standard 2.0, and we will use its libraries to demonstrate unit tests.
You are able to use built-in .NET Core 2.0 and later unit test project templates for C#, F# and Visual Basic as a starting point for your personal project.
Integration tests
Try not to introduce dependencies on infrastructure when writing unit tests. These make the tests slow and brittle, and should be reserved for integration tests.
Infrastructure concerns include databases, file systems, and network resources.
Why unit test?
Less time performing functional tests:
Functional tests are expensive. They typically involve opening up the application and performing a series of steps that you (or someone else), must follow in order to validate the expected behavior. These steps may not always be known to the tester, which means they will have to reach out to someone more knowledgeable in the area in order to carry out the test. Testing itself could take seconds for trivial changes, or minutes for larger changes. Lastly, this process must be repeated for every change that you make in the system.
Unit tests, on the other hand, take milliseconds, can be run at the press of a button and do not necessarily require any knowledge of the system at large. Whether or not the test passes or fails is up to the test runner, not the individual.
Protection against regression:
Regression defects are defects that are introduced when a change is made to the application. It is common for testers to not only test their new feature but also features that existed beforehand in order to verify that previously implemented features still function as expected.
With unit testing, it’s possible to rerun your entire suite of tests after every build or even after you change a line of code. Giving you confidence that your new code does not break existing functionality.
Executable documentation:
It may not always be obvious what a particular method does or how it behaves given a certain input. You may ask yourself: How does this method behave if I pass it a blank string? Null?
When you have a suite of well-named unit tests, each test should be able to clearly explain the expected output for a given input. In addition, it should be able to verify that it actually works.
Less coupled code:
When code is tightly coupled, it can be difficult to unit test. Without creating unit tests for the code that you’re writing, coupling may be less apparent.
Writing tests for your code will naturally decouple your code, because it would be more difficult to test otherwise.
Characteristics of a good unit test
Fast. It is not uncommon for mature projects to have thousands of unit tests. Unit tests should take very little time to run. Milliseconds.
Isolated. Unit tests are standalone, can be run in isolation, and have no dependencies on any outside factors such as a file system or database.
Repeatable. Running a unit test should be consistent with its results, that is, it always returns the same result if you do not change anything in between runs.
Self-Checking. The test should be able to automatically detect if it passed or failed without any human interaction.
Timely. A unit test should not take a disproportionately long time to write compared to the code being tested. If you find testing the code taking a large amount of time compared to writing the code, consider a design that is more testable.
Fake
A fake is a generic term which can be used to describe either a stub or a mock object. Whether it is a stub or a mock depends on the context in which it’s used. So in other words, a fake can be a stub or a mock.
The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub.
https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-best-practices
Mock - a type of Fake object
A mock object is a fake object in the system that decides whether or not a unit test has passed or failed. A mock starts out as a Fake until it is asserted against.
Stub - a type of Fake object
A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly. By default, a fake starts out as a stub.
Wrong usage of stub as Mock
var mockOrder = new MockOrder(); var purchase = new Purchase(mockOrder);
purchase.ValidateOrders();
Assert.True(purchase.CanBeShipped);
This would be an example of stub being referred to as a mock. In this case, it is a stub. You’re just passing in the Order as a means to be able to instantiate Purchase (the system under test). The name MockOrder is also very misleading because again, the order is not a mock.
Stub example
var stubOrder = new FakeOrder(); var purchase = new Purchase(stubOrder);
purchase.ValidateOrders();
Assert.True(purchase.CanBeShipped); By renaming the class to FakeOrder, you've made the class a lot more generic, the class can be used as a mock or a stub. Whichever is better for the test case. In the above example, FakeOrder is used as a stub. You're not using the FakeOrder in any shape or form during the assert. FakeOrder was just passed into the Purchase class to satisfy the requirements of the constructor.
Mock example
var mockOrder = new FakeOrder(); var purchase = new Purchase(mockOrder);
purchase.ValidateOrders();
Assert.True(mockOrder.Validated);
In this case, you are checking a property on the Fake (asserting against it), so in the above code snippet, the mockOrder is a Mock.
It’s important to get this terminology correct. If you call your stubs “mocks”, other developers are going to make false assumptions about your intent.
mocks versus stubs
The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub.
Best Practices:Naming your tests
The name of your test should consist of three parts:
1:The name of the method being tested.
2:The scenario under which it’s being tested.
3:The expected behavior when the scenario is invoked.
Why?
Naming standards are important because they explicitly express the intent of the test.
Tests are more than just making sure your code works, they also provide documentation. Just by looking at the suite of unit tests, you should be able to infer the behavior of your code without even looking at the code itself. Additionally, when tests fail, you can see exactly which scenarios do not meet your expectations.
Bad:
C#
Copy
[Fact]
public void Test_Single()
{
var stringCalculator = new StringCalculator();
var actual = stringCalculator.Add("0");
Assert.Equal(0, actual); } ------ Better: C#
Copy
[Fact]
public void Add_SingleNumber_ReturnsSameNumber()
{
var stringCalculator = new StringCalculator();
var actual = stringCalculator.Add("0");
Assert.Equal(0, actual); } Arranging your tests Arrange, Act, Assert is a common pattern when unit testing. As the name implies, it consists of three main actions:
Arrange your objects, creating and setting them up as necessary.
Act on an object.
Assert that something is as expected.
Why?
Clearly separates what is being tested from the arrange and assert steps.
Less chance to intermix assertions with “Act” code.
Readability is one of the most important aspects when writing a test. Separating each of these actions within the test clearly highlight the dependencies required to call your code, how your code is being called, and what you are trying to assert. While it may be possible to combine some steps and reduce the size of your test, the primary goal is to make the test as readable as possible.
Bad:
C#
Copy
[Fact]
public void Add_EmptyString_ReturnsZero()
{
// Arrange
var stringCalculator = new StringCalculator();
// Assert
Assert.Equal(0, stringCalculator.Add(""));
}
Better:
C#
Copy
[Fact]
public void Add_EmptyString_ReturnsZero()
{
// Arrange
var stringCalculator = new StringCalculator();
// Act
var actual = stringCalculator.Add("");
// Assert
Assert.Equal(0, actual);
}
Best Practices:Arranging your tests -Arrange, Act, Assert
Arrange, Act, Assert is a common pattern when unit testing. As the name implies, it consists of three main actions:
Arrange your objects, creating and setting them up as necessary.
Act on an object.
Assert that something is as expected.
Why?
Clearly separates what is being tested from the arrange and assert steps.
Less chance to intermix assertions with “Act” code.
Readability is one of the most important aspects when writing a test. Separating each of these actions within the test clearly highlight the dependencies required to call your code, how your code is being called, and what you are trying to assert. While it may be possible to combine some steps and reduce the size of your test, the primary goal is to make the test as readable as possible.
Bad:
C#
Copy
[Fact]
public void Add_EmptyString_ReturnsZero()
{
// Arrange
var stringCalculator = new StringCalculator();
// Assert
Assert.Equal(0, stringCalculator.Add(""));
}
Better:
C#
Copy
[Fact]
public void Add_EmptyString_ReturnsZero()
{
// Arrange
var stringCalculator = new StringCalculator();
// Act
var actual = stringCalculator.Add("");
// Assert
Assert.Equal(0, actual);
}
Best Practices:Write minimally passing tests
The input to be used in a unit test should be the simplest possible in order to verify the behavior that you are currently testing.
Why?
Tests become more resilient to future changes in the codebase.
Closer to testing behavior over implementation.
Tests that include more information than required to pass the test have a higher chance of introducing errors into the test and can make the intent of the test less clear. When writing tests you want to focus on the behavior. Setting extra properties on models or using non-zero values when not required, only detracts from what you are trying to prove.
Bad:
C#
Copy
[Fact]
public void Add_SingleNumber_ReturnsSameNumber()
{
var stringCalculator = new StringCalculator();
var actual = stringCalculator.Add("42");
Assert.Equal(42, actual); } Better: C#
Copy
[Fact]
public void Add_SingleNumber_ReturnsSameNumber()
{
var stringCalculator = new StringCalculator();
var actual = stringCalculator.Add("0");
Assert.Equal(0, actual); }
Best Practices:Avoid magic strings
Naming variables in unit tests is as important, if not more important, than naming variables in production code. Unit tests should not contain magic strings.
Why?
Prevents the need for the reader of the test to inspect the production code in order to figure out what makes the value special.
Explicitly shows what you’re trying to prove rather than trying to accomplish.
Magic strings can cause confusion to the reader of your tests. If a string looks out of the ordinary, they may wonder why a certain value was chosen for a parameter or return value. This may lead them to take a closer look at the implementation details, rather than focus on the test.
Tip
When writing tests, you should aim to express as much intent as possible. In the case of magic strings, a good approach is to assign these values to constants.
Bad:
C#
Copy
[Fact]
public void Add_BigNumber_ThrowsException()
{
var stringCalculator = new StringCalculator();
Action actual = () => stringCalculator.Add("1001");
Assert.Throws(actual); } Better: C#
Copy
[Fact]
void Add_MaximumSumResult_ThrowsOverflowException()
{
var stringCalculator = new StringCalculator();
const string MAXIMUM_RESULT = “1001”;
Action actual = () => stringCalculator.Add(MAXIMUM_RESULT); Assert.Throws(actual); }
Best Practices:Avoid logic in tests
When writing your unit tests avoid manual string concatenation and logical conditions such as if, while, for, switch, etc.
Why?
Less chance to introduce a bug inside of your tests.
Focus on the end result, rather than implementation details.
When you introduce logic into your test suite, the chance of introducing a bug into it increases dramatically. The last place that you want to find a bug is within your test suite. You should have a high level of confidence that your tests work, otherwise, you will not trust them. Tests that you do not trust, do not provide any value. When a test fails, you want to have a sense that something is actually wrong with your code and that it cannot be ignored.
Tip
If logic in your test seems unavoidable, consider splitting the test up into two or more different tests.
Bad:
C#
Copy
[Fact]
public void Add_MultipleNumbers_ReturnsCorrectResults()
{
var stringCalculator = new StringCalculator();
var expected = 0;
var testCases = new[]
{
"0,0,0",
"0,1,2",
"1,2,3"
};
foreach (var test in testCases)
{
Assert.Equal(expected, stringCalculator.Add(test));
expected += 3;
}
}
Better:
C#
Copy
[Theory]
[InlineData("0,0,0", 0)]
[InlineData("0,1,2", 3)]
[InlineData("1,2,3", 6)]
public void Add_MultipleNumbers_ReturnsSumOfNumbers(string input, int expected)
{
var stringCalculator = new StringCalculator();
var actual = stringCalculator.Add(input); Assert.Equal(expected, actual); }
Best Practices:Prefer helper methods to setup and teardown
If you require a similar object or state for your tests, prefer a helper method than leveraging Setup and Teardown attributes if they exist.
Why?
Less confusion when reading the tests since all of the code is visible from within each test.
Less chance of setting up too much or too little for the given test.
Less chance of sharing state between tests which creates unwanted dependencies between them.
In unit testing frameworks, Setup is called before each and every unit test within your test suite. While some may see this as a useful tool, it generally ends up leading to bloated and hard to read tests. Each test will generally have different requirements in order to get the test up and running. Unfortunately, Setup forces you to use the exact same requirements for each test.
Note
xUnit has removed both SetUp and TearDown as of version 2.x
Bad:
C#
Copy
private readonly StringCalculator stringCalculator;
public StringCalculatorTests()
{
stringCalculator = new StringCalculator();
}
C#
Copy
// more tests…
C#
Copy
[Fact]
public void Add_TwoNumbers_ReturnsSumOfNumbers()
{
var result = stringCalculator.Add(“0,1”);
Assert.Equal(1, result); } ---- Better: C#
Copy
[Fact]
public void Add_TwoNumbers_ReturnsSumOfNumbers()
{
var stringCalculator = CreateDefaultStringCalculator();
var actual = stringCalculator.Add("0,1");
Assert.Equal(1, actual); } C#
Copy
// more tests…
C#
Copy
private StringCalculator CreateDefaultStringCalculator()
{
return new StringCalculator();
}
Best Practices:Avoid multiple asserts
When writing your tests, try to only include one Assert per test. Common approaches to using only one assert include:
Create a separate test for each assert.
Use parameterized tests.
Why?
If one Assert fails, the subsequent Asserts will not be evaluated.
Ensures you are not asserting multiple cases in your tests.
Gives you the entire picture as to why your tests are failing.
When introducing multiple asserts into a test case, it is not guaranteed that all of the asserts will be executed. In most unit testing frameworks, once an assertion fails in a unit test, the proceeding tests are automatically considered to be failing. This can be confusing as functionality that is actually working, will be shown as failing.
Note
A common exception to this rule is when asserting against an object. In this case, it is generally acceptable to have multiple asserts against each property to ensure the object is in the state that you expect it to be in.
Bad:
C#
Copy
[Fact]
public void Add_EdgeCases_ThrowsArgumentExceptions()
{
Assert.Throws(() => stringCalculator.Add(null));
Assert.Throws(() => stringCalculator.Add(“a”));
}
Better:
C#
Copy
[Theory]
[InlineData(null)]
[InlineData(“a”)]
public void Add_InputNullOrAlphabetic_ThrowsArgumentException(string input)
{
var stringCalculator = new StringCalculator();
Action actual = () => stringCalculator.Add(input); Assert.Throws(actual); }
