Week 5 - Software Delivery and Testing Flashcards
What are the four releases of traditional product delivery?
The alpha release
The beta release
The release candidate
The release
What are the three environments used by modern feature delivery?
The development environment
The staging environment
The production environment
What does the development environment do?
The development environment is the shared server where the feature is developed
What does the staging environment do?
The staging environment is where the feature is set up in a mirror of production environment
What does the production environment do?
The production environment is where the feature becomes available to the world.
What are the 8 Principles of Software Delivery
- Create a repeatable process
- Automate almost everything
- Version control for everything
- If it hurts, do it more frequently
- Build quality in
- Done means released
- Everyone is responsible
- Continuous improvement
What does the “create a repeatable process” software delivery principle mean?
Releasing software should be easy, because it has been done a thousand times before
What does the “automate almost everything” software delivery principle mean?
The build should be a push button operation
What does the “version control for everything” software delivery principle mean?
Everything you need to build, deploy, test and release your software should be kept in versioned storage
What does the “if it hurts, do it more frequently” software delivery principle mean?
If the build is painful, do it on every check-in
What does the “build quality in” software delivery principle mean?
Defects are fixed as soon as they are found
What does the “done means released” software delivery principle mean?
A feature is only considered “done” once it is in production
What does the “everyone is responsible” software delivery principle mean?
At a glance, the status of the build is visible to everyone
What does the “continuous improvement” software delivery principle mean?
Teams should reflect on what has gone well, what has gone badly, and discuss how to improve (retrospectives)
What is shift left testing?
Beginning testing as early as practical in the lifecycle
What does the test pyramid describe?
Tests carried out in development
What are the three points of the test pyramid?
Unit tests, service tests, end-to-end tests
What is a unit test and how and when are they run?
Unit tests are tests on functions in code
Before the code for a function is accepted into the development environment, its unit tests are run to test it works properly
There might be thousands of unit tests, performed in seconds by testing frameworks
What is a service test and how and when are they run?
They test if services work
Before the code for an individual microservice is accepted into the staging environment, its service tests are run to test it works properly
Overall, there might be hundreds of service tests, performed in a few minutes by scripts, stubbing out collaborating services
What are end-to-end tests and how and when are they run?
Tests of the full customer experience
Before the code for a system of microservices is accepted into the production environment, its end-to-end tests are run to test it works properly
Overall, there may be tens of end-to-end tests, performed in several minutes by mimicking user interaction, often through a GUI
What is a common anti-pattern to the testing pyramid?
The test snow cone
What is the test snow cone?
All coverage is end-to-end tests
What does the test snow cone lead to?
Slow test runs and long feedback cycles
What is an anti-pattern?
Something you shouldn’t do (the opposite of a design pattern)
The term “integration test” should be avoided. What does it commonly mean?
Some modules will work together properly OR an entire system of modules will work together properly
What are the two ways to implement a service test?
Stubbing and mocking
Describe stubbing.
A stub is a dummy services, calls to which are not verified as part of the test
Describe mocking.
A mock is a dummy service, calls to which are verified as part of the test
What two things can an end-to-end test be?
Brittle and flaky
What does it mean if an end-to-end test is brittle?
A brittle test is one that fails because another service fails
What does it mean if an end-to-end test is flaky?
A flaky test is one that sometimes fails because another service fails, perhaps due to a time-out or race condition
Name 5 cautions about end-to-end tests.
Individual team tests
Dedicated testing teams
The normalisation of deviance
The time to run
The lack of independent testability
Why is individual team tests a caution when it comes to end-to-end testing?
End-to-end tests may be created by every individual team, leading to slow runs and a lack of ownership
Why is having dedicated testing teams a caution when it comes to end-to-end testing?
End-to-end tests may be created by a dedicated testing team, leading to bottlenecks and adversarial friction
Why is the normalisation of deviance a caution when it comes to end-to-end testing?
Flaky end-to-end tests lead to the normalisation of deviance - developers lose faith in a test suite that “always fails like that”
Why is the time to run a caution when it comes to end-to-end testing?
End-to-end tests may steadily pile up, increasing the time to run (maybe even to days)
Why is the lack of independent testability a caution when it comes to end-to-end testing?
End-to-end tests may require teams to coordinate with each other or wait for each other
What is the key to effective testing?
Automation
