Week 5 - Software Delivery and Testing

Question 1

What are the four releases of traditional product delivery?

Answer 1

The alpha release

The beta release

The release candidate

The release

Question 2

What are the three environments used by modern feature delivery?

Answer 2

The development environment

The staging environment

The production environment

Question 3

What does the development environment do?

Answer 3

The development environment is the shared server where the feature is developed

Question 4

What does the staging environment do?

Answer 4

The staging environment is where the feature is set up in a mirror of production environment

Question 5

What does the production environment do?

Answer 5

The production environment is where the feature becomes available to the world.

Question 6

What are the 8 Principles of Software Delivery

Answer 6

1. Create a repeatable process
2. Automate almost everything
3. Version control for everything
4. If it hurts, do it more frequently
5. Build quality in
6. Done means released
7. Everyone is responsible
8. Continuous improvement

Question 7

What does the “create a repeatable process” software delivery principle mean?

Answer 7

Releasing software should be easy, because it has been done a thousand times before

Question 8

What does the “automate almost everything” software delivery principle mean?

Answer 8

The build should be a push button operation

Question 9

What does the “version control for everything” software delivery principle mean?

Answer 9

Everything you need to build, deploy, test and release your software should be kept in versioned storage

Question 10

What does the “if it hurts, do it more frequently” software delivery principle mean?

Answer 10

If the build is painful, do it on every check-in

Question 11

What does the “build quality in” software delivery principle mean?

Answer 11

Defects are fixed as soon as they are found

Question 12

What does the “done means released” software delivery principle mean?

Answer 12

A feature is only considered “done” once it is in production

Question 13

What does the “everyone is responsible” software delivery principle mean?

Answer 13

At a glance, the status of the build is visible to everyone

Question 14

What does the “continuous improvement” software delivery principle mean?

Answer 14

Teams should reflect on what has gone well, what has gone badly, and discuss how to improve (retrospectives)

Question 15

What is shift left testing?

Answer 15

Beginning testing as early as practical in the lifecycle

Question 16

What does the test pyramid describe?

Answer 16

Tests carried out in development

Question 17

What are the three points of the test pyramid?

Answer 17

Unit tests, service tests, end-to-end tests

Question 18

What is a unit test and how and when are they run?

Answer 18

Unit tests are tests on functions in code

Before the code for a function is accepted into the development environment, its unit tests are run to test it works properly

There might be thousands of unit tests, performed in seconds by testing frameworks

Question 19

What is a service test and how and when are they run?

Answer 19

They test if services work

Before the code for an individual microservice is accepted into the staging environment, its service tests are run to test it works properly

Overall, there might be hundreds of service tests, performed in a few minutes by scripts, stubbing out collaborating services

Question 20

What are end-to-end tests and how and when are they run?

Answer 20

Tests of the full customer experience

Before the code for a system of microservices is accepted into the production environment, its end-to-end tests are run to test it works properly

Overall, there may be tens of end-to-end tests, performed in several minutes by mimicking user interaction, often through a GUI

Question 21

What is a common anti-pattern to the testing pyramid?

Answer 21

The test snow cone

Question 22

What is the test snow cone?

Answer 22

All coverage is end-to-end tests

Question 23

What does the test snow cone lead to?

Answer 23

Slow test runs and long feedback cycles

Question 24

What is an anti-pattern?

Answer 24

Something you shouldn’t do (the opposite of a design pattern)

Question 25

The term “integration test” should be avoided. What does it commonly mean?

Answer 25

Some modules will work together properly OR an entire system of modules will work together properly

Question 26

What are the two ways to implement a service test?

Answer 26

Stubbing and mocking

Question 27

Describe stubbing.

Answer 27

A stub is a dummy services, calls to which are not verified as part of the test

Question 28

Describe mocking.

Answer 28

A mock is a dummy service, calls to which are verified as part of the test

Question 29

What two things can an end-to-end test be?

Answer 29

Brittle and flaky

Question 30

What does it mean if an end-to-end test is brittle?

Answer 30

A brittle test is one that fails because another service fails

Question 31

What does it mean if an end-to-end test is flaky?

Answer 31

A flaky test is one that sometimes fails because another service fails, perhaps due to a time-out or race condition

Question 32

Name 5 cautions about end-to-end tests.

Answer 32

Individual team tests
Dedicated testing teams
The normalisation of deviance
The time to run
The lack of independent testability

Question 33

Why is individual team tests a caution when it comes to end-to-end testing?

Answer 33

End-to-end tests may be created by every individual team, leading to slow runs and a lack of ownership

Question 34

Why is having dedicated testing teams a caution when it comes to end-to-end testing?

Answer 34

End-to-end tests may be created by a dedicated testing team, leading to bottlenecks and adversarial friction

Question 35

Why is the normalisation of deviance a caution when it comes to end-to-end testing?

Answer 35

Flaky end-to-end tests lead to the normalisation of deviance - developers lose faith in a test suite that “always fails like that”

Question 36

Why is the time to run a caution when it comes to end-to-end testing?

Answer 36

End-to-end tests may steadily pile up, increasing the time to run (maybe even to days)

Question 37

Why is the lack of independent testability a caution when it comes to end-to-end testing?

Answer 37

End-to-end tests may require teams to coordinate with each other or wait for each other

Question 38

What is the key to effective testing?

Answer 38

Automation