Week 5

Question 1

What is the tradition method of product delivery?

Answer 1

Traditional product delivery (over years/months) uses 4 release:
Alpha
Beta
Release candidate
Release

Question 2

What is the alpha release?

Answer 2

First release for sophisticated users, contains bugs but is for gathering feedback. Release for limited users.

Question 3

What is the beta release?

Answer 3

Follows the alpha release, more stable but still requires feedback and contains some bugs.

Question 4

What is a release candidate?

Answer 4

Comes after the beta release, it’s what the company plans to release, and is a final chance to make changes, if no bugs are found this is what will be released. Should be fairly stable. No new features.

Question 5

When software was distributed on CDs why was it important that it was thoroughly tested before release?

Answer 5

Expensive and difficult to fix bugs, if you have a warehouse of 50000 CDs to update them all. Once the software is distributed to users it is far harder to update and fix their system when it is on a CD compared to digital download.

Question 6

Issues with traditional software delivery

Answer 6

Adding features is hard the further down the cycle you go,
Fixing bugs is hard the further down the cycle you go
Very slow
Stressful for developers, working to time frames

Question 7

What is Modern Feature Delivery?

Answer 7

Works within weeks rather than months
MVP moves between three environments:
Development
Staging
Production

Question 8

What is the development environment?

Answer 8

The development environment you share with your scrum team, where the features are built, adds changes, does testing. At the end of the sprints, they are sent to staging.

Question 9

What is the staging environment?

Answer 9

Staging is where the feature is set up in a mirror of the production environment.
Looks like production, we can use it to test on various platforms like Linux, etc. Tease out any bugs by moving environments, before we deploy it to customers using different OS. Perform more testing.

Question 10

What is production?

Answer 10

Production is where the program runs and is operated by users in the real-world. If bugs are found here they are by real users affecting their real work. Failures are public.

Question 11

Why is better to get your product to production quicker?

Answer 11

Get feedback faster, roll out updates quicker, stay on top of competitors.

Question 12

What are the principles of software delivery?

Answer 12

1. create a repeatable process
2. automate almost everything
3. version control for everything
4. if it hurts, do it more frequently
5. build quality in
6. done means released
7. everyone is responsible
8. continuous improvement

Question 13

What is ‘create a repeatable process?’

Answer 13

Releasing software should be easy, because it has been done thousands of times before. (minimize mistakes, faster, less stress)

Question 14

What is ‘automate almost everything?’

Answer 14

The build should be push button operation. Scripts don’t make mistakes. (humans make mistakes)

Question 15

What is ‘version control for everything?’

Answer 15

Everything you need to build, deploy, test and release your software should be kept in versioned storage. (helps you collaborate with team and back up code files)

Question 16

What is ‘if it hurts, do it more frequently?’

Answer 16

If a build is painful, do it on every check-in. (people get used to it, less work to do in one go, better to merge code monthly than every 4 years)

Question 17

What is ‘build quality in?’

Answer 17

Defects are fixed as soon as they are found. TEST EARLY AND AUTOMATICALLY. (fix it earlier up the delivery chain, cheaper and easier to fix)

Question 18

What is ‘done means released?’

Answer 18

A feature is only considered ‘done’ once it is in production. You don’t know what problems you will encounter or how long they will take to fix. By production these should be done.

Question 19

What is ‘everyone is responsible?’

Answer 19

At a glance, the status of the build is visible to everyone. Having multiple teams such as test and dev creates an adversarial relationship. (you can’t pass your problems to another team).

Question 20

What is ‘continuous improvement?’

Answer 20

Teams should reflect on what has gone well, what has gone badly, and discuss how to improve. Don’t witch hunt people who make errors, we learn the lesson as an organization. For example, testing should have picked up the error. People will become scared to innovate otherwise.

Question 21

What is the iterative/incremental model?

Answer 21

Exploration, development and operation at the same time.
Iterative because the feed-forward between activities is augmented with the feedback between them
Incremental because the interleaves activities regularly deliver small addition pieces of functionality.

Question 22

What is shift left testing?

Answer 22

Beginning testing as early as practical in the lifecycle. Traditionally in a waterfall cycle it happens at the end which can happen at the end of a number of years, really long time to fix and expensive. Stops us overrunning.

Question 23

What is the test pyramid?

Answer 23

Describes tests carried out in development
Unit
Service
End to end

Question 24

What are unit tests?

Answer 24

Before code is accepted into dev, unit tests are ran, there may be thousands of unit tests, performed in seconds using testing frameworks (such as JUnit)

Question 25

What are service tests?

Answer 25

Before code for a MICROSERVICE is accepted into staging, service tests are run. There may be hundreds of them, performed in a few minutes, stubbing out collaborating services. Done in isolation. Test just your service (fake its results).

Question 26

What are end-to-end tests?

Answer 26

Before microservices are accepted into production environment, end-to-end tests are run. There may be tens of them, performed in several minutes mimicking user interaction through a GUI, testing interaction with over services.

Question 27

What is the test snow cone anti-pattern? (things you shouldnt do)

Answer 27

Test snow cone, where all test coverage is in end-to-end tests. Leads to slow test runs and long feedback cycles. (more tests should be in unit tests testing erroneous errors etc)

Question 28

What 2 things may a service test be implemented using?

Answer 28

stubbing,
mocking

(we want to test only our service, not others)

Question 29

What is stubbing?

Answer 29

A stub is a dummy service, calls to which are not verified as part of the test (pretend the output was 709 for example). Its consistent, fast, simple, and allows you to test before devs have finished their services. if your service fails its your problem.

Question 30

What is mocking?

Answer 30

A mock is a dummy service, calls to which are verified as part of the test. It makes a call to a dummy service, and returns the same result i.e. 709. It tests we can make http calls with the same protocol as well, which we dont do in stubbing.

Question 31

What 2 things might an end-to-end test implementation be?

Answer 31

brittle,
flaky

Question 32

What is a brittle end-to-end test?

Answer 32

A brittle test is one that fails because of another service fails.
e.g. in a mortgage system one service fails because the database call failed, but we see it as a failure of the first service.

Question 33

What is a flaky end-to-end test?

Answer 33

A flaky test is one that sometimes fails because another service fails, perhaps due to a time-out or race condition. (much harder to detect and replicate, and fix (its flaky))

Question 34

What 5 cautions may we have about end-to-end tests?

Answer 34

1. individual team tests
2. dedicated testing teams
3. the normalisation of deviance
4. the time to run
5. the lack of independent testability

Question 35

What is the ‘individual team test’ caution in end to end testing?

Answer 35

end to end tests may be created by every individual team, leading to slow runs and a lack of ownership.

Question 36

What is the ‘dedicated testing teams’ caution in end to end testing?

Answer 36

They can lead to bottlenecks and adversarial friction. They are a team in the company that are often disliked.

Question 37

What is the ‘normalisation of deviance’ caution in end to end testing?

Answer 37

Flaky tests lead to the normalisation of deviance, devs lose faith in a test suit that ‘always fails like that’. We need to solve flaky tests asap, not just allow them to fail every now and then.

Question 38

What is the ‘time to run’ caution in end-to-end testing?

Answer 38

End to end tests may steadily pile up, increasing the time to run, perhaps to days, as teams take longer to finish their features, and requesting new end-to-end tests.

Question 39

What is ‘the lack of independent testability’ caution in end to end testing?

Answer 39

End to end tests may require teams to coordinate with each other or to wait for each other.