Week 8 - Testing in Production and DevOps Flashcards
Name 4 approaches to testing in production.
Blue/green testing
Canary testing
Feature testing
Resilience testing
What is blue/green testing?
The production environment (blue) is exchanged with the staging environment (green) - this may be done by a routing table
If (after some minutes) customers are unhappy, the exchange is reversed; otherwise, it is made permanent
What is canary testing?
A small percentage of the customer traffic is sent to a new version in the production environment - this may be done by tinkering with a load balancer
If (after some minutes) customers are unhappy, all customer traffic is sent to the old version in the production environment
What is A/B feature testing?
A small percentage of the customer traffic is sent to a new version in the production environment
If (after some measurements) customers are unhappy, all customer traffic is routed to the old version in the production environment
What is resilience testing?
Random virtual machines or containers in the production environment are terminated
If the system is tolerant of failure, it will recover by restarting the virtual machine or container
Name 3 players in the Netflix Simian Army and state what they do.
The Chaos Monkey - terminates random virtual machines to see how well automatic recovery mechanisms work. The environment is carefully monitored, with engineers standing by to address any problems
The Chaos Gorilla - terminates (links to) entire availability zones. Services should automatically switch to alternate zones without user-visible impact or manual intervention
The Latency Monkey - delays client-server communication to simulate service degradation and measures if upstream services response appropriately. Very large delay can simulate a node or service downtime
What are the two goals of enterprise computing and who is responsible for them?
- Provide to a rapidly changing competitive landscape
- Provide a stable, reliable and secure service
Responsible for them is: - Development (Dev)
- Operations (Ops)
What do DevOps do?
Sees Development and Operations teams work together to deploy changes hundreds or even thousands of times per day
What does the CALMS acronym for DevOps stand for?
Culture
Automation
Lean
Measurement
Sharing
Why is culture important for DevOps?
DevOps involves changing culture, so that development, quality control, security and operations people are “all on the same team”, working together to discover, triage and fix problems. This new culture is one where mistakes are allowed as long as lessons are learned from them
Why is automation important for DevOps?
DevOps involves automating manual tasks to eliminate toil, defined as work that “tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows”. Extensive use of tooling eliminates this toil, giving consistency, predictability and transparency
Why is being lean important for DevOps?
DevOps involves keeping everything minimal, but useful, so that batches of improvements are small, but worthwhile, and their deployment to production is frequent. The idea is to give the user a perhaps imperfect product today, rather than a perfect one in six month’s time
Why is measurement important for DevOps?
DevOps involves obsessively monitoring metrics in real
time, so that problems become apparent immediately and can be fixed quickly. This monitoring may be of development metrics, such was work-in-progress or the burn-down rate, or of operations metrics, such as the response time or incident rate
Why is sharing important for DevOps?
DevOps involves sharing goals, priorities and
responsibilities, so creating a common sense of purpose. This sharing is managed through the creation of frictionless information feedback loops from development to operations, and from operations to development
How is the DevOps lifestyle usually described?
As an infinite loop to symbolise the need for constant collaboration and iterative improvement
