Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Learning > AWS: CDA > Flashcards

AWS: CDA Flashcards

1
Q

What is Elastic Beanstalk?

A
  • Developer-centric view of deploying an app on AWS
  • Fully managed service
    • Handles capacity provisioning, load balancing, scaling, monitoring
  • Free service but underlying AWS resources will have costs involved
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the components involved in Elastic Beanstalk?

A
  • Application
    • Collection of components
  • Application version
    • Iteration of app code
  • Environment
    • Collection of AWS resources running an app version
    • Env tiers to support diff types of apps
    • Multiple env can be created ie. dev/prod
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the use cases for the different environment tiers?

A
  • Web server environment tier
    • Website
    • Web app
    • Web app serving HTTP requests
  • Worker environment tier
    • Processing long-running workloads on demand
    • Perform tasks on a schedule
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

List the different deployment options available for Elastic Beanstalk

A
  • All-at-once deployment
    • Fastest
    • Instances are down during update
    • No additional cost
  • Rolling deployment
    • Zero downtime
    • Deployment time depends on number of instances
    • No additional cost
  • Rolling deployment with additional batches
    • Zero downtime
    • Deployment time depends on number of instances
    • Small additional cost
  • Immutable deployment
    • Zero downtime
    • Longest deployment
    • High cost, double capacity
    • Quick rollback in case of failure
  • Blue/green deployment
    • Zero downtime
  • Traffic-splitting deployment
    • Zero downtime
    • Quick rollback in case of failure
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How do rolling deployments work?

A
  • Apps running below capacity with set bucket size
  • Instances in the bucket will be down during update
  • Once instances in the bucket are updated, process repeats for next bucket (batch of instances)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

How does rolling deployments with additional batches work?

Deployments

A
  • App running at capacity with set bucket size
  • New instances created with upgraded version
  • Existing instances go through rolling deployments
  • After rolling deployment is complete, the newly-created instances will be terminated
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How does blue/green deployments work?

A
  • Create a new environment (green) where new version is deployed
  • Easy rollback to old environment (blue) if issues arise
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

How do traffic-splitting deployments work?

Deployments

A
  • Used for canary testing
  • New app version deployed to temp ASG with same capacity
  • Small % of traffic sent to temp ASG for a configurable amount of time
  • New instances migrated from temp to original ASG and then old version is terminated
  • Automated rollback if issues arise
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is the lifecycle policy for Elastic Beanstalk?

A
  • A configurable policy to limit no. of app versions to retain for future deployments
    • Limit by count
    • Limit by age
  • Must be enabled first to configure policy
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What happens under the hood for Elastic Beanstalk?

A
  • Relies on CloudFormation
    • CloudFormation is infra as code
    • Used to provision other AWS services
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is Elastic Beanstalk cloning?

A
  • Clone an environment with exact same configuration
    • All resources and config are preserved
  • After cloning an environment, you can modify settings
  • Useful for deploying a “test” version of your app
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is API Gateway?

A
  • Serverless service to manage and secure APIs
  • A single interface for all microservices
  • Use API endpoints with various resources
    • Apply forwarding and transformation rules at API Gateway level
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are some features of API Gateway?

Development

A
  • Support websocket protocols
  • Transform and validate requests/responses
  • Handle request throttling
  • Cache API responses
  • Handle API versioning
  • Handle different environments
  • Handle security
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are the different endpoint types for API Gateway?

A
  • Edge-optimised (default)
    • Requests routed through CloudFront Edge locations to improve latency
    • API Gateway still only lives in one region
  • Regional
    • For clients within same region
    • Could be manually combined with CloudFront
  • Private
    • Only accessed from your VPC using interface VPC endpoint
    • Use resource policy to define access
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are the user authentication strategies available for API Gateway?

A
  • IAM roles
    • Useful for internal applications
  • AWS Cognito
    • Useful for external users
  • Custom authoriser (your own logic via Lambda function)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How can you have security with your own custom domain name by integrating API Gateway with ACM?

A
  • If using edge-optimised endpoint, certificate must be in us-east-1
  • If using regional endpoint, certificate must be in API Gateway region
  • Must setup CNAME or A-alias record in Route 53
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What are stage variables in API Gateway?

Development

A
  • Similar to environment variables
  • Used to update frequently changing config values
  • If used in Lambda functions, they are passed to the “context” object
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are the use cases for stage variables in API Gateway?

A
  • They can be used in:
    • Lambda function ARN
    • HTTP endpoint
    • Parameter mapping templates
  • Use cases:
    • Configure HTTP endpoints that the stages talk to (dev, test, prod)
    • Pass config parameters to Lambda functions through mapping templates
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

How to perform canary deployments in API Gateway?

A
  • Usually done with prod
  • Choose % of traffic the canary channel receives
  • Metrics/logs are separate for better monitoring
  • Stage variables can be overridden for canary deployments
  • Once canary deployments have been tested and if all good, they can be promoted to entire stage
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are the different integration types for API Gateway?

A
  • Mock integration type
    • API Gateway returns a response without sending a request to backend
  • HTTP/AWS services
    • Both integration req and res must be configured
    • Setup data mapping using mapping templates for req and res
  • AWS proxy integration type
    • Incoming req from client is the input to Lambda
    • Lambda function is responsible for logic of req/res
    • No mapping template/headers/query params are passed as arguments
  • HTTP proxy integration type
    • HTTP req is passed to backend
    • HTTP res from backend is forwarded by API Gateway
    • No mapping templates
    • Optionally add HTTP headers if needed eg. API key
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What are mapping templates in API Gateway?

A
  • Templates used to modify req/res
    • Rename/modify query string params
    • Modify body content
    • Add headers
    • Filter result output
  • To set the template, the content-type must be set to either application/json or application/xml
  • Not used for proxy integration types
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

How can API Gateway utilise request validation?

A
  • Importing Open API definitions
  • The spec is used to verify if req corresponds to proper schema before proceeding with req
    • If validation fails, API Gateway immediately fails req
    • Reduces unnecessary calls to backend
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

How does caching work in API Gateway?

A
  • Caching reduces number of calls to backend
    • API Gateway will first check cache
    • If cache miss, call backend
  • Default TTL is 300s
  • Cache is expensive - makes sense for prod but may not make sense for dev/test
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is the difference between latency vs integration latency in API Gateway?

A
  • Integration latency
    • Time between when API Gateway relays req to backend and receives a response from backend
  • Latency
    • Time between when API Gateway receives req from client and when it returns response to client
    • Includes integration latency and other API Gateway overhead
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What are WebSockets?

A
  • Two-way interactive communication between user’s browser and a server
  • Server can push information to client
  • Enables stateful application use cases
  • Often used for real-time apps
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

How does API Gateway handle WebSocket routing?

A
  • API Gateway uses a route key table that incoming JSON messages are evaluated against
    • If no routes, sent to $default
    • Route is then connected to the backend setup through API Gateway
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

What are the different security strategies for API Gateway?

A
  • IAM
    • Great for existing users/roles in AWS
    • Authentication via IAM
    • Authorisation via IAM policies
    • Can be combined with resource policies for cross-accounts
    • Leverages sig 4 where IAM creds are in headers
  • Custom authoriser
    • Great for 3rd party tokens
    • Authentication via 3rd party system but verified in Lambda
    • Authorisation via Lambda fn
    • Lambda must return IAM policy for user - result is cached
  • Cognito User Pool
    • Great for fully managed service
    • Tokens expire automatically
    • Authentication via Cognito User Pools
    • Authorisation via API Gateway methods
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What is DynamoDB?

A
  • Managed NoSQL database
  • Highly available with replication across multi AZ
  • Scales to massive workloads
  • Fast and consistent performance
  • Low cost and auto-scaling capabilities
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

What are features of NoSQL databases?

A
  • Non-relational databases
    • eg. MongoDB, DynamoDB
  • Distributed
    • Scale horizontally
  • Do not support query joins/aggregation computations
  • All data needed is presented in one row
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

Describe DynamoDB tables

A
  • Each table has a primary key
    • Must be decided at creation time
    • Non-null
  • Each item has attributes
    • Similar to columns but more powerful
    • Can be added over time - can be null at creation time
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

What are the different strategies of choosing a primary key for DynamoDB tables?

Development

A
  • Partition key (hash)
    • Unique for each key
    • Diverse so data is distributed
    • Example: “user_id” for “users” table
  • Partition key + sort key
    • Combination must be unique for each item
    • Data grouped by partition key
    • Example: “user_id” for partition key and “game_id” for “users_games” table
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

What happens when read and write throughout is exceeded for DynamoDB?

A
  • Table must have provisioned read/write capacity units
    • Can setup auto-scaling
  • Throughput can be exceeded temporarily using burst capacity
    • If burst capacity has been consumed, there will be a ProvisionThroughputExceededException
    • It’s then advised to do an exponential backoff retry or distribute partition keys
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

What is the difference between eventually consistent read vs strongly consistent read?

A
  • Eventually consistent read
    • Possibility of a lag where data has not been replicated but a read has been made
  • Strongly consistent read
    • Ensures no data staleness
    • Consumes twice the RCUs
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

What are operations for writing data to DynamoDB?

A
  • PutItem
    • Creates new item or replaces old item (same primary key)
    • Consumes WCUs
  • UpdateItem
    • Edit an existing item’s attributes or adds a new item if it doesn’t exist
    • Can be used to implement atomic counters
  • Conditional writes
    • Accepts a write/update/delete only if conditions are met
    • Helps with concurrent access to items
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

What are the operations for reading data from DynamoDB?

Development

A
  • GetItem
    • Read based on primary key
    • Eventually consistent read (default) and have option to use strongly consistent reads (more RCUs)
  • Query
    • Returns items based on KeyConditionExpression and FilterExpression
    • Ability to paginate results
  • Scan
    • Scans entire table and then filter data (inefficient)
    • Consumes a lot of RCUs
    • Use parallel scans for faster performance - consumes significantly higher RCUs
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

What are the operations for deleting items from DynamoDB?

A
  • DeleteItem
    • Deletes individual item
    • Ability to perform conditional delete
  • DeleteTable
    • Delete entire table and its contents
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

What are the benefits of batching operations in DynamoDB?

A
  • Reduce latency by reducing number of API calls
  • Operations are done in parallel for better performance
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

What are the operations for batching in DynamoDB?

A
  • BatchGetItem
    • Returns items from one or more tables
    • Items retrieved in parallel to reduce latency
  • BatchWriteItem
    • Can’t update items
  • UnprocessedItems
    • Failed write operations
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

What is PartiQL?

A
  • SQL-like syntax to manipulate DynamoDB tables
  • Run queries across multiple DynamoDB tables
  • Supports some (not all) SQL statements
    • INSERT
    • UPDATE
    • SELECT
    • DELETE
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

What is the difference between GSI vs LSI?

A
  • Global Secondary Index (GSI)
    • Query a specific index that spans all data in base table, across all partitions
    • Support eventual consistency only
    • Can be added/updated after table creation
    • Must provision RCUs and WCUs for index
    • Queries or scans on this index consume capacity units from the index, not from the base table
    • If writes are throttled, main table will also be throttled
  • Local Secondary Index (LSI)
    • Only added at table creation
    • Uses RCUs and WCUs of main table
    • No special throttling consideration
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

What is optimistic locking in DynamoDB?

A
  • Conditional writes
  • A strategy to ensure an item hasn’t changed before it is updated/deleted
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

What is DynamoDB Accelerator (DAX)?

A

Fully managed, highly available, in-memory cache for DynamoDB

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

What are the key features of DAX?

A
  • Microseconds latency for cached reads
  • Compatible with existing DynamoDB APIs
    • Doesn’t require any modifications to application logic
  • Solves the “hot key” problem (too many reads)
  • Secure
  • Multi AZ
    • Min 3 nodes recommended for prod
  • 5 mins TTL
  • Up to 10 nodes per cluster
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

What are DynamoDB streams?

A
  • Ordered stream of item-level modifications (create/update/delete) in a table
  • Streamed records can be:
    • Sent to Kinesis
    • Read by Lambda
  • Data retention up to 24 hours
  • Records are not retroactively populated after it is enabled
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
45
Q

What are some use cases for DynamoDB streams?

A
  • React to changes in real-time
  • Analytics
  • Implement cross-region replication
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
46
Q

How do DynamoDB streams work with Lambda?

A
  1. Define Event Source Mapping to poll from DynamoDB streams and receive records in batches
  2. Ensure Lambda function has appropriate permissions to read from stream
  3. Lambda function is invoked synchronously with batch of records
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
47
Q

How does the TTL feature work in DynamoDB?

A
  • Automatically delete items after an expiry timestamp
    • Expired items are deleted within 48 hrs
    • Deleted from both GSI and LSI
  • TTL attribute must be a “number” data type with Unix epoch timestamp value
  • Doesn’t consume any WCUs
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
48
Q

What are the use cases for enabling a TTL in DynamoDB?

A
  • Reduce stored data by keeping only current items
  • Adhere to regulatory obligations
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
49
Q

In DynamoDB CLI, what does —projection-expression flag do?

A

One or more attributes to retrieve

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
50
Q

In DynamoDB CLI, what does —filter-expression flag do?

A

Filter items before being returned

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
51
Q

How does the transactions feature work in DynamoDB?

A
  • Co-ordinated all-or-nothing operations
  • Provides ACID (atomicity, consistency, isolation, durability)
  • Read modes:
    • Eventual consistency
    • Strong consistency
    • Transactional consistency
  • Write modes:
    • Standard consistency
    • Transactional consistency
  • Consumes 2x WCUs and RCUs
    • Performs 2 operations for every item (prepare and commit)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
52
Q

What are the use cases for transactions?

A
  • Financial transactions
  • Managing orders
  • Multi-player games
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
53
Q

What is the difference between using DynamoDB as session state cache vs ElastiCache or EFS?

A
  • ElastiCache
    • ElastiCache is in-memory but DynamoDB is serverless
    • Both are key/value stores
    • DynamoDB has auto-scaling
  • EFS
    • Must be attached to EC2 instances as network drives
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
54
Q

What are the different write types of DynamoDB?

A
  • Concurrent writes
  • Conditional writes
  • Atomic writes
  • Batch writes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
55
Q

What are some AWS services that can be used to decouple applications?

A
  • SQS
  • SNS
  • Kinesis
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
56
Q

What is SQS?

A
  • Fully managed service that queues messages
  • Consist of:
    • Producer(s) - sends messages to the queue
    • Consumer(s) - polls and processes messages from the queue
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
57
Q

What are the different types of SQS queues?

A
  • Standard
  • FIFO
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
58
Q

What are key features of SQS?

A
  • Unlimited throughput
  • Unlimited no. of messages in the queue
  • Retention of messages
    • Default: 4 days
    • Max: 14 days
  • Low latency (<10 ms)
  • Message size limit of 256 KB/message
  • Can have duplicate messages - at least once delivery
  • Can have out of order messages - best effort ordering
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
59
Q

How does SQS produce messages?

A
  • Send messages using SDK
  • SendMessage API
  • Message persisted in SQS until a consumer deletes it which signifies that it has been processed
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
60
Q

How does SQS consume messages?

Development

A
  • Poll SQS for messages
    • Receive up to 10 messages at a time
  • Process the messages
  • Delete the message using SDK
    • DeleteMessage API
  • Scale consumers horizontally to improve throughout
    • Can have multiple consumers process messages in parallel
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
61
Q

How can SQS be used with ASG to increase throughput?

A
  • Have multiple EC2 instances in an ASG consuming SQS messages
  • SQS has built-in CloudWatch metrics that can trigger an alarm if messages go over a certain number
    • ApproximateNumberOfMessages
    • Alarm can trigger ASG to scale
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
62
Q

What are the security strategies for SQS?

A
  • Encryption
    • In-flight encryption using HTTPS API
    • At-rest encryption using KMS keys
    • Client-side encryption (client will need to handle encryption/decryption itself)
  • Access controls
    • IAM policies to regulate access to SQS API
  • SQS queue access policies
    • Resource policy (similar to S3 bucket policies)
    • Useful for cross-account access
    • Useful for other services to write to SQS
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
63
Q

What does message visibility timeout mean in SQS?

A
  • After message is polled by consumer, it becomes invisible to other consumers
    • Default 30s for messages to be processed
  • After message visibility timeout lapses, message is then visible again in SQS
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
64
Q

What are dead letter queues in SQS?

A
  • If consumer fails to process a message within visibility timeout, then message goes back into the queue
  • Threshold can be set to limit how many times a message can go back into the queue
  • After threshold (MaximumReceives) is exceeded, message sent to DLQ
  • DLQ must inherit its queue type
    • DLQ of FIFO queue must also be FIFO queue
    • DLQ of standard queue must also be standard queue
  • Useful for debugging
  • Set expiry time (14 days retention) to process messages before expiry
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
65
Q

What is the “re-drive to source” feature of DLQ?

A
  • Help consume messages in DLQ to understand what is wrong
    • Allow manual inspection and debugging
  • When code is fixed, we can re-send message back into source queue in batches to be reprocessed
    • No custom code needed
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
66
Q

What are delay queues in SQS?

A
  • Delays a message so consumers can’t receive it immediately
  • Default is 0s - message available immediately
    • Can be delayed up to 15 mins
  • Default can be overridden on send using DelaySeconds parameter
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
67
Q

What is long polling in SQS?

A
  • When a consumer requests messages from the queue, it can optionally wait for messages to arrive if there are none in the queue
    • Wait time can be 1-20s
  • Long polling decreases no. of API calls while increasing latency
  • Can be enabled at:
    • Queue level
    • API level using ReceiveMessageWaitTimeSeconds
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
68
Q

What is SQS Extended Client?

A
  • Java library
  • Used to send large messages (ie. 1GB) due to standard size is 256 KB
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
69
Q

How does SQS Extended Client work?

A
  1. Producer stores the large message in S3 bucket
  2. Producer sends metadata message to SQS which references the path to S3 bucket
  3. Consumer receives metadata message from SQS and uses it to retrieve large message from S3 bucket
70
Q

What are the key features of FIFO SQS queues?

A
  • Messages are processed in order by consumer
  • Exactly-once send capability by removing duplicates
  • Limited throughput
    • 300 messages without batching
    • 3000 messages with batching
71
Q

What are the ways FIFO queues handle message deduplication?

A
  • Deduplication interval is 5 mins
  • Two methods:
    • Content-based deduplication via hashing the message body
    • Explicitly provide Message Deduplication ID
72
Q

How does FIFO queues handle message grouping?

A
  • If same MessageGroupID is used in a FIFO queue, you can only have 1 consumer and all messages are in order
  • Ordering at the level of a subset of messages, specify different values for MessageGroupID
    • Each group ID can have a different consumer
    • Ordering across groups is not guaranteed
73
Q

What is SNS?

A
  • Simple Notification Service
  • Uses the pub/sub model
    • The “event producer” sends messages to SNS topic
    • “Event receivers” subscribe to SNS notifications
    • Each subscriber will get all the messages
74
Q

What are the key features of SNS?

A
  • Up to 12 million subscriptions per topic
  • 100k topics limit
  • Integrates with a lot of AWS services
75
Q

How to publish events using SNS?

A
  • Topic publish using the SDK
    • Create a topic
    • Create a subscription (or many subscriptions)
    • Publish to topic
  • Direct publish using mobile apps SDK
    • Create a platform app
    • Create platform endpoint
    • Publish to platform endpoint
76
Q

What are the security strategies for SNS?

A
  • Encryption
    • In-flight encryption using HTTP APIs
    • At-rest encryption using KMS keys
    • Client-side encryption (client will handle encryption/decryption itself)
  • Access controls
    • IAM policies to regulate access to SNS API
  • SNS access policies
    • Resource policy (similar to S3 bucket policies)
    • Useful for cross-account access
    • Useful for other services to write to SNS
77
Q

What is the fan out pattern involving SNS and SQS?

A
  • Push once in SNS and receive the event in multiple SQS queues that are subscribers
  • Fully decoupled with no data loss
78
Q

How does SNS handle message filtering?

A
  • JSON policy used to filter messages sent to SNS topic subscribers
  • If subscriber does not have a filter policy, it receives every message
79
Q

What is Kinesis?

A
  • Collect, process and analyse streaming data in real-time
  • Ingest real-time data eg. logs, metrics, IoT telemetry data
80
Q

What are the different data types for Kinesis?

A
  • Kinesis Data Streams
    • Capture, process and store data streams
  • Kinesis Data Firehose
    • Load data streams into AWS data stores
  • Kinesis Data Analytics
    • Analyse data streams with SQL or Apache Flink
  • Kinesis Video Streams
    • Capture, process and store video streams
81
Q

How do Kinesis Data Streams work?

A
  • Consist of multiple shards
    • Shards are numbered
    • Must be provisioned ahead of time
  • Producers produce records into Kinesis Data Stream
    • Producers can send data at a rate of 1MB/sec (1000 messages/sec/shard)
  • Kinesis Data Stream will send data via records to multiple consumers
82
Q

What are Kinesis producers?

A
  • Puts data records into data streams
    • PutRecord API
    • Use batching to reduce costs and increase throughput
  • Example producers:
    • AWS SDK
    • Kinesis Producer Library (KPL)
    • Kinesis Agent
83
Q

What are Kinesis consumers?

A
  • Get data records from data streams and process them
  • Example consumers:
    • AWS SDK
    • AWS Lambda
    • Kinesis Data Analytics
    • Kinesis Data Firehose
    • Kinesis Client Library (KCL)
84
Q

What do records consist of in Kinesis Data Streams?

A
  • Partition key
    • Determines which shard the record was from
  • Sequence number
    • Unique per partition key
    • Represents where the record was in the shard
  • Data blob
    • Value itself
    • Up to 1MB
85
Q

What are the two consumption modes for Kinesis Data Streams?

A
  • Shared (classic)
    • Consumers poll data from Kinesis
    • 2MB/sec/shard shared across all consumers
    • Latency ~200s
    • Inexpensive
  • Enhanced
    • Kinesis pushes data to consumers
    • 2MB/sec/shard per consumer
    • Latency ~70s
    • Expensive
86
Q

What are the key features of Kinesis Data Streams?

A
  • Retention between 1-365 days
  • Ability to reprocess (replay) data
  • Immutability
    • Once data is inserted in Kinesis, it can’t be deleted
  • Ordering
    • Data sharing the same partition goes to the same shard
87
Q

What are the capacity modes for Kinesis Data Streams?

A
  • Provisioned mode
    • Choose the number of shards provisioned
    • Pay per shard provisioned per hour
  • On-demand mode
    • Auto-scaling
    • Pay per stream per hour
88
Q

What are the security strategies for Kinesis Data Streams?

A
  • Control access/authorisation using IAM policies
  • Encryption
    • In-flight encryption using HTTPS APIs
    • At-rest encryption using KMS keys
    • Client-side encryption
  • VPC endpoints available for Kinesis to access within VPC without going through internet
89
Q

What is the ProvisionThroughputExceeded error and its solution?

A
  • Caused by over producing into a shard
    • Limit is 1MB/sec/shard (1000 records/sec)
  • Solutions:
    • Use a highly distributed partition key
    • Implement retries with exponential backoff
    • Increase shards (shard splitting)
90
Q

What is the Kinesis Client Library?

A
  • Java library that helps read records from Kinesis Data Streams with distributed applications sharing read workload
  • Each shard is read by 1 KCL instance
    • 1 shard = max 1 KCL instance
    • 5 shards = max 5 KCL instances
91
Q

How can we scale Kinesis?

A
  • Scale up via shard splitting
  • Scale down via shard merging
92
Q

What does shard splitting mean in Kinesis?

A
  • Shard split into 2 new shards and old shard is closed - old shard deleted once data expires
  • Used to increase stream capacity
  • Used to divide a “hot shard”
  • Increased cost
  • No automatic scaling
  • Shards can’t be split into more than 2 shards in a single operation
93
Q

What does shard merging mean in Kinesis?

A
  • Merge 2 shards with low traffic (aka “cold shards”) - new shard is created and old shard is closed with data deleted once it expires
  • Used to decrease stream capacity and save costs
  • More than 2 shards can’t be merged in a single operation
94
Q

What is Kinesis Data Firehose?

A
  • Fully managed, serverless service
  • Automatic scaling
  • Only pay for data going through Firehose
  • Near real-time
    • Data is sent in batches
    • Min 60s latency for non-full batches or min 1 MB at a time
  • Support data transformations using Lambda function
95
Q

What are the components that make up Kinesis Data Firehose?

A
  • Producers:
    • Kinesis Data Streams
    • Kinesis Agent
    • SDK, KPL
  • Optionally transform data via Lambda function
  • Optionally send failed writes or all data to a backup S3 bucket
  • Consumers (aka destinations):
    • 3rd party partner destinations
    • AWS destinations
      • S3
      • Redshift via copying through S3
      • OpenSearch
    • Custom destination (HTTP endpoint)
96
Q

What is the difference between Kinesis Data Streams vs Kinesis Data Firehose?

A
  • Kinesis Data Streams
    • Streaming service to ingest data at scale
    • Manage scaling
    • Write custom code for producers/consumers
    • Real-time
    • Data retention between 1-365 days
    • Supports replay capability
  • Kinesis Data Firehose
    • Streaming service to load data into destinations
    • Fully managed
    • Near real-time
    • Automated scaling
    • No data retention
    • No support for replay capability
97
Q

What is Kinesis Data Analytics?

A
  • Real-time analytics on Kinesis Data Streams and Firehose using SQL
  • Fully managed
  • Automated scaling
  • Pay for actual consumption rate
  • Use cases:
    • Time-series analytics
    • Real-time dashboard/metrics
98
Q

What is the difference between SQS vs SNS vs Kinesis?

A
  • SQS
    • Consumer pull data
    • Data deleted after being consumed
    • Message ordering only on FIFO queues
  • SNS
    • Uses pub/sub model
    • Pushes data to many subscribers where they each get a copy of the data
    • Data is not persisted (lost if not delivered)
  • Kinesis
    • Standard mode (pull data)
    • Enhanced mode (push data)
    • Message ordering at shard level
99
Q

What are the differences between EC2 vs Lambda?

A
  • EC2
    • Virtual servers in the cloud
    • Limited by RAM and CPU
    • Continuously running
    • Scaling requires intervention to add/remove servers
  • Lambda
    • Virtual functions - no servers to manage
    • Limited by time (max 15 mins)
    • Runs on-demand and billed only when invoked
    • Automatic scaling
100
Q

What are the requirements for running Lambda on a container?

A
  • Container image must implement Lambda Runtime API
  • ECS/Fargate is preferred for running Docker images
101
Q

Which services work synchronously with Lambda?

A
  • ALB
  • API Gateway
  • CloudFront (Lambda@Edge)
  • S3 batch
  • Cognito
  • Step functions
102
Q

Which services work asynchronously with Lambda?

A
  • S3
  • SNS
  • CloudWatch Events
  • CodeCommit
  • CodePipeline
103
Q

How does Lambda work with ALB?

A
  • Expose Lambda function as HTTP(s) endpoint
  • Lambda function must be registered in a target group
  • Payload converted from HTTP to JSON to be consumed by Lambda
  • Response converted from JSON to HTTP to be consumed by ALB
104
Q

How does Lambda work with ALB having multi-header values?

A
  • ALB support multi-header values which can be enabled from target group attribute settings
  • When enabled, HTTP headers and query string params that are sent with multiple values are shown as an array when passed to Lambda
105
Q

How does Lambda work with CloudWatch Events/EventBridge?

A
  • Using cronjob or Rate EventBridge rule
    • Triggered by a schedule for Lambda to perform a task
  • Using CodePipeline EventBridge rule
    • Triggered by state changes for Lambda to perform a task
106
Q

How does Lambda work with S3 event notifications?

A
  • S3 event notifications deliver events in seconds but can sometimes take longer
  • Enable versioning on the S3 bucket to ensure notifications are sent for every successful write
  • Examples:
    • S3:ObjectCreated
    • S3:ObjectRemoved
    • S3:ObjectRestored
  • Use cases:
    • Generating thumbnails of images uploaded to S3
    • Saving file metadata into database
107
Q

How does Lambda use Event Source Mapping?

A
  • Used in:
    • Kinesis Data Streams
    • DynamoDB Streams
    • SQS
  • Lambda will poll records from source and Event Source Mapper will invoke Lambda with event batch
    • Synchronous invocation
108
Q

How does Lambda work with streams?

A
  • Relates to Kinesis Data Streams and DynamoDB streams
  • Event Source Mapper creates an iterator for each shard
    • Items processed in order at shard-level
  • Processed items are not removed from stream
    • Other consumers can still read them
  • Low traffic stream
    • Use batch window to accumulate records before processing
  • High traffic stream
    • Process multiple batches in parallel at shard-level
    • Items processed in order for each partition key
109
Q

How does Lambda handle errors with streams?

A
  • By default, if function returns an error then entire batch is reprocessed until it succeeds or items in batch expire
  • To ensure in-order processing, processing affected shard is paused until error resolved
  • Event Source Mapping can be configured to:
    • Discard old events - can be sent to Lambda destination
    • Restrict number of retries
    • Split batch on error to work around timeout issue
110
Q

How does Lambda work with queues?

Development

A
  • Relates to SQS
  • Event Source Mapper will poll SQS using long-polling
  • Recommended to set queue visibility timeout to 6x the Lambda function timeout
  • Use Lambda destination (or DLQ) for failures
    • If using DLQ, set it up on SQS not on Lambda as DLQ on Lambda is only for asynchronous invocations and this is synchronous
111
Q

How does Lambda ensure in-order processing of queues?

A
  • Supports in-order processing of FIFO queues
  • For standard queues, items are not necessarily processed in order
112
Q

How does Lambda handle errors with queues?

A
  • Batches are returned to the queue as individual items and might be reprocessed in different grouping to original batch
  • Lambda deletes the items from queue after they’re processed successfully
  • Source queue can be configured to send failed items to DLQ
113
Q

How does Lambda handle scalability with streams and queues?

Development

A
  • Streams
    • One invocation per stream per shard
    • If parallelisation used, up to 10 batches/shard
  • SQS standard
    • Up to 1000 batches of messages processed
  • SQS FIFO
    • Scales up to number of active message groups
114
Q

What is the difference between event object vs context object in Lambda?

A
  • Event object
    • Contains data around the invoking service
    • Lambda runtime converts event to an object
    • Examples:
      • Input arguments
      • Invoking service arguments
  • Context object
    • Contains metadata about the invocation itself
    • Passed to Lambda at runtime
    • Examples:
      • function_name
      • invoked_function_arn
115
Q

What are Lambda destinations?

A
  • Async invocations can define destinations to send a result (successful or failed)
  • Recommended to use destinations rather than DLQ
    • Allow more targets to be destinations whereas DLQ can only be used with SQS/SNS
116
Q

What are Lambda environment variables?

A
  • Adjusts function behaviour without updating code
  • Key/value pair in a string datatype
  • Helpful to store secrets encrypted by KMS
117
Q

What are Lambda Execution Roles and provide examples

A
  • IAM role to grant Lambda function permissions to AWS services
  • When using Event Source Mapping, Lambda uses the execution role to read event data
  • Recommended to create 1 Lambda Execution Role per function
  • Examples:
    • AWSLambdaBasicExecutionRole
      • Upload logs to CloudWatch
    • AWSLambdaKinesisExecutionRole
      • Read from Kinesis
    • AWSLambdaDynamoDBExecutionRole
      • Read from DynamoDB streams
118
Q

How does Lambda handle logging, monitoring and tracing?

A
  • CloudWatch logs
    • Lambda execution logs are stored in CloudWatch
    • Lambda function requires appropriate execution role to write to CloudWatch
  • CloudWatch metrics
    • Lambda metrics displayed in CloudWatch metrics
  • X-Ray
    • Can be enabled in Lambda config (active tracing)
    • Runs X-Ray daemon
    • Lambda function requires IAM Execution Role
      • AWSXRayDaemonWriteAccess
119
Q

How does Lambda work with CloudFront?

A
  • CloudFront functions
  • Lambda@Edge
120
Q

What is the difference between CloudFront functions vs Lambda@Edge?

A
  • CloudFront functions
    • Light-weight functions written in JS
    • For high-scale, latency-sensitive CDN customisations
    • Modify viewer req/res
  • Lambda@Edge
    • Functions written in NodeJS or Python
    • Modify all req/res from CloudFront incl. origin req/res
121
Q

How does Lambda work with VPCs by default?

A
  • Lambda is launched outside of VPC
  • Unable to access resources within VPC
122
Q

What are the requirements for Lambda to be deployed within VPC?

A
  • Define VPC ID, subnets and assign security group to Lambda function
    • Lambda will create ENI in your subnets (happens in background) and uses that to access resources within VPC
  • Ensure appropriate access execution role is setup
    • AWSLambdaVPCAccessExecutionRole
  • Ensure appropriate access is given to resources to allow access by ENI
123
Q

How can Lambda gain access to internet when deployed within VPC?

A
  • By default, Lambda will not have internet access (even if deployed in public subnet or public IP)
  • Lambda is required to be deployed within private subnet and route outbound traffic to a NAT gateway/instance in public subnet
  • Use VPC endpoints to privately access AWS services without a NAT
124
Q

What is the execution context in Lambda?

A
  • Temp runtime environment that initialises any external dependencies used by Lambda code
  • Great for db connections, HTTP clients, SDK clients
  • Incl. /tmp directory
  • Execution context is maintained for some time in anticipation of another invocation
    • Context is reused to save time during initialisation
125
Q

What are the benefits of using Lambda layers?

A
  • Custom runtimes
    • eg. C++, Rust
  • Externalise dependencies to reuse them
    • Enable faster function deployments
    • Don’t need to repackage dependencies every time
    • Can be reused across Lambda functions
126
Q

How does Lambda handle concurrency and throttling?

A
  • Concurrency limit
    • Can request AWS for higher quota
  • Can set a reserved concurrency limit at function-level
  • Each invocation over concurrency limit will trigger throttling
  • Throttling behaviours:
    • Synchronous invocations: ThrottleError 429
    • Aysnc invocations: auto retry and then DLQ
127
Q

What can you do to avoid cold starts in Lambda?

A
  • Cold starts
    • First request served by new instances have higher latency due to initialisation
  • Provision concurrency in advance prior to being invoked
    • Cold starts never happen
    • All invocations will have lower latency
  • ASG can be used to manage concurrency
128
Q

What are the benefits of using CodeDeploy with Lambda?

A
  • Help automate traffic shift for Lambda aliases
  • Feature is integrated within SAM framework
  • Deployment types:
    • AllAtOnce
    • Linear: grow traffic every x minutes until 100%
    • Canary: grow traffic by x percent then 100%
  • Can create pre/post hooks to check health of function and roll back if necessary
129
Q

What are the execution and deployment limitations for Lambda?

A
  • Execution
    • Memory: 128MB - 10GB (1MB increments)
      • The more RAM added, the more vCPU
      • If app is CPU-bound (heavy computations), increase RAM
    • Max execution time: 15 mins
      • If more than 15 mins, better to use ECS/Fargate/EC2 instead
    • Disk capacity in /tmp: 512MB - 10GB
    • Concurrent executions: 1000
  • Deployment
    • Compressed: 50MB
    • Uncompressed: 250MB
    • Can use /tmp to load larger files at startup
    • Environment variables: 4KB
130
Q

How can we deploy Lambdas with CloudFormation?

A
  • Inline
    • Used for simple functions without dependencies
    • Add code to Code.ZipFile property in template
  • Stored as a zip file in S3
    • Refer to S3 zip file location in template
      • S3Bucket
      • S3Key
      • S3ObjectVersion
    • If new version uploaded to S3 but template is not updated, CloudFormation will not update the function
131
Q

What are the requirements to deploy Lambdas with CloudFormation across multiple accounts?

A
  • S3 bucket policy to allow access
  • Accounts that are deploying the Lambda function to have an execution role attached
132
Q

How does Lambda versioning work?

A
  • When working on a Lambda function, it will be tagged as $LATEST
  • New version will be created upon publish
    • Versions are immutable
    • Versions have their own ARN
  • Each prior version can still be accessed
133
Q

What are Lambda aliases?

A
  • Pointers that point to a Lambda function version
  • Multiple pointers can be defined to point to different versions
    • eg. dev/test/prod
  • Aliases are mutable and have own ARN
  • Useful for canary deployments by assigning weights to Lambda functions
  • Provide end user with stable endpoint as each Lambda publish will result in a new version
  • Aliases cannot reference aliases, only versions
134
Q

What are step functions?

A
  • Low code, visual workflow service
    • Manages failures, retries, service integrations and parallelisations
  • Used to orchestrate services, automate business processes and build serverless apps
  • Model workflows as state machines
    • One state machine per workflow
    • Useful for order fulfilment, data processing etc
  • JSON format
135
Q

How can workflows be started in step functions?

A
  • SDK call
  • API Gateway
  • Event Bridge (CloudWatch event)
136
Q

What are task states in a step function?

A
  • Does the work in your state machine
  • Can be used to invoke AWS service or run an activity task
    • Use cases for AWS service
      • Invoke Lambda function
      • Run batch job
      • Launch another step function workflow
    • Use cases for activity tasks
      • Activities poll step functions for work
      • Activities send results back to step functions
137
Q

How do step functions perform error handling?

A
  • State can encounter runtime errors for various reasons
  • Use retry and catch to handle errors in step functions instead of in application code
138
Q

What is the result path in a step function?

A
  • A path that determines what input is sent to the next state specified in “Next” field
139
Q

What is the “wait for task token” feature of step functions?

A
  • Allows you to pause step functions during a task until task token is returned
  • Used to wait for human approval, other services, 3rd party integration etc
  • Enabled by appending .waitForTaskToken to Resource field
140
Q

What is the difference between standard vs express step function workflow?

A
  • Standard step function workflow
    • Execution model: exactly once
    • Execution rate: > 2000/s
    • Use cases: non-idempotent actions eg. payment processing
  • Express step function workflow
    • Execution model: at least once (async) or at most once (synchronous)
    • Execution rate: > 100 000/s
    • Use cases: streaming data, mobile backends
141
Q

What is AppSync?

A
  • Managed service that uses GraphQL
  • Retrieve real-time data with web socket
  • Local data access and synchronisation with mobile apps
142
Q

What makes up CloudWatch metrics?

A
  • Metrics
    • Variable to monitor eg. CPUUtilisation, NetworkIn
    • Metrics belong to namespaces
    • Metrics have timestamps
  • Dimensions
    • Attribute of a metric eg. instance ID, environment
143
Q

What are CloudWatch logs?

A
  • Stores application logs
  • Logs can be sent to:
    • S3
    • Kinesis Data Stream/Kinesis Data Firehose
    • Lambda
    • OpenSearch
  • Can define log expiration dates
  • Logs are encrypted by default
    • Can setup KMS encryption with your own keys
144
Q

What are the requirements to obtain CloudWatch logs on EC2 instances?

Monitoring

A
  • By default, no logs from EC2 are sent to CloudWatch
  • You need to run a CloudWatch agent on EC2 to push logs to CloudWatch
    • CloudWatch log agent can be set up on-premises too
  • Ensure correct IAM permissions
145
Q

What is the difference between CloudTrail vs CloudWatch vs X-Ray?

A
  • CloudTrail
    • Audit API calls made by users/services
    • Detect unauthorised calls or root cause of changes
  • CloudWatch
    • CloudWatch Metrics: monitoring
    • CloudWatch Logs: storing application logs
    • CloudWatch Alarms: sending notifications for unexpected metrics
  • X-Ray
    • Troubleshoot app performance and errors
    • Distributed tracing of microservices
    • Useful for latency, errors and fault analysis
146
Q

What are the use cases for CloudWatch EventBridge?

A
  • Schedule: cron jobs
    • ie. schedule Lambda function to run every hour
  • Event pattern: event rules to react to a service
    • ie. send to SNS with email notification for user log-in events
  • Trigger Lambda functions
  • Send SQS/SNS messages
147
Q

What are event buses in EventBridge?

A
  • Event buses receive events from various sources and match them to rules in your account
  • Different types of event buses receive events from different sources
    • Event bus: event from AWS service
    • Partner event bus: events from partner application
  • Enabling event discovery on an event bus will generate EventBridge schemas for events on that bus
148
Q

How can we enable X-Ray?

A
  • Application code must import X-Ray SDK
    • SDK will then capture:
      • Calls to AWS services
      • HTTP/HTTPS requests
      • Calls to database
      • Calls to SQS
  • Install X-Ray Daemon or enable X-Ray integration
149
Q

What are common troubleshooting steps for X-Ray running on EC2 vs Lambda?

A
  • If X-Ray is not working on EC2 instance:
    • Ensure EC2 IAM role has permissions
    • Ensure EC2 is running X-Ray Daemon
  • If X-Ray is not working on Lambda:
    • Ensure Lambda has correct IAM execution role
    • Ensure X-Ray imported in the application code
    • Enable Lambda X-Ray active tracing
150
Q

What are the components that make up X-Ray?

A
  • Segments
    • Sent by each application/service
  • Subsegments
    • Granular details on segments
    • Represent your application’s view of downstream calls as a client
    • ie. calls to AWS services, HTTP API, SQL db
  • Service graph
    • JSON document containing information about services/resources that make up your application
    • Service graph data is retained for 30 days
  • Edges
    • Connect services that work together to serve requests
  • Trace
    • Segments collected together to form an e2e trace from a single request
  • Sampling algorithm
    • Reduce amount of requests sent to X-Ray by determining which requests get traced
  • Annotations
    • Key/value pairs used to index traces
    • Used with filter expressions
  • Metadata
    • Key/value pairs not used for indexing/searching
151
Q

List the X-Ray write APIs and their function

A
  • PutTraceSegments: upload segments to X-Ray
  • PutTelemetryRecords: used by X-Ray Daemon to send telemetry
  • GetSamplingRules: retrieve sampling rules
  • GetSamplingTargets
  • GetSamplingStatisticsSummaries
152
Q

List the X-Ray read APIs and their function

A
  • GetServiceGraph: main graph
  • BatchGetTraces: retrieve list of traces specified by ID
  • GetTraceSummaries: retrieves IDs and annotations for traces available for a specified time period using optional filter
  • GetTraceGraph: retrieve service graph for one or more specific trace IDs
153
Q

What are the requirements to integrate X-Ray with Elastic Beanstalk?

A
  • Ensure correct IAM permissions
  • Application code is importing X-Ray SDK
  • Elastic Beanstalk platform includes X-Ray Daemon
    • Run the Daemon by enabling the option in the console or with config file
    • X-Ray Daemon is not provided for multi-container docker
154
Q

What are the requirements to integrate X-Ray with ECS?

A
  • ECS cluster
    • X-Ray container as a Daemon
      • Running X-Ray Daemon container on each EC2 instance
    • X-Ray container as a “sidecar”
      • Running X-Ray Daemon container alongside each application container within EC2 instance
  • Fargate cluster
    • Can only run X-Ray container as a “sidecar” pattern
155
Q

What are task definitions in ECS?

A
  • Metadata in JSON format to instruct ECS on how to run docker container
    • Image name
    • IAM role
    • Port bindings for container and host
    • Memory and CPU
    • Environment variables
    • Logging config
  • Can define up to 10 containers in a task definition
156
Q

What is the difference between ECS vs EKS vs Fargate vs ECR?

A
  • ECS (Elastic Container Service)
    • Container orchestration service to easily scale containerised applications
    • Integrated with ECR and Docker
  • EKS (Elastic Kubernetes Service)
    • Managed Kubernetes service
  • Fargate
    • Serverless compute engine for containers
    • Works with both ECS and EKS
  • ECR (Elastic Container Registry)
    • Fully managed Docker container registry
    • Store and deploy container images
157
Q

What is the difference between EC2 vs Fargate launch types in ECS?

A
  • EC2 launch type
    • Must provision/maintain infra
    • Each EC2 instance must run Docker ECS Agent to register in ECS cluster
    • AWS takes care of starting/stopping containers
  • Fargate launch type
    • Serverless
    • Do not need to provision/maintain infra
    • Need to create task definitions
    • AWS takes care of running ECS tasks based on CPU/RAM
      • For scalability, just increase/decrease no. of tasks
158
Q

What are port mappings and how do they get configured in ECS?

A
  • Port mappings allow containers to access ports on host container to send/receive traffic
  • Specified as part of container definition which is configured in task definition
159
Q

How can ALB find the right port on EC2 instances on EC2 launch type?

A
  • Using dynamic host port mapping if you define only the container port in task definition
  • Must allow “any port” on the EC2 instance’s security group from ALB security group
160
Q

What are task placements in EC2 launch types?

A
  • Contains task placement strategy and constraints for ECS to determine which EC2 instance to add/terminate a new container
161
Q

What process does ECS follow to select container instances in EC2 launch types?

A
  1. Identify instances that satisfy CPU, memory and port requirements in task definition
  2. Identify instances that satisfy task placement constraints
  3. Identify instances that satisfy task placement strategies
  4. Select the instance for task placement and place the task there
162
Q

What are the different task placement strategies in EC2 launch types?

A
  • Binpack
    • Place tasks based on least amount of available CPU/memory
    • Minimise no. of instances in use to reduce costs
  • Random
    • Place tasks randomly
  • Spread
    • Place tasks evenly based on specified value
    • ie. spread on AZ or instanceId
163
Q

What are the task placement constraints in EC2 launch types?

A
  • distinctInstance
    • Place each task on a different container instance
  • memberOf
    • Place tasks on instances that satisfies an expression
164
Q

What is the difference between EC2 instance profile role vs ECS task role?

A
  • EC2 instance profile role
    • Only for EC2 launch type
    • Make container calls to ECS service
    • Pull docker image from ECR
  • ECS task role
    • Available for both launch types
    • Allows each task to have specific role
    • Task role defined in task definition
165
Q

How can ECS persist data?

A
  • Using data volumes by mounting EFS onto ECS tasks
    • Tasks running in any AZ will share same data in EFS
  • Works for both EC2 and Fargate launch types
  • S3 cannot be mounted as file system
  • Use cases:
    • Persistent multi AZ shared storage for containers
166
Q

What are the different ways for ECS to handle service auto scaling?

A
  • ECS service auto scaling (task level) is not the same as EC2 auto scaling (instance level)
    • Use Fargate auto scaling if no EC2 instances involved as it’s much easier to setup
  • Strategies:
    • Target tracking: scale based on target value for specific CloudWatch metric
    • Step scaling: scale based on specified CloudWatch alarm
    • Scheduled scaling: scale based on specified date/time
167
Q

What is the difference between ASG scaling vs ECS cluster capacity provider for EC2 launch type?

A
  • ASG scaling
    • Scale ASG based on CPU utilisation
    • Add EC2 instances over time
  • ECS cluster capacity provider (smarter approach)
    • Capacity provider paired with ASG
    • Automatically provision and scale infrastructure for ECS tasks
    • Add EC2 instances when you’re missing capacity (CPU, RAM etc)
168
Q

How does ECS handle environment variables?

A
  • These values are fetched and resolved at runtime and injected as environment variables within ECS task
  • Non-sensitive information
    • Hard-coded eg. URLs
  • Sensitive information
    • SSM parameter store eg. API keys
    • Secrets manager eg. DB passwords
  • Environment files (bulk) can also be loaded into S3
169
Q

How can ECS share data?

A
  • Data volumes (bind mounts)
    • Mount EFS volume
  • Share data between multiple containers in same task definition
  • Works for both EC2 and Fargate launch types
    • EC2 launch type
      • Uses EC2 instance storage
      • Data tied to lifecycle of EC2 instance
    • Fargate launch type
      • Uses ephemeral storage
      • Data tied to containers using them
  • Use cases:
    • Share ephemeral storage between containers
    • “Sidecar” container pattern where sidecar is used to send metrics/logs to other destinations
170
Q

How does ECS handle load balancing for Fargate launch type?

A
  • Each ECS task has unique private IP
    • ECS ENI security group allow port 80 from ALB
    • ALB security group allow 80/443 from web
  • Only define container port
    • Host port not applicable for Fargate
171
Q

What is CodeDeploy?

A
  • Service that automates application deployment
    • Defined in appspec.yml file
  • Deploy new app versions to:
    • EC2/on-prem servers
    • Lambda functions
    • ECS
  • Automated rollback or trigger CloudWatch alarm
  • Gradual deployment control
172
Q

How does CodeDeploy work with EC2/on-prem platform?

A
  • Perform in-place deployments or blue/green deployments
  • Must run CodeDeploy Agent on target instances
    • Installed as a prerequisite or via Systems Manager
  • EC2 instances must have appropriate permissions to access S3 for deployment bundles
  • Define deployment speed:
    • AllAtATime: most downtime
    • HalfAtATime: reduced capacity by 50%
    • OneAtATime: slowest, but lowest impact
    • Custom

Learning (7 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions