UDEMY 2 Flashcards
Two flavors of Windows FsX
SSD: faster more expensive
HDD: broad spectrum of workloads
can be multi az and vpn or direct connect
FsX for Lustre
Linux + Cluster
High performing HPC
Machine learning
seamless integration with S3
Datasync
Prserves the metadata..replications of large data s3, efs, fsx (can be used with snowcone).
moves data between on premises and AWS Storage services
Long polling
optimize api calls
decreases latency
fifo sqs
lower throughput
messages processed in order
SQS with database
requests go into SQS que and then autoscaling automatically as a buffer
decoupling sudden time outs
sns topic publish
create topic
create subscription(s)
Publish topic
SNS direct publish
create a platform app
create a platform endpoint
publish to platform endpoint
sns + sqs fanout
sns topic feeds into appropriate sqs cues
SQS access policies
work like bucket policies
SNS message filtering
json policy to filter message in the sns message topics
Kinesis
collect process and analyze data real time
Kinesis Data Streams
Stream big data (Shards)
data gets split into shards
come from several producers at once then read by many different consumers
assigned Partition keys: go to the same shard
provisioned mode or pay per shard per hours
On demand: autoscaling per stream per hours gb inout Real time and customer managed
storage btwn 1 -365 days
Kinesis Firehose
producers can be applications, kiniesis data stream and kiesis data streams. data is batched and written to destinations.
could be amazon s3, redshift, or opensearch (or 3rd party) or custom destination Near real time and fully managed no storage
amazon mq
traditional message protocl when migrating w/o changing.
runs on servers
no scaliing
ECS
Instance profile and roles per task
manage ec2
efs is ideal
fargate
dont’ manage ec2
efs is ideal
ECS auto scaling
cpu
alb reuest count
memory
ecs cluster capacity provider
AWS app runner service
fully managed ease to deploy web applications and api calls/use with containers
provisioned iops ssd ecs
io1 io2
database
(multi attach 16 instances supported
write and read)
st 1sc1
cannot be boot volume
archived data lowest cost
efs
linux not windows
bursting provisioned elastic
storage tiers
standard, or infrequent IA (lifecycle policy)
Gateway Load balancer
Level 3 GENEVE Port 6081
target groups: instances, private ips
sticky sessions
ElB works with cookies instead of a spread. client goes to same instance. network, classic, and application Lb only
custom cookies or application cookies
cross load balancing
Network and gateway: disabled by default. charged to enable.
application: enabled by default.
classic: disabled by default-no charge to enable
ssl and tll certificates
encrypted connections btwn client and load balancer
x509 certificate managed by ACM (amazon certificate manager)
SNI
loads multiple ssl and ttl onto one server works for ALB and NLB: not cloufront or Classic
connection draining or derigestration delay
EC2 shuts down bc of a health check, default 300 sec. request finished first then ec2 is drained
Scaling Groups
Dynamic (target tracking)
simple/step triggered by defined cloudwatch alarms
scheduled
reserved cookie names alb
ELB (AWSALB, AWSALBAPP, AWSALBTG).
When using an Application Load Balancer to distribute traffic to your EC2 instances, the IP address you’ll receive requests from will be the ALB’s private IP addresses. To get the client’s IP address, ALB adds an additional header called “X-Forwarded-For” contains the client’s IP address.
cron job
scheduled function-serverless lambda
lamba limits
128 M in 10m\gb increments
maximum execution-15min
m1000 concureent functions
size 50MB max or compressed 250MB
lambda snapstart
java 11 or above
differences between lamada and cloufront edge functions
cloudfront: Java high scale cdn functions: viewer requests/response only. (millions of requests per sec) less then one milisecond execution time
Lambda: Nodejs or Python. Viewer request/Origin Request/Origin,viewer response (thousands of request per sec)
dynamo
rapidly evolving schema
up to 400kb
provisioned mode
On demand mode (auto scaling) (more expensive for sudden spikes and unpredictible data)
DAX (DB acceleoator)
in-memory cache
microsecond latency
made of clusterzs
dyanmo Stream processor
real time analytics
Real time changes reaction
cross region replication
dynamo DB streams or
Kinesis Data Streams (longer retention and users) more way to analyze
Dynamo DB TTL
delte items after expire timestamep (auto) good for regulatory obligations, or session handling
Dynamo Backup
PITR (point in time recovery) (export into s3) 35 days
or dynamo backups.
or aws backup service
API Gateway
use it to proxy lambda requests. serverless
support for websocket protocol
use Swagger
http (enpoints or load balancer)
API gateway enpoint types
edge optimized
regional
private within vpc
cognito
Users get identigy to access a web mobile identity.
User pool: app users
identy pool: temporary aws resources
SAML 100’s of users
microserves interact how?
Rest api
Keyspaces
no squl db apache Cassandra on demand and provisioned
ALDB
quantum ledger db, manages application data overtime. No deletes
financial regulation rules
no decenttalization
timestream
time series db
Athena
columna query data for less mney. apaceh parquqy or ORC (using Glue)
partition data sets for data your are often querying.
Resshift
OLAP Redshift postsql
data has to be loaded, but faster queries then athena
Redshift has Indexes.
supports many queries
Leader and compute noeds
AMAZON OPEN SEARCH
No primary key required
provides search abilty in front of the main DB (ex: dynamo)
can used with kinesis, or cloudwatch log
EMR
Hadoop clusters: big data clusters large amounts of data.
master node, core node, task node (optional can run spot).
on demand or reserved
Glue
ETL service extracts, tranforms and loads data for analytics
Converts the data to parquet format
Glue data catalog: essentail to emr, redshift, and redshift
glue elastic views; spreads across dbs
glue job bookmarks
keeps you from re-unning old data
Lake formation
Creates data lakes
fully managed
layered on top of glue
athena, redshift emr
Centralized permissions are a huge advantage. access control and row column security
Kinesis data analytics
2 kinds
sql applicaitons: data streams and firehose
pay for consumption
for apache flink: write using java or sql. must be write as code on a dedicated cluster. flink is more powerful then sql
Amazon msk
alternative for Kenisis
Kafka
stream data with consumers and producers from “brokers”
kafka topics
add partitions but no removing
Amazon Rekognition
Facial Analysis db
machine learning
content moderation: rmoves inappropriate conetnt. set a min confidence threshold
amazon augumented ai A2i
manual human review
amazon transcribe
convert speech into text
vsar
automatically removes PII
multi-lingual
Polly
Text into speech for deep learning
uses lexicon and ssml
amazon lex
Automatic speech reconition (ASR)
helps build chatbots and
amazon connect
cloud based virtual contact center
Comprehend
NLP
sagemaker
fully managed to build machine learning. developers create and build models. train and tune
forecast
fully managed to build forecasting
Kendra
extract exerpts within a document-builds a knowlege base Machine learning
personalize
ML build apps to reccomend items (amazon.com reccommending products)
textract
extracts text for ML
when cloud trail is enabled, What should you use to detect unusual activity in your AWS Account?
Cloud Trail Data Insights
cloudwach metric streams
continually stream metrics into kinesis firehose into s3 or redshfit, or opensearch
cloudwatch logs
Store cloudwatch logs
expiration: 1 day to 10 years
cloudwatch logs
set up ec2 IAM premissions to stream logs from ec2
two kids, agent and unified
Unifed can do metrics and logs
Cloudwatch Insights and visibility
container insights
lambda insights
contributors Insights
application insights
cloudtrail
api calls, governance and compliance
accumulate into one s3 bucket
Config
Auditing and compliance of your services
can run events as configs change, or on an event schedule
link to cloudtrail.
you can remediate using an ssm automation document.
