AWS services Flashcards
Key points of each service
AWS DataSync
- migrate data in a simple and secure way
- migrate data between on-premise and cloud
- migrate data between cloud providers
Application Discovery Service
Gathering information about their on-premises data centers
AWS Resource Access Manager (RAM)
- enables you to share AWS resources easily and securely with any AWS account or within your AWS Organization.
- You can create resources centrally in a multi-account environment, and use RAM to share those resources across accounts
AWS WAF
helps protect your web applications from SQL injection, cross-site scripting attacks (things at HTTP level 7 layer)
AWS Shield
DDoS protection service (infrastructer, layer 3 & 4 network and transport layer)
AWS Shield Advanced - higher level protection
AWS Secrets Manager
- protect secrets needed to access your applications, services, and IT resources
- easily rotate, manage and retrieve credentials, API keys and other secrets
- secrets are accessed by making an API call to the Secrets Manager API
- built-in rotation of secrets for RDS, Redshift, DocumentDB
- encryption at rest
AWS CloudHSM (Hardware Security Module)
- AWS CloudHSM allows you to generate, store, and manage cryptographic keys securely.
- helps you meet corporate, contractual, and regulatory compliance requirements for data security by using FIPS 140-2 Level 3 validated HSMs.
AWS KMS (Key Management Store)
- managed service that enables you to easily encrypt your data
- centrally manage and securely store your keys
- consists of Customer Managed Keys (CMK) and AWS Managed Keys
AWS Certificates Manager
Create, store, and renew SSL/TLS X.509 certificates
AWS GuardDuty
- Intelligent threat detection service
- Continuously monitors for malicious activity and delivers detailed security findings for visibility and remediation.
- Monitors AWS accounts, workloads, and data in Amazon S3.
Amazon GuardDuty can generate findings based on suspicious activities such as requests coming from known malicious IP addresses, changing of bucket policies/ACLs to expose an S3 bucket publicly, or suspicious API call patterns that attempt to discover misconfigured bucket permissions.
AWS Trusted Advisor
Trusted Advisor is an online resource that helps to reduce cost, increase performance, and improve security by optimizing your AWS environment.
AWS Inspector
- automated security assessment service that helps improve the security and compliance of applications deployed on AWS.
- Amazon Inspector automatically assesses applications for exposure, vulnerabilities, and deviations from best practices.
AWS Cognito
- user sign-up, sign-in, and access control to your web and mobile apps quickly and easily.
- authentication, authorization, and user management for your web and mobile apps.
- Your users can sign in directly with a user name and password, or through a third party such as Facebook, Amazon, or Google.
Amazon Cognito - Web Identity Federation
- Federation allows users to authenticate with a Web Identity Provider (e.g. Google, Facebook, Amazon).
- AWS Cognito works with external identity providers that support SAML or OpenID Connect, social identity providers (such as Facebook, Twitter, Amazon)
AWS Cognito - User Pools and Identity Pools
- User pools are user directories that provide sign-up and sign-in options for your app users.
- Identity pools enable you to grant your users access to other AWS services.
Active Directory
Active Directory (AD) is Microsoft’s proprietary directory service. It runs on Windows Server and enables administrators to manage permissions and access to network resources.
Active Directory (AD) is a database and set of services that connect users with the network resources they need to get their work done. The database (or directory) contains critical information about your environment, including what users and computers there are and who’s allowed to do what.
Active Directory Service for Microsoft Active Directory
- Best choice if you have more than 5000 users and/or need a trust relationship set up
- Runs on Windows Server
Simple AD (Active Directory)
- An inexpensive Active Directory-compatible service with common directory features.
- Standalone, fully managed, directory on the AWS cloud.
- Best choice for less than 5000 users and don’t need advanced AD feature.
AD Connector (Active Directory)
- AD Connector is a directory gateway for redirecting directory requests to your on-premises Active Directory.
- AD Connector eliminates the need for directory synchronization and the cost and complexity of hosting a federation infrastructure.
- Connects your existing on-premises AD to AWS.
- Best choice when you want to use an existing Active Directory with AWS services.
AWS IAM (Identity & Access Management)
securely control individual and group access to AWS resources
IAM can be used to manage:
* Users
* Groups
* Access policies
* Roles
* User credentials
* User password policies
* Multi-factor authentication (MFA).
* API keys for programmatic access (CLI)
AWS IAM (Identity & Access Management)
- User is an entity that represents a person or service.
- Groups are collections of users and have policies attached to them
- Roles are created and then “assumed” by trusted entities and define a set of permissions for making AWS service requests
- Policies are documents that define permissions and can be applied to users, groups, and roles.
AWS Security Token Service (STS)
AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for IAM users or for users that you authenticate (federated users).
Cross Account Access
Useful for situations where an AWS customer has separate AWS account – for example for development and production resources.
Cross Account Access makes is easier to work productively within a multi-account (or multi-role) AWS environment by making is easy to switch roles within the AWS Management Console.
Identity & Access Management (IAM) Best Practices
To secure AWS resources it is recommended that you follow these best practices:
- Lock away your AWS account root user access keys.
- Use roles to delegate permissions.
- Grant least privilege.
- Get started using permissions with AWS managed policies.
- Validate your policies.
- Use customer managed policies instead of inline policies.
- Use access levels to review IAM permissions.
- Configure a strong password policy for your users.
- Enable MFA.
- Use roles for applications that run on Amazon EC2 instances.
- Do not share access keys.
- Rotate credentials regularly.
- Remove unnecessary credentials.
- Use policy conditions for extra security.
- Monitor activity in your AWS account.
Amazon SNS (Simple Notification Service)
Fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication
pub/sub functionality provides messaging for high-throughput, push-based, many-to-many use cases
SNS uses a pub-sub model whereby users or applications subscribe to SNS topics.
When subscribing to an SNS topic the following endpoint types are supported:
* HTTP/HTTPS.
* Email/Email-JSON.
* Amazon Kinesis Data Firehose.
* Amazon SQS.
* AWS Lambda.
* Platform application endpoint (mobile push).
SMS.
SNS Fanout
Your publisher systems can fanout messages to many subscriber systems including Amazon SQS queues, AWS Lambda functions and HTTPS endpoints, for parallel processing, and Amazon Kinesis Data Firehose.
You can subscribe one or more Amazon SQS queues to an Amazon SNS topic from a list of topics available for the selected queue. When you publish a message to a topic, Amazon SNS sends the message to every subscribed SQS queue. Amazon SQS manages the subscription and any necessary permissions
Amazon Simple Queue Service (SQS)
distributed queue system that enables web service applications to quickly and reliably queue messages
Messages in the queue need to be polled by the consumer (e.g. EC2 instance)
Amazon SQS is pull-based, not push-based (like Amazon SNS).
Messages can be kept in the queue from 1 minute to 14 days.
The default retention period is 4 days.
The SQS queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.
You can use an AWS Lambda function to process messages in an Amazon Simple Queue Service (Amazon SQS) queue. Lambda polls the queue and invokes your Lambda function synchronously with an event that contains queue messages.
Lambda reads messages in batches and invokes your function once for each batch.
When your function successfully processes a batch, Lambda deletes its messages from the queue.
Dead-Letter Queue
dead-letter queue handles message failure (applies to SNS and SQS)
It lets you set aside and isolate messages that can’t be processed correctly to determine why their processing didn’t succeed
Amazon Simple Workflow Service (SWF)
Amazon SWF is used for processing background jobs that have parallel or sequential steps. You can think of Amazon SWF as a fully managed state tracker and task coordinator
AWS Step Functions
AWS Step Functions can be used to coordinate the components of distributed applications as a series of steps in a visual workflow
This is orchestration (a controller who controls interactions between services) as opposed to choreography (lots of small, independany services working together, loosly coupled)
Amazon OpenSearch
It is an open source, distributed search and analytics suite based on Elasticsearch.
Successor to the Amazon Elasticsearch Service
Elasticsearch is a distributed search and analytics engine built on Apache Lucene.
Elasticsearch is a popular search engine commonly used for log analytics, full-text search, security intelligence, business analytics, and operational intelligence use cases.
With OpenSearch you can perform log analytics interactively, perform real-time application monitoring, website search, performance metric analysis and more.
AWS Glue
- Fully-managed, pay-as-you-go, extract, transform, and load (ETL) service that automates the time-consuming steps of data preparation for analytics.
- Glue can automatically discover both structured and semi-structured data stored in data lakes on Amazon S3, data warehouses in Amazon Redshift, and various databases running on AWS.
- automatically discovers and profiles data via the Glue Data Catalog, recommends and generates ETL code to transform your source data into target schemas
- runs the ETL jobs on a fully managed, scale-out Apache Spark environment to load your data into its destination
- You can create and run an ETL job with a few clicks in the AWS Management Console.
Amazon Athena
- analyze data in Amazon S3 using standard SQL
- Can use with other services such as Amazon RedShift (data warehouse)
Overview of 4 services
- Athena = query S3 data using SQL
- RedShift = data warehouse (columnar storage)
- EMR(Elastic MapReduce) = data processing (big data), running tasks such as machine learning, graph analytics, data transformation, streaming data.
- Glue = ETL service, transform data from S3 bucket, RedShift, other databases and move to various destinations.
Amazon Kinesis
- makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information
- Data is processed in “shards” – with each shard able to ingest 1000 records per second
- max default limit of 500 shards that can be increased
- A record consists of a partition key, sequence number, and data blob (up to 1 MB).
- Transient data store – default retention of 24 hours but can be configured for up to 7 days.
Kinesis Data Streams
- real-time processing of streaming big data.
- stores data for later processing by applications (key difference to Firehose which delivers data directly to AWS services).
- data is captured and stores for processing in shards
- each shard can support up to 1000 PUT records per second
- records of a stream are accessible for up to 24 hours from the time they are added to the stream (can be raised to 7 days by enabling extended data retention).
- Splitting increases the number of shards in your stream and therefore increases the data capacity of the stream.
- Splitting increases the cost of your stream (you pay per-shard).
Kinesis Data Firehouse
- easiest way to load streaming data into data stores and analytics tools.
- Captures, transforms, and loads streaming data
- Kinesis Data Streams can be used as the source(s) to Kinesis Data Firehose
- You can configure Kinesis Data Firehose to transform your data before delivering it.
- With Kinesis Data Firehose you don’t need to write an application or manage resources
- Each delivery stream stores data records for up to 24 hours.
- Data can be delivered to S3, RedShift, ElasticSearch, Splunk
Kinesis Data Streams vs Firehouse
Streams - Allows you to write custom consumers
Firehouse - Simplifies data transforming and data storing
Streams - Guarantees order delivery
Firehouse - Messages can be delivered more than once as the order is not guaranteed
Streams - Failure block the shard until succession or expiration
Firehouse - It has a built-in retry mechanism for each delivery
Streams - Allows you to set the number of shard
Firehouse - Allows you to set the data volume. Shards are managed by the service
Streams - It supports multiple types of consumers along with multiple consumers
Firehouse - Stream can only have one destination
SQS vs SNS vs Kinesis
SQS:
* Consumers pull data.
* Messages still persist in the queue after being read. An API call to delete the message from the queue is required after successfull receipt of the message from the consumer.
* One consumer by default only possible
* No need to provision throughput.
* No ordering guarantee (except with FIFO queues).
* Individual message delay.
SNS:
* Push data to many subscribers.
* Up to 10,000,000 subscribers.
* Data is not persisted (lost if not deleted).
* Pub/sub.
* Up to 10,000,000 topics.
* No need to provision throughput.
* Integrates with SQS for fan-out architecture pattern.
Kinesis:
* Consumers pull data.
* As many consumers as you need.
* Possible to replay data.
* Meant for real-time big data, analytics, and ETL.
* Ordering at the shard level.
* Data expires after X days.
* Must provision throughput.
Amazon Elastic MapReduce (EMR)
- big data processing and analysis.
- web service that enables businesses, researchers, data analysts, and developers to process vast amounts of data
- EMR utilizes a hosted Hadoop framework running on Amazon EC2 and Amazon S3
- With EMR you can run petabyte-scale analysis at less than half of the cost of traditional on-premises solutions and over 3x faster than standard Apache Spark
- Most used for log analysis, financial analysis, or extract, translate and loading (ETL) activitie
AWS Elastic Transcoder
- convert (or “transcode”) video and audio files from their source format into versions that will playback on devices like smartphones, tablets and PCs.
- Picks up files from an input S3 bucket and saves the output to an output S3 bucket.
- You are charged based on the duration of the content and the resolution or format of the media
AWS Systems Manager
- AWS Systems Manager is a central hub to control and view your entire AWS infrastructure
*
AWS Config
- AWS Config continually assesses, audits, and evaluates the configurations and relationships of your resources on AWS, on premises, and on other clouds.
- enables security and governance
- AWS Config records point-in-time configuration details for your AWS resources as Configuration Items
