Section 20: AWS Monitoring & Audit, CloudWatch, X-Ray and CloudTrail: CloudWatch Flashcards
What does monitoring help us monitor/mitigate/do?
- Monitoring helps us monitor app latency, app outages, trends (scaling patterns), performance and cost.
- Monitoring helps mitigate the chances of users contacting IT to complain about some bad behavior and generally help us precent issues before they happen.
- Monitoring can help us troubleshoot remediate, learn about and improve our apps .
What can monitoring help with in terms of deployment?
Monitoring can help us determine how to deploy apps safely, automatically, using best aws components and using infrastucture as code.
What is AWS CloudWatch used to do/collect (four main things)
- Metrics (collect and track)
- Logs (collect, monitor, analyze, store log files)
- events (send notifications when certain events happen in your aws)
- alarms (react in real time to metrics/events)
You want metrics, logs, events notifications, and/or alarms. What AWS service could you use?
CloudWatch
What does AWS X-Ray help do?
- Troubleshoot application performance and errors
- distribute tracing of microservices
You want to use an aws service to help with troubleshooting application performance and errors and with distributed tracing of microservices. What aws service do you use?
AWS X-Ray.
What does AWS CloudTrail help with?
- Internal monitoring of API calls being made
- audit changes to aws resources made by your users
You want an aws service to help with internal monitoring of api calls being made, or with auditing changes made to aws resources by your users. What AWS service would help with this?
AWS CloudTrail.
Is CloudWatch capable of providing metrics for every AWS service?
Yes
What are two examples of CloudWatch metrics?
CPUUtilization, NetworkIn
What grouping do CloudWatch metrics belong to?
Namespaces.
“A namespace is a container for CloudWatch metrics. Metrics in different namespaces are isolated from each other, so that metrics from different applications are not mistakenly aggregated into the same statistics.
There is no default namespace. You must specify a namespace for each data point you publish to CloudWatch. You can specify a namespace name when you create a metric. These names must contain valid ASCII characters, and be 255 or fewer characters. Possible characters are: alphanumeric characters (0-9A-Za-z), period (.), hyphen (-), underscore (_), forward slash (/), hash (#), and colon (:). A namespace must contain at least one non-whitespace character.
The AWS namespaces typically use the following naming convention: AWS/service. For example, Amazon EC2 uses the AWS/EC2 namespace. For the list of AWS namespaces, see AWS services that publish CloudWatch metrics.”
What are metric attributes called, what are a couple examples of those, and how many of those can you have per metric?
Dimensions are metric attributes. Two dimension examples are instance.id (in the aws docs called InstanceId) and environment.name (probably EnvironmentName in aws docs, though it’s trickier to find). You can have up to 30 dimensions per metric.
- Do metrics have timestamps?
- Can you create CloudWatch dashboards of metrics?
yes and yes.
CloudWatch and ec2 monitoring:
* ec2 instances send metrics to CloudWatch every 5 minutes. What can you use to get metrics sent to CloudWatch more often, how often would you get them, and is there a cost?
- ec2 detailed monitoring
- every 1 minute
- there is an extra cost sometimes (AWS free tier allows us to have 10 detialed monitoring metrics)
Would ec2 detailed monitoring (the thing that helps your ec2 instance send metric data to CloudWatch every 1 minute) help your ASG be able to scale faster?
Yes. This is probably because without detailed monitoring you’re metric data is getting sent to CloudWatch every 5 minutes, so you’ll probably know about issues requiring ASG action up four minutes sooner when you use detailed monitoring.
Lets say you wanted ec2 memory usage to be sent to cloudwatch with the rest of the ec2 metrics, but it’s not showing up. What’s the problem/solution?
Ec2 memory usage is not pushed to CloudWatch by default. It must be pushed from inside the instance as a custom metric.
- What do you use when you want to define and send your own custom metrics to CloudWatch (name of the subservice and the api call)?
- What are some examples of what kinds of custom metrics people are interested in?
- CloudWatch custom metrics
- API call PutMetricData
- memory (RAM usage), disk space, number of logged in users
Can you still segment metrics into dimentions when using CloudWatch Custom Metrics?
yes.
- What are the two categories (and their respective values) for metric resolution (idk why they didn’t call it metric retreival frequency)
- What is the API parameter used to set metric resolution (hint, it’s not MetricResolution)
- Standard (1 minute)
- High Resolution (1/5/10/30 seconds) - Higher cost
- StorageResolution API parameter. The two possible values are the standard and high ones mentioned above (which kind of makes me suspect StorageResolution takes a String, ‘Standard’, or a dictionary, like {“High Resolution”: 5}). Haven’t double checked that, just guessing.
Better, though completely only anna-fficial, name for CloudWatch Custom Metric metric resolution (just the metric resolution part)
Metric Retreival Frequency. Or Metric Push Frequency. Not sure which direction things are going in. Perhaps Metric Receiving Frequency? Metric Receival Frequency.
IMPORTANT
What range of time can CloudWatch Custom Metrics accept data points for?
2 weeks in the past to two hours in the future (make sure to configure your ec2 instance time correctly).
Okay, CloudWatch Logs.
* What are log groups?
* What are log streams?
* When do logs expire (what are the log expiration policies)?
- Log groups are arbitrary names, usually representing an applicatoin
- log streams are instances within an application / log files/ containers.
- can define log expiration policies (never expire, 1 day to 10 years)
What serices can CloudWatch logs send logs to?
- S3 (exports)
- Kinesis Data Streams
- Kinesis Data Firehose
- AWS Lambda
- OpenSearch (not even sure this is an aws thing)
When using CloudWatch Logs, are logs encrypted by default? Can you setup KMS based encryption with your own keys.
- Logs are encrypted by default
- you can setup KMS based encryption with your own keys.
What are some popular CloudWatch Logs sources?
- SDK, CloudWatch Logs Agent, CloudWatch Unified Agent
- Elastic Beanstalk (collection of logs from app)
- ECS (collection from containers)
- aws lambda (collection from function logs)
- vpc flow logs (vpc specific logs)
- api gateway
- cloudTrail based on filter
- Route 53: Log DNS queries
How do you generate CloudWatch Logs insights?
- Search and analyze log data stored in CloudWatch logs
- for example: find a specific IP inside a log, count occurrences of “Error” in our logs
*
Does CloudWatch Logs Insights provide a purpose build query language? Provide any other information
Yes.
* It automatically discovers fields from aws services and json log events
* fetch desired event fields, filter based on conditions, calculate aggregate stats, sort events, limit number of events
* can save queries and add them to cloudwatch Dashboards
* –
- Can you query multiple logs groups in different aws accounts using CloudWatch Logs Insights?
- Is CloudWatch Logs Insights a query engine or a real time engine?
- Yes
- a query engine
Whats an example of the CloudWatch Logs Insights query used to:
* find the 25 most recently added log events
* return the number of exceptions logged every 5 minutes
* get a list of log events that are not exceptions
- If you want to use CloudWatch Logs to get logs to S3, how long can it take log data to become available for S3 export?
- What is the API call used to export CloudWatch logs by first sending them to S3?
- Is the data in this export near real-time or real-time? If it’s not, what would you use instead?
- up to 12 hours
- API call is CreateExportTask
- not near real-time or real-time. Use Logs Subscriptions instead.
What can CloudWatch Logs Subscriptions help with?
Getting real time log events from CloudWatch Logs for processing and analysis.
To what aws services does CloudWatch Logs Subscriptions send data?
CloudWatch Logs Subscriptions can send data to Kinesis Data Streams, Kinesis Data Firehose, or Lambda. From here, the data is sent to other aws service instances as needed.
What is a CloudWatch Logs Subscriptions subscription filter?
CloudWatch Logs Subscriptions filter which logs are events delivered to your destination
You have some real time CloudWatch Logs that you want to use to search sites (or for real time app monitoring or analytics). What aws services might you use?
CloudWatch Logs -> CloudWatch Logs Subscription Filter -> AWS Lambda -> AWS OpenSearch