XRAY Flashcards
Xray overview
Visual Analysis of application
- can see distributed applications and see requests that do well or fail
- can visualize how services call other services
Xray Advantages
when and where to use it
what it can see
Troubleshoot bottlenecks
Understand dependencies in Microservice architecture
-see how microservices interact
-method calls to one another.
Pinpoint service issues
See how each request behaives
Find error and exceptions
See if we are meeting SLA, time requirements
See Throttle location issues
Identity users that are impacted
XRAY tracing
request
trace
security
trace: End to end tracking of “request”
request: Services that call to one another, each call leaves a “trace”
Trace: made of segments and subsegments,
Trace:
- Every request
- Sample request: % rate per minute
X-ray Security:
IAM authorization
KMS for encryption at rest
Segment: The compute resources running your application logic send data about their work as segments. A segment provides the resource’s name, details about the request, and details about the work done. For example, when an HTTP request reaches your application, it can record the following data about:
The host – hostname, alias or IP address
The request – method, client address, path, user agent
The response – status, content
The work done – start and end times, subsegments
Issues that occur – errors, faults and exceptions, including automatic capture of exception stacks.
SUBSEGMENTS: A segment can break down the data about the work done into subsegments. Subsegments provide more granular timing information and details about downstream calls that your application made to fulfill the original request. A subsegment can contain additional details about a call to an AWS service, an external HTTP API, or an SQL database. You can even define arbitrary subsegments to instrument specific functions or lines of code in your application.
X-ray how to enable
code
X-ray Daemon
X-ray aws integration
1: VIA CODE: import aws X-ray SDK
- very little code modding is needed
- Application SDK will capture:
Calls to AWS services
HTTP / HTTPS request
DB calls MYSQL, postresql, Dynamodb
Queue Calls (SQS)
- X-ray Daemon, X-ray integration
A. X-ray Daemon: Works like UDP packet interceptor
(linux, windows, mac)
B. X-ray aws integration: already done, AWS lambda/ other services run the daemon for you
each application MUST HAVE IAM RIGHTS TO write data to X-ray.
DIAGRAM
- Ec2: application code needs to add AWS XRAY SDK
- Traces sent to XRAY DAEMON, which runs on machine
- XRAY DAEMON SENDS BATCH 1 EVERY SECOND TO AWS XRAY (NEED IAM PERMISSIONS)
X-RAY data aggregation
- Collection of data from Xray batches from daemon
- Service Map is created/computed from all SEGMENTS and TRACE
x-ray Troubleshooting:
ec2
lambda
NOT WORKING ON EC2
- permissions issue
- instance needs to run daemon
AWS LAMBDA
- needs to have IAM EXECUTION ROLE with proper policy (AWSX-RayWriteOnlyAccess)
- MAKE SURE CODE IS IMPORTED AND INTEGRATION IS ENABLED ON LAMBDA
X-ray Instrumentation in code
Instrumentation :
- measure of performance
- Diagnose errors
- write trace information
Instrument an AWS application via, XRAY SDK
Many SDK require only requires config changes
Modification of app code to customize annotation of SDK data sent to Xray VIA:
Interceptors
Filters
Handlers
Middleware
*takeway: you can customize how x-ray works via code configuration changes.
XRAY Segments Subsegments trace sampling annotations metadata annotation vs metadata
Segments: Each application and service send them. how we see things in UI
Subsegments: Give more details in segments
trace: aggregation of segments, show end to end trace of entire API call
Sampling: Decrease amounts of requests sent to Xray to reduce cost
Annotations: KeyValue pairs used to INDEX traces and use with FILTERS (able to search traces with NEW indexes)
Metadata: Keyvalue pairs, NOT INDEXED, NOT used for searching
Xray daemon send trace accross account:
make sure IAM permissions are correct, agent assumes role
allows central account for all app tracing
XRAY SAMPLING RULES
Default
Sampling rules
Control amount of data you record, can be modified without changing rule.
*CHANGING RULES IN CONSOLE WILL NOT NEED CODE MODIFICATION
BY DEFAULT:
SDK records first request each second**, then five percent* of additional requests.
**One request per second: RESIVOIR. ensures at least one! trace is recorded each second as long as the service is serving requests.
*Five percent is: RATE; rate at which additional requests beyond the reservoir rate are sampled.
XRAY CUSTOM SAMPLING RULES
Define custom
RESIVOIR and RATE
example
Reservoir:10 Rate:0.10
10 requests per second will be sent to x-ray, then also 10% of rest
Reservoir:1 rate 1.0
one will be sent per second, and also 100% of rest.
*CHANGING RULES IN CONSOLE WILL NOT NEED CODE MODIFICATION
X-RAY API X-Ray WRITE PutTraceSegments: PutTelemetryRecords GetSamplingRules GetSamplingTargets GetSamplingStatistics
PutTraceSegments:
Upload segment document into X-ray
PutTelemetryRecords: X-ray daemon, upload information of metrics. Segmentsreceivedcount: Segments received Segmentrejectedcounts: Segments rejected BackendConnectionErrors: backend errors
GetSamplingRules: Retrieve all sampling rules
GetSamplingTargets: related to above
GetSamplingStatistics related to above
**Get rules allow Daemon to understand sampling changes done in console, these changes in console do not need to be changed on each machine’s code.
FOR API CALLS TO WORK:
X-ray Daemon needs IAM authorization for these calls to function correct.
X-RAY API
X-Ray READ
GetServiceGraph:
BatchGetTraces:
GetTraceSummaries:
GetTraceGraph:
GetServiceGraph: Main graph in console
GetTraceSummaries:
Get ID and Annotations for traces available for TIME:
for full traces pass this into BatchGetTraces
BatchGetTraces: FULL TRACES
*Retrieves a list of traces specified by ID. Each trace is a collection of segment documents that originates from a single request. Use GetTraceSummaries to get a list of trace IDs.
GetTraceGraph: Get a service graph for for one or more trace ID’s
TRACE ID:
A trace ID tracks the path of a request through your application. A trace collects all the segments generated by a single request. That request is typically an HTTP GET or POST request that travels through a load balancer, hits your application code, and generates downstream calls to other AWS services or external web APIs. The first supported service that the HTTP request interacts with adds a trace ID header to the request, and propagates it downstream to track the latency, disposition, and other request data.
XRAY ECS
- ECS cluster container as daemon
- ECS cluster container as sidecar
- ECS Fargate cluster sidecar
- ECS cluster container as daemon
in every EC2 instance we have multiple App containers, these app containers will be mapped through network configuration and UDP routing to a daemon XRAY container separate
EX 2 containers connect to same daemon shared in one ec2 instance - ECS cluster container as sidecar
In one Ec2 instance we have multiple containers that are linked to an X-ray Sidecar, each container has its own x-ray daemon - ECS Fargate cluster sidecar
Each fargate task launches a sidecar with an Xray Daemon.
XRAY ECS
TASK DEFINITON
SIDECAR
Hostport: 0
containerport: 2000
protocol UDP
Enviroment variable
AWS_XRAY_DAEMON_ADDRESS: variable that allows SDK to find Xray daemon
Value: is Xray-daemon:2000
Links: X-ray Daemon, : allows containers to map above
process
- map container port to 2000 udp
- set environment variable
- link via networking
XRAY BEANSTALK
methods to enable
AWS beanstalk comes with X-ray can be enabled two ways 1. Set option in elastic beanstalk console 2. with config file placed in .EBextensions/xray-daemon.cfg
AWS:elasticbeanstalk:xray
Xrayenabled: True
Make sure to give instance profile correct IAM permissions so that X-ray Daemon can function correctly.
Application code in instrumented with the X-ray SDK: add sdk capability in app code.
DAEMON DOES NOT WORK SAME IN MULTIDOCKER CONTAINER!
