IBM / Instana Flashcards
How to uninstall and reinstall the agent?
- To debug this at your cluster, you have to use a kubectl version which match the cluster kubernetes-api.
-Then you could run the following commands to delete the Agent and install the Agent via your normal approach:
helm uninstall instana-agent -n instana-agent
-kubectl delete ns instana-agent
-kubectl apply -f PATH_TO_YAML
-wait until you see CrashLoopBackOff (I am using watch ‘kubectl get pods -n instana-agent’ to watch the namespace for changes) - Get the events for the pod with status CrashLoopBackOff via kubectl get event -n instana-agent –field-selector involvedObject.name=instana-agent-REPLACE_WITH_CORRECT_IDENTIFIER
-Also get the output from the describe command of this pod kubectl describe pod instana-agent-REPLACE_WITH_CORRECT_IDENTIFIER -n instana-agent
Where should you disable python autotracing?
In com.Instana.plugin.python section in the agent’s configuration.yaml file, you can disable the python autotrace for that agent. Either use an exclude that matches your application or disable the python autotrace for that agent.
Are gunicorn or uvicorn supported by Instana auto tracing?
No gunivorn support was removed because recent versions of gunicorn cannot reliably reload middleware once started. This can work once the Instana restores support for gunivorn via the webhook is ready.
And Uvicorn is meant to be used with a manual wrapper.
Context Guide
It is a GPS that shows you how the infrastructure works together with the architecture and business logic and you will get only the relevant data.
What does agent do?
Agent automates issue detection, discover of technology, metric collection, and tracing.
The agent is the hub where agents deliver telemetry and trace info to.
Instana agent uses multiple sensors (there is a Java sensor) that initiates a discovery data collection and checks every 20 seconds
For node what do you need to install?
One line of code
What is the fastest cadence to see spikes?
1 second timeframe
What does the back-end of the agent do?
It collects data, does its calculations, and correlate dependencies among applications, services, and end points.
How are Instana tenants usually partitioned?
Tenants are usually partitioned into three units: production, training, staging - all of which reside in the same tenant.
Can all of the tenant units spare the same config?
Yes.
What settings can each of the tenant units be configured with different settings?
Different user permissions, agents, group memberships, and applications.
How to deploy agent from Instana Dashboard
Click Deploy Agent to walk through a dialogue on deploying an agent based on the environment and install a one-liner.
How often does Instana release new features?
Every two weeks
Where can you find new Instana releases and read about old releases?
Http://www.Instana.com/docs/releases
How does the dynamic agent check for updates and what is the exception deployed in China?
Once we push a dynamic update to a stable branch, the agent will check the repo for new updates by 4:30am if there are new updates. It does this for all units across the tenants.
In China they use a static agent, which comes with all of the info and doesn’t dynamically download from the repo because if they did there would be latency and bandwidth issues from China downloading from repos residing in Europe.
Can you favorite each section of the Instana dashboard, can you share it?
Yes, you can favorite it and highlight on application perspective and share it
What can you see on the Infrastructure page and how can users organize in groups and what is the auto organization setting?
On the Infrastructure page, you will see hosts monitored by Instana.
User have put hosts in zones but you can override the zones.
Instana auto groups it by server zone.
How can you customize the Infrastructure UI by containers?
Lower right-hand corner to change the views. You can zoom in or zoom out.
What does red, yellow, or green show on the infrastructure ui?
Red, yellow, or green show health of the host/container when Instana backend has over 200 health rules.
How do you search for the host/container?
You can search for it on the comparison table that by default shows all of your hosts.
What is the second column of the comparison table?
Click on the second column and then you will be brought to a dashboard for that host and see what agents are collecting and measuring for you.
What do agents collect and Instana visualize on dashboard?
It will collect and visualize data on context switches, cpu load, etc. and you will see what you are actually running with that host, so called tags, data on network interfaces, Amazon EC2 instances, spring boot application, etc.
What does a time slicer do on a graph and what does this help you do?
Use the mouse to get all highlights on all of the chart drop down or zoom to timeline. This helps you correlate the same time on all graphs and you can share intervals of a graph with a short form url.
What two ways do you find Kubernetes data?
To find the Kubernetes namespace page, You can see label, spec, and pods, memory resource, annotations, events, etc.
Second way is click on the Kubernetes icon on the left hand side and you will see all of your Kubernetes clusters. For each Kubernetes dashboard, there is a stack button and upstream/downstream button.
What does the stack button do and upstream/downstream button do?
Instana gives you relevant full stack overview for this cluster and you can see all of the application and calls there.
For upstream/downstream, it will show you who is sending calls to this cluster?
What do you type in the search bar to find processes for garbage collection?
Type in entity.type.jvm in the search bar and it will find processes for garage collection.
Besides search bar, how else do you find the garbage collection section?
If you click on spring boot application, there is a garbage collection section.
How does one know what we support?
You you go to the supported technologies page on Instana, it will show you what frameworks, platforms, tracing, sensors, jvms, libraries, which metrics we support.
How is an “application” defined?
An application is defined as always dynamic and defined by the user.
Application is the 3 circle icon.
What is services on Instana?
Services are what Instana automatically collects and they are based on a library of rules.
*Service names are derived via Instana by the predefined rules
What can you do on the predefined rules page?
On the predefined rules page, if it will hit one of these predefined rules, it will stop processing the data and start applying the predefined rule for that service.
Service names are derived via Instana by the predefined rules
How to find out how to Install Instana?
Go to the “Install Instana” page on Instana’s documentation.
Why self hosted for Apple?
For security and hipa reasons, They want to hoard and lock their data not send their own data. They don’t want our agents to send data outside of their environment.
Do we need to set up mounts?
No, customers do and customers need to setup mount points and external volumes:
- Data stores (defaults to /mnt/data
- Cassandra, should not utilize the same volume for production installs (defaults to /mnt/metrics)
-Clickhouse, should not utilize the same volume for production installs (defaults to /mnt/traces)
-Logs (defaults to /var/log/instana). If required you can create a different directory.
How is tenant units structured in a url?
{unit}-{tenant}
What is the lowest level of endpoints?
Database
How to create a license?
Go to main dashboard and go to upper right-hand corner and click on “Create License” and select “NFR-Free License”.
What does the Instana 3 Step wizard for application perspectives do?
*Instana has a 3 step wizard for an application perspective. Advanced mode allows you to pick tags, which can be a specific call or params. You can also define level of app like no downstream is jvm level and nothing further, then pick top option. If you want downstream db and messaging service, then pick middle option. For all hubs from there, pick last option.
What can you configure application perspective to do?
You can configure application perspective that shows only calls from a source.
And You can change the flow maps on the top so that it does it for In order for Instana to detect downstream services.
What are the issues with tracing or tracking Kafka?
*Sometimes you see tech going into Kafka but not going out and this happens when Instana doesn’t monitor this tech service. Or this can also happen when the trace going into Kafka is not correlated with the trace going out. A good question is to ask if Instana can figure out if these calls belongs to this trace.
What happens to the Infrastructure ui once you restart the hosts?
The “infrastructure” sub-tab will show all containers start up simultaneously if the host restarts
Log messages sub tab
*There are “log messages” sub-tab. You can look up all calls with a particular error message. Same idea with ‘error messages’.
Error message sub-tab
In the sub “error messages” tab, if you click on a sub-error, then you will see all the calls causing this error message. It goes to the analytics page. The analytics icon is the magnifying glass icon. Each call is correlate to a user accessing that call. The trace was a post and delete statement. Then you find more logs of the stack trace in the lower right hand corner. We can also see what the profiling is of this call and see cpu, memory, wait time hotspot every few minutes by technology profiling. You can deep dive into metrics by technology service. For Java, it is always on. Try the flame graph
Why disable certain default events when Kubernetes is decommissioned?
*If you are running in kubernetes, kubernetes will spin up a new pod. But if not, Instana won’t automatically know it is decommissioned. You can disable certain default events
What does system rules->offline and detection do?
System rules->offline and detection will check if entity is still there if host go offline.
You can filter out for technology process criteria under system rules.
What do you on the Settings->Alert page?
On Settings->alert page, you can assign events to each alert
How many ways can you send an alert?
• *They send web hook, slack, email, etc. an alert can send for multiple events
What is a scope for an alert?
scope is how often alerts will trigger
What does the agent management page do?
Agent Management page you can view memory and cpu resources of the agent
• *Triangle icon
Events
•*How do you search calls by latency?
Analytics->Applications-> Calls: you can search calls by latency
How is to create alert button
• On a service page, there is a create alert button
What is the advanced mode to configure alerts?
You can also set advance mode to configure alert to average performance across some or all calls and services. If your application overnight has less traffic, you have the ability to config alerts less often and reduce the false positives.
What are the training resources?
Http://www.instana.com/training-resources
To instrument inside the method for Instana
To instrument inside the method, you can either add annotation inside the method and in the code and have Instana agent, like in the yaml file, that defines the method/class name and capture the return value and parameters.
The agent can also capture the stack trace
What should you put in a yaml file?
If there is a class within a class, you need to define it in the yaml file.
yaml is a hard deployment where if you change anything you have to restart agent on production.
You can add/edit entry, span, and edit types in yaml.
Apple engineers call this load testing at the method level.
In the future, Ted said instana engineers are trying to build a sensor to auto detect this (what?) so we would have to use a config yaml file
Where do you find the analyze->call feature and what does it do?
• On service page, there is Analyze Calls button that brings to Calls page grouped by endpoint. If you filter this page for browser, you can group by end point
Is there a doc for configuration for Java trace?
There is an Instana doc for “configuration for Java trace’”.No shotgun or over instrumentation like at Dynatrace
What does it mean that Instana can’t do constructors for Java?
• Can’t do for constructors for Java
Why does Instana link everything to the infrastructure?
We link everything to the infrastructure that is running the call so makes it easier for us to track the root cause analysis
How do you learn more about custom http with Instana?
• *There is a Custom HTTP Headers instana document
Does Instana support open jdk?
*We are supporting open jdK.
When was Instana founded and why was it founded?
• Instana was started in 2015. Our founder, Mirko, was working with working with our competitors that there were great difficulties getting the current visibility tools to work and the current problems didn’t solve the problem of visibility into dynamic applications
• They noticed that people want to do things quickly. Before, you had to stop multiple times on a Kansan sprint to monitor
What does the dynamic graph do?
• Dynamic graph pin underpins Instana. A lot of things can go wrong and we want to get you to the root cause as soon as possible
What do sensors do?
• The sensors digest and collect data, then it sees if data meets auto health rules, and you can set your own. Based on info, it decides if we are going to send out events
Which 4 type of metrics do we collect and how is Instana different?
We collect these 4 types of metrics: metrics, events, configuration, traces. A lot of other app tools do not collect info on configuration tool
How come there isn’t data retention beyond a year?
• *We are discussing internally if we are gonna do a life contract. We notice that people don’t look beyond a year
What does infrastructure map use?
• *Our infrastructure map uses open gl
What does the time picker do?
• Time picker is sticky. We will have a release tab as well. You have a calendar picker at the bottom
Can you test agents locally?
• You can install instana agents manually and test locally
What two values does the agent need for the license and where does it go?
Sales key and agent key and it goes into the HCL file
What is the difference between an on prem license and a saas license?
When creating a saas license from portal, it goes directly to the dashboard. When creating an on prem license, it gives me a license
How do you telnet an Instana on google cloud?
Gc_shh ted_feyler Ted robotshop Telnet the external ip/onpremip 80
If you are running an onprem backend, it is
Telnet localhost 80
Whenever you are in the Linux server of the cloud instance you need to go to /opt/Instana for aws and do a “telnet localhost 80” command or do a “telnet localhost 443”.
You can edit /etc/host file for what?
To ping a name rather than an ip address
How to install Kubernetes
Different ways like operator, helm chart, yaml, etc.
What is helm?
It is a package manager for Kubernetes and half of our customers already use helm and 40% use yaml, operator is the new cool kids
Why yaml over helm?
Yaml has everything in one file and helm splits it across different charts
On GCP, what do you need to run an agent?
Stand up a virtual machine and a cluster on gcp
What to enter on gcp form to create a cluster and how many nodes does it come with and do you pick autopilot or standard mode?
Cluster name, zone
It comes with 3 nodes
Standard mode is good
How to get command line script for gcp and do you need to install anything first?
Checkbox the cluster and right-click and hit “Connect”.
Install the gcloud command first
What does Kubernetes install on your computer and in which directory?
It installs a ./kube directory in your home computer and inside ./kube directory it installs cache/, config, and http-cache folders and these folders run Kubernetes
How to run kubectl
Install kubectl
Cd into .kube
Kubectl get pods
You need to point at right cluster by
“Kubectl get pods -n Instana-agent”
What is another name for yaml?
It is daemonset
How to apply yaml file?
Kubectl apply -f Instana-agent.yaml
Whenever you should edit the yaml file what should you do or else
It will fail?
Kubectl delete -f Instana-agent.yaml
Then
Kubectl apply -f Instana-agent.yaml
What is the command to get pods?
kubectl get pods -n instana-agent automatically
What command will allow you to list all of the yaml files?
l *.yaml
What happens on the dashboard after you delete the yaml?
The cluster name also is deleted
How does deploying Instana on Kubernetes make people’s lives easier
One way is you have to run only one command like installing the agent and it does this for 500 nodes
What port does robotshop run locally if not kubernetes?
Port 8080
What command can you find external ip address that is listening to port 8080 for robot shop and what do you do for the external ip?
Kubectl get svc web -n robot-shop
It is http://externalip.com
How do you edit the Kubernetes web definition document to double check what port it is listening on?
Kubectl edit svc web -n robot-shop
What is Instana’s elevator pitch?
Instana is AI powered APM solution that automatically monitors containerized micro service applications and solves the problem of visibility into dynamic application
What are 6 key concepts of Instana?
-Auto Discovery
-1 second granularity
-100 percent tracing
-Comprehensive dependency graph
-Easy to focus & filter
-Machine learning to automate issue detection
How does automated discovery process of all technology works?
You deploy one agent for all languages and it will auto detect technologies in that host or cluster and then reaches out to a remote Instana repo to download and deploy sensors but the sensors have different specialities like the mongodb sensor, nodejs sensor. Every 4am, Instana agent will reach out to the remote repo to see if there are updates to the sensor.
How do you find out the sensors each Instana agent has downloaded?
Go to Manage Agents and then click on sensors
What type of sensors do each languages has downloaded?
-Automatic metric collection sensor at 1 second granularity
-Automatic tracing
Which 4 languages don’t have automatic tracing sensors and what do you do if customers are missing tracing?
Nodejs
Golang
Ruby
Crystal
Point them to documentation
Dynamic graph
Instana’s internal model that continuously correlates dependencies between traces in the code and the infrastructure “. Instana’s entire ui is built on the graph.
What are the boxes on infrastructure page and what do the height of the boxes represent?
They represent hosts and the height of the box represents horsepower
What is the new and better way to navigate through the dynamic graph in the Instana ui?
It is the Context guide or the grey stack and upstream downstream buttons. In drop-down you can drill down through all of the entities and correlated metrics
How to use telnet
Telnet {url or localhost} (port number)
How does Instana reduce footprint in network transmission?
There are some optimization techniques on metric side. We don’t send the same value in every time. If your host cpu value, we hit 10 percent. If it is still 10 percent we don’t send a new value. If it goes to 11 percent we won’t send in a new value.
Can users configure the built in events at Instana and how do you find the built in health rules?
No and there is Instana documentation about the built in health rules
You can workaround it and disable built in events and create a custom event around your unique thresholfd
What is a simplified way of capturing metrics vs. traces?
Metrics capture for infrastructure and traces capture for application on top of infrastructure
What is 5-prong metric data retention policy of Instana?
1 second granularity is stored for last 24 hours. We have 100 percent of traces in last 24 hours up to last 7 days.
5 seconds for 24 hours
60 seconds for 1 month
5 min for 3 months
1 hour for 13 months
What is the traces data retention for Instana?
100 percent full traces for last 7 days
Significant calls and kpis for 1 or 13 months or more.
We keep the significant not the fast traces. All customers should focus on the last 7 days of traces but nothing is guaranteed.
How long do you keep dynamic graph configuration and events configuration?
Life of contract
What is a call?
A call described an activity within a monitored process, typically a request between two services
What is a trace?
A trace is imposed of one or more calls
Yes does Instana do distributed traces?
We trace different apps across different hosts and clusters
When are traces analyzed?
In real time
How does Instana know what the services and endpoints are?
They are automatically discovered
And relationships to other services, endpoints, and infrastructure are automatically correlated and stored in the dynamic graph
When are kpis for thoughpit, latency and error rate calculated?
On the fly
