Introduction To Anomaly Detector Flashcards
Introduction to anomaly detector:
Discover the services available on azure to detect anomalies in your time series or real-time data
Anomaly detection turn on new line anomaly detection is an artificial intelligence technique used to determine where the values in a series are with unexpected parameters.
There are many scenarios where anomaly detection is helpful for example a smart hvac system ideas anomaly detection to monitor temperatures in a building and raise an alert if the temperature goes above or below expected value for a given period of time
Other scenarios include:
Monitoring blood pressure Evaluating mean time between failures for hardware products
Comparing month-over-month expenses for product costs
As you’re normally detector service is a cloud-based service that helps you monitor and detect abnormalities in your historical time series and real-time data.
What is an anomaly detector of?
Anomalies are values that are outside the expected value or range of values.
Anomaly detection is considered the act of identifying events or observations that differ in a significant way from the rest of the data are being evaluated.
Accurate anomaly detection leads to prompt troubleshooting which helps to avoid revenue loss and maintain brand reputation
Address anomaly detected service:
Anomaly detected is a part of the decision services category within as a cognitive services.
It is a cloud-based service that enables you to monitor time series data and to detect anomalies in that data. It does not require you to know machine learning.
You can use the rest API to integrate anomaly detector into your applications with relative ease. The service uses the concept of a one parameter strategy.
The main parameter you need to customise is sensitivity which is from 1 to 99 to adjust the outcome to fit the scenario. The service can detect anomalies in historical time series data and also in real-time data such as streaming input from iot devices sensors or other streaming put sources
How anomaly detector works:
The anomaly detector service identifies anomalies that exist outside the scope of a boundary.
The boundary is said using a sensitivity value. By default the upper and lower boundaries for anomaly detection are calculated using concept known as expected value of Upper margin and lower margin.
The upper and lower boundaries are calculated by these three values. If a value exceeds either boundary it will be identified as an anomaly.
You can adjust the boundaries by applying a margin scale to the upper and lower margins.
Upper boundary= expected value plus (100 - marginScale) x upper margin
Data format:
The anomaly detector service accepts data in json format. You can use any numerical data that you have recorded over time.
The key aspects of the data being sent in occludes the granularity a timestamp and a value that was recorded for that timestamp.
And example of Jason object that you might send to the API is shown in a code sample. The granularity is set as hourly and is used to represent temperatures in degrees Celsius that were recorded at the time stamps indicated
Granularity set two-hourly series
Timestamp is 2021-over three-other credentials common
Value value is set to -10.56
Timestamp new information
Value new information
Continues like this for two more times
The service will support a maximum of 8,640 data points however sending this many data points in the same Jason project can result in latency for the response. You can improve the response by breaking your data points into smaller chunks or windows and sending these in a sequence.
The Jason object format is used in a streaming scenario. The main difference is that you will send a single value in each request. The streaming detection method will compare the current value being sent and the previous value sent.
Data consistency recommendation:
Notes-the anomaly detector service will provide the best results if your time series data is evenly distributed. If the data is more randomly distributed you can use an aggregation method to create a more even distribution dataset.
If your data may have missing values in the sequence consider the following recommendation:
Sampling occurs every few minutes and has less than 10% of the expected number of points missing. In this case the impact should be negligible on the detection results.
If you have more than 10% missing there are options to help fill the dataset full-stop consider using a linear interpolation methods to fill the missing values and complete the dataset. This will fill gaps with evenly distributed values.
When to use anomaly detectors:
The anomaly detector service supports batch processing of Time series data and last point anomaly detection for real-time data.
Batch detection carillon
Batch detection involves applying the algorithm to an entire data series at one time.
The concept of Time series data involves evaluation of a dataset as a batch. Use your time series to detect any anomalies that might exist throughout your daughter. This operation generates a model using in your entire time series data with each point analysed using the same model. When using batch detection mode anomaly detector create a single statistical model based on the entire data set that you passed to the surface. From this model each data point in the data set is evaluated and anomalies are identified.
Batch detection is best used when your data contains:
Flat train time series data with occasional spikes or dips
Seasonal time series data with occasional anomalies
Seasonality is considered to be a pattern in your daughter that occurs at regular intervals full-stop examples would be hourly daily or monthly patterns. Using seasonal data and specifying a period for that button can help to reduce the latency in detection.
Batch detection anomaly example
ie. Where items need to be stored at specific temperatures the temperature and we have to evaluate whether the items remain stored in a safe temperature range over the past months:
The minimum allowable temperature
The maximum allowable temperature
The acceptable duration of time for temperatures to be outside of the safe range.
If you are interested in evaluating compliant service oracle readings you can extract the required time series starter packaged it into a Jason Aldridge and send it to the anomaly detected service for evaluation. He will then have as oracle view of the temperature readings overtime.
Real-time detection turn on new line real-time detection uses streaming data by comparing previously seen data points to the last data point to determine if your latest one is an anomaly.
This operation generates a model using the data points you send and determines if the target current point is an anomaly.
By calling the service with each new data point to generate you can monitor your data as it’s created.
Real-time detection example Consider a scenario in the carbonated beverage industry with real-time anomaly detection may be useful.
The carbon dioxide added to soft drinks during the bottling and Canning process needs to stay in a very specific temperature range. You can use the anomaly detector service to create a monitoring application configured with the above criteria to perform real-time temperature monitoring.
You can perform anomaly detection using both streaming and watch detection techniques.
Streaming detection is most useful for monitoring protocol storage requirements that must be acted on immediately.
Sensors or monitor the temperature inside the compartment and send these readings to your application or an event our born as your. Anomaly detector will evaluate the streaming server the streaming data points and determine if a point is an anomaly
Seasonal data indicates data occurring at regular intervals
Seasonal time series is considered to be a pattern in your data that occurs at regular intervals. Examples would be hourly daily or monthly patterns
