20-anomaly detection

Question 1

What is an outlier / anomoly?

Answer 1

A pattern in the data that does not conform to the normal/standard/expected behaviour

Question 2

What are applications of anomaly detection?

Answer 2

Fraud detection
Ecosystem disturbance
Medicine and public health
Aviation safety

Question 3

What are the types of anomaly?

Answer 3

Point/global anomaly
Contextual/conditional anomalies
Collective anomalies

Question 4

What is a point / global anomaly?

Answer 4

An individual data instance is anomalous with respect to the data

Question 5

What are the two types of attributes in reference to anomaly detection?

Answer 5

Contextual attributes
Behavioural attributes

Question 6

What are collective anomalies?

Answer 6

A subset of data points are anomalous

Question 7

What is the difference between anomaly and noise?

Answer 7

Noise is random error and not interesting. Anomalies are interesting and

Question 8

What is supervised anomaly detection?

Answer 8

Labels are available for normal data and anomalies. Classifiers distinguish between normal data and anomalies

Question 9

What is semi-supervised anomaly detection?

Answer 9

Labels are only available for normal data. Model normal objects and report those not matching the model as outliers

Question 10

What are the challenges with semi-supervised anomaly detection?

Answer 10

Requires labels from normal class. Possibly high false alarm rate

Question 11

What is unsupervised point anomaly detection?

Answer 11

Proximity based, density based, clustering based, statistical anomaly detection

Question 12

What is statistical anomaly detection?

Answer 12

Anomalies are objects that are fit poorly by aW statistical model.

Question 13

What is proximity-based anomaly detection?

Answer 13

An object is an anomaly if the nearest neighbours of the object is far away. Compute distance between every pair of data points

Question 14

What are the ways to detect anomalies through proximity?

Answer 14

• Data points for which there are fewer than p neighboring points within a distance D
• The top n data points whose distance to the kth nearest neighbor is
  greatest
• The top n data points whose average distance to the k nearest neighbors is greatest

Question 15

What are the pros of proximity based anomaly detection?

Answer 15

Easier to determine proximity compared to statistical distribution
Quantitative measure of degree to which an object is an outlier

Question 16

What are the cons of proximity based anomaly detection?

Answer 16

O(n^2) complexity
Outlier score is based on k
Does not work well when there is variable density

Question 17

What is density based outlier detection?

Answer 17

Outliers are objects in regions of low density. Outlier score is inverse of density around object

Question 18

What are examples density scores?

Answer 18

Number of objects within distance. Inverse of average distance to k nearest neighbours

Question 19

What is cluster based anomaly detection?

Answer 19

Outliers are objects that don’t belong strongly to a cluster. Assess degree to which object belongs to any cluster. Use relevant distance to centroids

Question 20

What are the pros of cluster based anomaly detection?

Answer 20

O(n) complexity. Extends concept of outlier from single objects to groups.

Question 21

What are the cons of cluster based anomaly detection?

Answer 21

Requires distance thresholds. Sensitive to the number of clusters. Outliers may impact clusters.