ML Security

Question 1

How is AI different from ML?

Answer 1

AI focuses on decision-making, while ML focuses on learning how to perform tasks from data.

Question 2

What are some threats to ML models?

Answer 2

1. Evasion attacks
2. Data poisoning
3. Membership inteference
4. Model stealing

Question 3

How is ML applied in security?

Answer 3

Used for tasks like malware detection, spam detection, intrusion detection, fraud detection, and cyber defence.

Question 4

What is an adversarial example in ML?

Answer 4

Slightly perturbed inputs designed to fool ML models into making incorrect predictions with high confidence.

Question 5

What is a data poisoning attack?

Answer 5

Injection malicious data into the training set to alter the model’s behavior, such as shifting decision boundaries or facilitating adversarial attacks.

Question 6

What is a membership interference attack?

Answer 6

An attack where an adversary determines if a specific data point was part of the model’s training data, compromising privacy.

Question 7

What is model stealing?

Answer 7

An attack where an adversary replicates a model’s functionality by querying it and learning its behavior.

Question 8

Why do security challenges require moving beyond I.I.D. assumptions?

Answer 8

Attackers can craft inputs that are not independent or identical to training data, exploiting vulnerabilities.

Question 9

What is adversarial training?

Answer 9

A defense technique where adversarial examples are included in the training process to improve the model’s robustness against such attacks.

Question 10

Why is deep learning considered vulnerable?

Answer 10

It relies heavily on large datasets, which may not always be trustworthy, and is resource-intensive.

Question 11

What is a key challenge related to fairness in ML?

Answer 11

Ensuring the model is not biased against protected classes or groups.

Question 12

What is certified robustness in ML security?

Answer 12

It refers to the formally verified guarantees about a model’s resistance to adversarial attacks.

Question 13

What are some safety concerns with ML systems?

Answer 13

Issues include susceptibility to fake mesia (e.g. deepfakes) and ensuring fair treatment across demographic groups.

Question 14

Who are the main entities in an ML system?

Answer 14

Data providers, model trainers, model evaluators, and model users.

Question 15

What is the difference between discriminative and generative models in ML?

Answer 15

• Discriminative: Predicts the output y given the input x (e.g. classifiers)
• Generative: Models that joint probability distribution P(x, y) or generates data similar to the input distribution (e.g., image generation).

Question 16

Define the following terms in ML:
1. Model
2. Input
3. Output
4. Training Algorithm
5. Parameters

Answer 16

1. Function f_0(x) that maps input to output
2. Independent variable (e.g. x)
3. Dependent variable (e.g. y)
4. Process that adjusts parameters to minimize loss.
5. Tunable components of the model learned during training

Question 17

What is adversarial noise in ML?

Answer 17

Small, carefully crafted perturbations added to inputs that cause a model to make incorrect predictions while appearing unchanged to humans.

Question 18

Why is trusting training data important in ML security?

Answer 18

Untrusted data can lead to vulnerabilities such as data poisoning attacks, which degrade model performance or enable adversarial examples.

Question 19

What is an example of ML-generated safety concerns?

Answer 19

Fake content, such as deepfake videos or neural fake news, which can spread misinformation or harm credibility.