ML Concepts

Question 1

Feature crossing

Answer 1

• Create new features from existing features by taking their cross product
• A feature cross is a synthetic feature that encodes nonlinearity in the feature space by multiplying two or more input features together.

Question 2

Example of crossing two features such as country and language

f1: [USA, China, England]
f2: [English, Chinese]

Answer 2

Generates 6 new features
- USA-English
- USA-Chinese
- China-English

Question 3

Feature selection

Answer 3

• Goal is to reduce the number of features to only those most useful
• can use GBDT to select a subset of features based on their importance

Question 4

Feature extraction

Answer 4

Reduce the number of features by creating new features from existing ones. The new features have more predictive power.

Question 5

Unsupervised learning

Answer 5

no labels

Question 6

Dimensionality Reduction

Answer 6

Techniques for reducing the number of input variables in training data.

Question 7

Over-fitting

Answer 7

The model gives accurate predictions for training data but not new data

Question 8

Under-fitting

Answer 8

• Model is unable to capture the relationship between inputs and output variables accurately
• High error rate for both training data and unseen data

Question 9

Pros of Feature Crossing

Answer 9

• Captures pair-wise, second-order feature interactions

Question 10

Cons of Feature Crossing

Answer 10

• Requires human with domain knowledge to choose pairs
• Won’t capture all complex interactions
• If original features are sparse, the cardinality of the crossed features can become much larger leading to even more sparsity

Question 11

One hot encoding

Answer 11

• Technique used to represent categorical variables as numerical values in a machine learning model

Question 12

Advantages of one hot encoding

Answer 12

• It allows the use of categorical variables in models that require numerical input.
• It can help to avoid the problem of ordinality, which can occur when a categorical variable has a natural ordering (e.g. “small”, “medium”, “large”).

Question 13

What would a categorical feature for fruit (apple, mango, banana) and associated price look like using one hot encoding

Answer 13

A vector. The columns of the vector would be apple, mango, banana, and price. The fruit columns would contain one or zero. The price column would contain a numerical value.

Question 14

Disadvantages of one hot encoding

Answer 14

• One-hot-encoding is a powerful technique to treat categorical data, but it can lead to increased dimensionality (extra cols), sparsity (most cols are zero), and overfitting (poor predictions). It is important to use it cautiously and consider other methods such as ordinal encoding or binary encoding.

Question 15

Embeddings

Answer 15

• Way to encode a categorical feature
• Alternative to one-hot encoding which can generate very sparse vectors
• Map high-dim vectors to low-dim vectors
• Embeddings make it easier to do machine learning on large inputs like sparse vectors representing words.
• Ideally, an embedding captures some of the semantics of the input by placing semantically similar inputs close together in the embedding space.

Question 16

Feature Extraction

Answer 16

Create new features from existing ones for dimensionality reduction

Question 17

Model Categories

Answer 17

Supervised (labels)
Unsupervised (no labels)
Reinforncement learning (trial and error; this is how your roomba learns the shape of your living room)

Question 18

Types of supervised learning

Answer 18

• Classification and regression

Question 19

How do you handle missing values in training data?

Answer 19

• Feature imputation techniques
• Use defaults
• Use the mean, median, or mode
• regression imputation (predict values based on correlated features)
• k-nearest neighbor (use avg of nearest neighbors)

Question 20

What is embedding learning

Answer 20

Learning an n-dim vector for each unique value a categorical feature may take

Question 21

Upsampling

Answer 21

• Strategy to handle data with imbalanced classes by replicating or generating new samples from the minority class to achieve a more balanced distribution
• Reduces model bias for the majority class

Question 22

Why deep neural networks

Answer 22

• One layer is good enough for many tasks but that single layer might have to be impractically large
• Fewer workers
• Shallow networks often perform poorly on high dim spaces like natural language

Question 23

Downsampling

Answer 23

Strategy to handle data with imbalanced classes by randomly removing samples from the majority class

Question 24

Epochs

Answer 24

• During an epoch, the model sequentially processes each training sample, calculates loss, updates its parameters based on the gradients
• The number of times the model iterates through the entire training set