Dimensionality Reduction (Unsupervised Learning)

Question 1

Dimension

Answer 1

Dimension is an attribute of an instance

Question 2

Another name of Dimension

Answer 2

Variable or Feature or Attribute

Question 3

Dimension Set

Answer 3

Set of attributes used to describe an instance

Question 4

Another name of Dimension Set

Answer 4

Feature set or Attribute set

Question 5

Feature Importance in Dimensionality

Answer 5

When an instance is described with a dimension set, then all the dimensions are not important to describe the instance.

Question 6

What happens if we use all possible attributes to describe an instance in machine learning?

Answer 6

Using all possible attributes can lead to overfitting.

Question 7

When does model goes to overfitting (in terms of attributes)

Answer 7

Model goes to overfitting if we use all possible attributes to describe an instance in machine learning

Question 8

What happens to computation time and efficiency when dimensions increase in machine learning?

Answer 8

When dimensions increase, computation time increases and efficiency decreases.

Question 9

What challenge arises when dealing with higher dimensions in data analysis?

Answer 9

When the dimensions are more, it is very difficult to plot the data and analyse the data

Question 10

How is data represented in different dimensions, and what challenge arises with higher dimensions?

Answer 10

• 1-D data is represented with a dot (.)
• 2-D data is represented with x-axis and y-axis
• 3-D data is represented with x-axis, y-axis, and z-axis
• Data with more than 3 dimensions is very difficult to plot.
  To address this, dimensionality reduction techniques are required to effectively train models in high-dimensional spaces.

Question 11

What is Dimensionality Reduction.

Answer 11

Dimensionality Reduction is the process of reducing the dimensions of input dataset under the consideration by selecting a smaller set of principal attributes.
–Here, consideration criteria is minimum attribute selection
–Here, Principal attribute is an attribute which contains minimum error

Question 12

Advantages of Dimensionality Reduction

Answer 12

• Decreases the complexity of an algorithm.
• Saves time by eliminating less important input variables.
• Allows simpler models to train the machine.
• Makes it easier to plot and analyze data.
• Simplifies knowledge extraction (i.e., identifying prediction patterns).
• Reduces computation time.
• Increases the efficiency of machine predictions.

Question 13

Dimensionality Reduction methods

Answer 13

Two methods
1.) Feature selection
2.) Feature extraction

Question 14

What happens in feature selection

Answer 14

‘k’ dimensions are selected from given d-dimensional set, meaning (d-k) dimensions are discarded

Question 15

Compare the efficiency of feature selection and feature extraction

Answer 15

Feature selection is less efficient than feature extraction

Question 16

Why feature selection is less efficient than feature extraction

Answer 16

Feature selection doesn’t cover all the dimensions given in the input set

Question 17

Which method is used in feature selection?

Answer 17

Subset selection method is used in the feature selection

Question 18

Type of Subset selection algorithms

Answer 18

1.) Forward Selection
2.) Backward Selection

Question 19

Forward Selection

Answer 19

It starts with an empty subset (No variable) and adding the variables one by one and select the subset which decreases the error the most that can’t decreased further by adding another variable

Question 20

Backward Selection

Answer 20

In this selection, initially take the complete set (full set). Later delete/remove the variables one by one till we decrease the error the most, that can’t be decreased further by removing the other variables.

Question 21

Advantages of forward selection(3)

Answer 21

1.)Faster for Large Datasets (since it starts with an empty set of features)
2.)Good understanding of the incremental benefits after adding each feature.
3.)Less Computationally Intensive compared with backward selection

Question 22

What is a key advantage of forward selection when dealing with large datasets and why?

Answer 22

It is faster for large datasets since it starts with an empty set of features.

Question 23

How does forward selection help in understanding model performance?

Answer 23

It provides a good understanding of the incremental benefits after adding each feature.

Question 24

Which is less computationally intensive (Forward selection or backward selection) and why?

Answer 24

Forward selection starts with an empty set and adds features gradually, making it less computationally intensive than backward selection, which starts with all features and removes them.

Question 25

Disadvantages of forward selection(2)

Answer 25

1.) It is computationally expensive when data set is very large
2.) May be miss out the interaction between the features those are more relevant

Question 26

What is a major disadvantage of forward selection with very large datasets?

Answer 26

It can be computationally expensive when the dataset is very large.

Question 27

What is a potential issue with feature interactions in forward selection?

Answer 27

Forward selection may miss out on interactions between features that are more relevant.

Question 28

Advantages of Backward selection(2)

Answer 28

1.) We can identify and remove the irrelevant attributes in the early stage
2.)We can consider the interactions between the features since it starts from full set

Question 29

What is an advantage of backward selection in terms of irrelevant attributes?

Answer 29

It allows us to identify and remove irrelevant attributes in the early stages of the model-building process.

Question 30

How does backward selection address feature interactions?

Answer 30

It considers the interactions between features since it starts from a full set of attributes.

Question 31

Disadvantages of backward selection(2)

Answer 31

1.)It is generally slower for the large data set because features are more in the large data set
2.)More computationally intensive when the data set is large

Question 32

What is a key disadvantage of backward selection when dealing with large datasets?

Answer 32

It is generally slower for large datasets because there are more features to evaluate.

Question 33

Why is backward selection considered computationally intensive for large datasets?

Answer 33

It requires high CPU usage, large memory consumption, extended execution time, high storage requirements, and may involve parallel processing.

Question 34

Feature extraction

Answer 34

• In this method,’k’ new dimensions are derived with the combination of ‘d’ dimensions given in the input data set
• In this method, k value is compulsory less than d value, means new dimensions are less than original dimensions
• In this method, all the input dimensions are covered in the new dimension. Therefore, it is more efficient than feature selection
• PCA(Principal Component Analysis) technique is used in the feature extraction

Question 35

What happens to the dimensions in feature extraction?

Answer 35

‘k’ new dimensions are derived by combining the ‘d’ dimensions given in the input dataset.

Question 36

In feature extraction, how does ‘k’ compare to ‘d’?

Answer 36

The value of ‘k’ is always less than ‘d’, meaning the new dimensions (k) are fewer than the original dimensions (d).

Question 37

Why is feature extraction more efficient than feature selection?

Answer 37

Because all input dimensions are covered in the new dimensions, making it more efficient.

Question 38

Which technique is commonly used in feature extraction?

Answer 38

Principal Component Analysis (PCA) is used in feature extraction.