5 - Preparing to Model the Data

Question 1

What is the first phase discussed in Chapter 3 of the Data Science Methodology?

Answer 1

Problem Understanding Phase

Question 2

What are the important tasks in the Setup Phase?

Answer 2

• Partitioning the data
• Validating the data partition
• Balancing the data
• Establishing baseline model performance

Question 3

What is the primary reason data science does not use the statistical inference paradigm?

Answer 3

Statistical significance can occur without practical significance in large sample sizes

Question 4

What is data dredging?

Answer 4

Uncovering spurious results due to random variation rather than real effects

Question 5

What technique helps avoid data dredging?

Answer 5

Cross-validation

Question 6

What are the two common methods of cross-validation?

Answer 6

• Twofold cross-validation
• K-fold cross-validation

Question 7

What does the training data set contain in a twofold cross-validation?

Answer 7

Records with complete data

Question 8

What is the purpose of the test data set in cross-validation?

Answer 8

To evaluate predictions against true target values

Question 9

What percentage of data is typically used for training in complex data sets?

Answer 9

75–90%

Question 10

What command is used to partition data in Python?

Answer 10

train_test_split()

Question 11

What function is used in R to set the random number generator seed?

Answer 11

set.seed()

Question 12

What statistical test is used for numerical variables to check for differences between training and test sets?

Answer 12

Two-sample t-test

Question 13

What is the purpose of balancing the training data set?

Answer 13

To provide a rich selection of records for each category

Question 14

What is resampling?

Answer 14

Sampling at random and with replacement from a data set

Question 15

What should the test data set never be?

Answer 15

Balanced

Question 16

What is the formula to calculate the number of resampled records needed?

Answer 16

x = (p * records) / rare

Question 17

In Python, what command is used to check the count of a specific response in the training data set?

Answer 17

value_counts()

Question 18

What command is used in Python to concatenate two data sets?

Answer 18

pd.concat()

Question 19

What does the output of bank_train_rebal[’response’].value_counts() show after rebalancing?

Answer 19

The new counts of ‘yes’ responses in the rebalanced training set

Question 20

What is the purpose of validating your partition?

Answer 20

To ensure the training and test sets do not differ systematically

Question 21

What is the desired proportion of fraudulent transactions in a balanced training set example?

Answer 21

25%

Question 22

True or False: The test set should be balanced to improve model performance.

Answer 22

False

Question 23

Fill in the blank: The command used in R to find the number of ‘yes’ responses is _______.

Answer 23

table()

Question 24

What percentage of the training data set had ‘yes’ responses before resampling?

Answer 24

11%

This percentage indicates the initial distribution of ‘yes’ responses in the training dataset.

Question 25

What percentage of 'yes' responses do we aim for after resampling?

Answer 25

30% ## Footnote This target percentage is set to increase the representation of rare records in the training data.

Question 26

How many 'yes' records do we need to resample to achieve the target percentage?

Answer 26

850 ## Footnote This number is calculated based on the total records and desired proportion.

Question 27

What command is used to identify the indices of records with 'yes' responses?

Answer 27

which() ## Footnote This command returns the row numbers corresponding to records meeting the specified condition.

Question 28

What does the command sample(x = to.resample, size = 850, replace = TRUE) do?

Answer 28

It randomly samples 850 records with replacement from the specified indices. ## Footnote The replace = TRUE input allows for the same record to be selected multiple times.

Question 29

What is the purpose of the rbind() command in the context of resampling?

Answer 29

To append the resampled records to the original training data set. ## Footnote rbind() combines two datasets by stacking them vertically.

Question 30

What is the output of the table command after rebalancing?

Answer 30

A table showing the count and proportion of 'yes' and 'no' responses. ## Footnote This helps confirm the effectiveness of the resampling process.

Question 31

What must a model's accuracy exceed to be considered useful in the fraud detection scenario?

Answer 31

99% ## Footnote This is the accuracy of an 'all negative' model that classifies all records as non-fraudulent.

Question 32

What are the two baseline models for binary classification?

Answer 32

* All Positive Model * All Negative Model ## Footnote These models serve as a reference for evaluating the performance of more complex models.

Question 33

What is the accuracy of the All Positive Model?

Answer 33

p ## Footnote p represents the proportion of positive records in the dataset.

Question 34

What is the accuracy of the All Negative Model?

Answer 34

1 - p ## Footnote This represents the proportion of negative records in the dataset.

Question 35

What is the Biggest Category Model in k-nary classification?

Answer 35

Assign all predictions to the largest category. ## Footnote Its accuracy is pmax, the highest proportion of any single category.

Question 36

What is the y y model used for in regression?

Answer 36

It compares estimates against the mean response. ## Footnote This model serves as a baseline for evaluating regression performance.

Question 37

What is the optimal benchmark for calibrating model performance?

Answer 37

The current gold standard model performance. ## Footnote This benchmark is based on established literature or proprietary business models.

Question 38

What are the four tasks that should be undertaken during the Setup Phase?

Answer 38

* Partitioning the data * Validating the data partition * Balancing the data * Establishing baseline model performance ## Footnote These steps are essential for preparing data for analysis.

Question 39

True or false: There is no baseline model for k-nary classification.

Answer 39

False ## Footnote The Biggest Category Model serves as a baseline for k-nary classification.

Question 40

What is balancing in the context of data preparation?

Answer 40

The process of adjusting the dataset to ensure a more even distribution of response categories. ## Footnote Balancing is particularly important for handling rare events in classification tasks.

Question 41

What is resampling?

Answer 41

The process of selecting records from a dataset to create a new sample. ## Footnote This technique can be used to adjust the proportions of different response categories in the dataset.