8.1 Data Analysis Steps

Question 1

What is the main difference between a “Traditional ML model” versus a “Textual ML model”?

Answer 1

A Traditional ML model uses data inputs; a textural ML model uses text inputs
.

Question 2

What are the 5 steps involved in analyzing data for financial forecasting?

Answer 2

1. Conceptualization of the modeling task.
2. Data collection.
3. Data preparation and wrangling.
4. Data exploration.
5. Model training.

Question 3

Data collection (or “________”) involves determining the sources of the data to be used.

Answer 3

curation;

Question 4

Data _________ deals with reducing errors in the raw data.

Answer 4

cleansing;

Question 5

Data _________ involving processing data for model use.

Answer 5

wrangling;

Question 6

Addressing the following are attributes of what process?

• Missing values;
• Invalid values (i.e., data outside of a meaningful range);
• Inaccurate values;
• Non-uniform values due to wrong use of format or unit of measurement;
• Duplicate observations;

Answer 6

data cleansing;

Question 7

Addressing the following are attributes of what process?

• Extraction (e.g., extracting number of years employed based on dates provided);
• Aggregation (consolidating two related variables into one, using appropriate weights);
• Filtration (removing irrelevant observations);
• Selection (removing features, e.g., data columns, not needed for processing);
• Conversion of data of diverse types (e.g., nominal, ordinal, etc.).

Answer 7

data wrangling;

Question 8

What is the term for removing outliers whereby the highest and lowest x% of observations are excluded?

Answer 8

trimming;

Question 9

What is the term for replacing extreme values (e.g., a person’s height) by the maximum value allowable for that variable?

Answer 9

winsorization;

Question 10

Which scaling method scales variable values between 0 and 1 in order to describe them as a normal distribution, and what is the formula?

Answer 10

normalization;

normalized xi = [x - min(x)] / [[max(x) - min(x)]

Question 11

Which scaling method centers the variables at 0 and scales them as units of standard deviations from the mean, and what is the formula?

Answer 11

standardization;

standardized xi = [x - x-bar] / std. deviation of x

Question 12

Addressing the following are attributes of what process?

• Remove HTML tags;
• Remove punctuations;
• Remove numbers;
• Remove white spaces;

Answer 12

text preparation or cleansing;

Question 13

Addressing the following are attributes of what process?

• Lowercasing;
• Removal of stop words (e.g., “is”, “the”, etc.);
• Stemming;
• Lemmatization;

Answer 13

normalization of cleansed text.

Question 14

Give an example of “stemming”.

Answer 14

Integrate, integration, integrating are all assigned a common value of “integrat”.

Question 15

What is the term for the conversion of inflected forms of a word into their “lemma” (i.e., morphological root). Similar to stemming but more computationally advanced and resource intensive.

Answer 15

lemmatization;

Question 16

In text wrangling, a ______ is a word, and ________ is the process of splitting a sentence into ________.

Answer 16

token;
tokenization;
tokens;

Question 17

The following are examples of _______ :

“the_market”; “market_is”; “is_up”; “up_today”;

Answer 17

bigrams, or more generally “N-grams”.