Lecture 7

Question 1

What is prescriptive analytics?

Answer 1

prescriptive analytics is concerned with what
foresight can be obtained after a predictive model has been built.

Question 2

What is the general framework for prescriptive analytics?

Although a general framework can be built, the details of the specific methodology are domain- and technology-specific.

Answer 2

The general framework is:
1. Identify alternative decisions and objectives
2. Model and simulate alternative decisions
3. Select an optimal decision
4. Perform Analysis

Although a general framework can be built, the details of the specific methodology are domain- and technology-specific.

Question 3

How does it vary in practice ?

Answer 3

In practice, as highlighted by the literature review, there is no single approach to prescriptive analytics.
The developed approaches are often domain-specific and techniques specific.
Usually, the combination of multiple approaches is employed.
The focus of the course is improving the robustness of the model

Question 4

What are some techniques for prescriptive analytics?

Answer 4

The techniques are:
* What-If analysis
* Simulation (and ML)

Question 5

What is a What-If analysis ?

Answer 5

It’s a qualitative approach to determine:
* Exceptional situation
* Reaction
* Likelihood of happening
* Consequence of happening
* Recommendation
For each of the previous 4, the What-If questions have to be developed and answered

Question 6

What is Simulation (and ML) ?

Answer 6

Machine Learning is by definition an inductive process:
A ML algorithm defines the optimal hypothesis (i.e. predictive model) by learning from the existing data.
On the other hand, in the conventional modelling approach, a human defines a model, tunes its parameter and produces new data (i.e.
simulation results)

Question 7

How do you integrate simulations and ML ?

Answer 7

Several integrations of ML and simulation models can exist, based on the integration point.
The focus of this course is on:
ML model as generator:
Once a ML has been fit on the existing data and has provided good performances, it can also
be used to make predictions on unseen data (i.e.
counterfactual analysis)

Question 8

What is the machine learning pipeline in practice?

Answer 8

ML in practice goes through the following steps:
1. Raw Data
* Collection
* Download
* Scraping
2. Data Preprocessing
* Data quality (cf. Diagnostic)
* Missing data
* Categorical variables
3. Train-test split
* Single validation
* Cross-validation
4. Model fit
* Fit on training data
* Test on testing data
5. Performane Evaluation
* Performance metric choice
* Evaluation on validation data

Question 9

How do you make a model more robust in the ML pipeline?

Answer 9

You change the pipeline such that:
ML in practice goes through the following steps:
1. Raw Data
* Collection
* Download
* Scraping
2. Data Preprocessing
* Data quality (cf. Diagnostic)
* Missing data
* Categorical variables
* Dimensionality reduction
3. Train-test split
* Single validation
4. Model fit
* Fit on training data
* Test on testing data
* Ensembling
5. Performane Evaluation
* Performance metric choice
* Evaluation on validation data
* Cross-validation

Question 10

What is dimensionality reduction ?

Answer 10

Dimensionality reduction is the process of transforming the original dataset:
From n columns/features

To k < n columns features
Through feature extraction and feature selection. The original features might or might not be preserved. The original variables are
lost, and replaced by projection/compressed
equivalents.

Feature selection: A subset of the original variables is chosen for the model

Question 11

What is feature construction in dimensionality reduction ?

Answer 11

Manual feature extraction.

Question 12

What is feature learning in dimensionality reduction ?

Answer 12

Automatic feature extraction

Question 13

What is feature transformation in dimensionality reduction ?

Answer 13

Usually denotes less sophisticated transformations over the features, like re-scaling data, bucketing, etc

Question 14

What is feature engineering in dimensionality reduction ?

Answer 14

Sometimes it is used as a synonym for feature extraction, although contrary to extraction, there seems to be a relatively universal consensus that engineering involves not only creativity constructions but pre- processing tasks and naïve transformations as well.

Question 15

What is feature selection ?

Answer 15

A subset of the original variables is chosen for the model. Uses different methods:
Filter methods
Wrapper Methods
Embedded Methods

Ex: Chi squared test, information gain and correlation coefficient scores.

Question 16

What are Filter methods ?

Answer 16

Filter feature selection methods apply a statistical measure to assign a scoring to each
feature. The methods are often univariate and consider the feature independently, or with
regard to the dependent variable.

Ex: Chi squared test, information gain and correlation coefficient scores.

Question 17

What are embedded methods?

Answer 17

Embedded methods learn which features best contribute to the accuracy of the model while the model is being created. The most common type of embedded feature selection methods are regularization methods. Regularization methods are also called penalization methods that introduce additional constraints into the optimization of a predictive algorithm (such as a regression algorithm) that bias the model toward lower complexity (fewer coefficients).

EX: Elastic Net and Ridge Regression.

Question 18

What are Wrapper methods ?

Answer 18

Wrapper methods consider the selection of a set of features as a search problem, where different combinations are prepared, evaluated and compared to other combinations. A predictive model is used to evaluate a combination of features and assign a score based on model accuracy. The search process may be methodical such as a best-first search, it may stochastic such as a random hill-climbing algorithm, or it may use heuristics, like forward and backward passes to add and remove features.

EX: Recursive feature elimination algorithm

Question 19

What is one of the benefits of dimensionality reduction ?

Answer 19

**Visualising/Clustering: **
The human perception is limited to three-dimensions.
Dimensionality reduction/feature selection
allows to reduce the number of dimensions in the data to three or less.
Working with limited dimensions can allow:
* Visualization
* Clustering and interpretation of the results

Question 20

How can you assess the stability of the model?

Answer 20

In order to assess the stability of the model, it is necessary to test it multiple times, we can do it with cross-validation.
A k-fold-cross-validation process splits the dataset in k parts:
* 1 part is used for testing
* k-1 are used for training
The process is repeated k times to ensure that the whole dataset is covered.
The performance of the model is the average across the k folds

Question 21

What can we do in case the data in unbalanced ?

unbalanced here signifies un-evenly distributed amongst every variable

Answer 21

In case of unbalanced data, the data sampling for cross-validation can be performed in order to maintain the original class balance. This cross-validation variant is called stratified cross-validation.

Question 22

What is ensemble modeling?

Answer 22

Emsemble modeling combines multiple models in a single predictor. Several parameters can be used for the model combination:
1. Combination can be
* Static
* Variable
2. Models can be
* Homogeneous
* Heterogeneous
3. Combination weights can be
* Fixed
* Time-varying
This is reflected in mulptiple methods:
1. Bagging
2. Stacking
3. Boosting

Question 23

What are the benefits of ensemble modeling?

Answer 23

Combining multiple models in a single predictor could help in:
* Reducing the variance/variability of the prediction
* Improving the predictive accuracy

Question 24

What is bagging?

Answer 24

Bagging is an ensemble method with static combination for Homogeneous/Hetereogeneous
models with fixed weight combination

Different models trained on different subsets of the data. In the test phase, predict from each weak model and vote their predictions to get the final prediction.

Question 25

What is stacking ?

Answer 25

It is a static combination for Homogeneous or Hetereogeneous models, giving a fixed weight combination.

Its a Two-level combination: Weak models are trained with one part of the dataset. A Meta-Model is used to with the outputs of the weak models from the second part of the dataset. To test, the inputs are given to the weak models, output is again collected and given to the meta-model. That is the final prediction.

Question 26

What is boosting?

Answer 26

It’s a dynamic combination for Homogeneous models with Time-fixed combination.
The final model is build by incrementally fitting
base models on the errors of the previous models.

Question 27

How do you deal with imbalanced data ?

Answer 27

A few techniques exist:
1. Collect more data
2. Changing performance metrics
3. Resampling dataset
4. Generating synthetic samples
5. Test different algorithms
6. Try penalized models
7. Different problem approaches
* Anomaly detection

Question 28

What are Machine Learning Operations (MLOps)

Answer 28

MLOps is a core function of Machine Learning engineering, focused on streamlining the process of taking machine learning models to production, and then maintaining and monitoring them. MLOps is a collaborative function, often comprising data scientists, devops engineers, and IT.

Question 29

Paper definition of MLOps

Answer 29

MLOps (Machine Learning Operations) is a paradigm, including aspects like best practices, sets of concepts, as well as a development culture when it comes to the end-to-end conceptualization, implementation, monitoring, deployment, and scalability of machine learning products. Most of all, it is an engineering practice that leverages three contributing disciplines: machine learning, software engineering (especially DevOps), and data engineering. MLOps is aimed at productionizing machine learning systems by bridging the gap between development (Dev) and operations (Ops). Essentially, MLOps aims to facilitate the creation of machine learning products by leveraging these principles: CI/CD automation, workflow orchestration, reproducibility; versioning of data, model, and code; collaboration; continuous ML training and evaluation; ML metadata tracking and logging; continuous monitoring; and feedback loops.