3_2: Market Analytics: Analysing and Predicting aggregated Demand and Competiton

Question 1

What is Customer Segmentation?

Answer 1

Segmentation: slicing a pie
is the process of dividing a heterogeneous customer base into distinct groups based on shared characteristics, behaviors, or needs

–>Groups are homogenous within and **heterogenous

Question 2

Name the 3 difficulties of customer segmentation?

Answer 2

Finding actionable outcome: is the result meaningful for the business need=

Choice of method: there is no method that is priori preferable to others

Iterative approach: much time and rounds of data collection

Question 3

RFM analysis is used to predict …?

Answer 3

used to predict the response rate and profitability generated by marketing campaings

–>tool to identify a organization´s best customers

Question 4

What is the meaning behind “RFM”?

Answer 4

Recency (R): numbe rof time units that have passed since last purchased

Frequency (F): average number of purchases per time unit

Monetary value (M): total doolar amount spent per time unit

Question 5

For what is the RFM method used

Answer 5

• it is used to identify the most engaged customers based on the observation that recency, frequency, and monetary metrics are often correlated with the probability of response and lifetime value
• common to use same discrete scoring for all three metrices –>three dimensional cube
• targeting decisions by selecting a subset of segment from the RFM cube

Question 6

4 Steps of the RFM Analysis

Answer 6

1. recency: sort databse in terms of most recent transactions and score your customers
2. frequency: re-sort the dartabase on frequency
3. Monetary value: Re-sort the database on sales dollar volume (monetary value)
4. Selection use the three columns and each customers total score
   –>highest scores are the best customers

Question 7

Advantages and Disadvantages of RFM Analysis?

Answer 7

Pros:
- Valuable for short term financial orientation
- requires no marketing strategy
- Fast, simple, and easy to use, exlain and implement

Cons:
- Liimited marketing usage as iit is only about engagement
- does not measure the factors that impact customer behavior

Question 8

What is Behavioral segmentation?

Answer 8

Goal to understand customer behavior (marketing orientation)

• uses behavioral data instead of financial data
• –>Customer core behavior doen´t change

Question 9

What is Supervised learning (Classification)?

Answer 9

Group affiliation: is known
Goal: to predict outcome data from independent variables

Question 10

From RFM Analysis to Behaviroal Segmentation

Picture

Answer 10

Question 11

What is Unsupervides learning (clustering)?

Answer 11

Group affiliation: unkown
Goal: discover grouppings from data structue

Question 12

What are the two methods of distance-based clustering?

Answer 12

Hierachircal clusteirng
K-Mean-based clustering

–>minimize the discance between the group member while max. distance to members of other groups

Question 13

What are the two methods of Model-based clustering?

General description of Model-based clustering?

Answer 13

Model-based clustering
Latent class analyis

–>Model data so that the observed variance can be represented by a small group with specific distrib. characteristics

Question 14

How does Heararchical clustering work?

Answer 14

observations are group acc. to their similarity (distance matrix) clust method used complete linkage method

Question 15

What are the steps in Hierachical Clustering?
(Distance-based clustering)

Answer 15

1. Calculation of distance between the observation by Euclidean distance dissimilarity matrix
2. the model uses the complete linkage method, comparing distance between all group members
3. Output dendogram which is interpreted by height and where observations are joined

Question 16

Dendogram: Hierarchical clusteirng
(distance-based clustering)

Answer 16

At the lowest level, the groups are combined into smaller groups that are relatively similar.
–>These groups are sucessively combiine with less similar groups

Height = dissiminlarity

Question 17

What is k-Mean-based clustering?

Answer 17

(also k-mean clustering) = find groups based of sum-squares deviation from the multivariate center of the assigned group

–>centers need to be specified

Question 18

What are the steps in (k-)mean-based clustering?

Answer 18

1. Choose number of clusters and maximum distance
   –>requires numeric data
2. Find observation for cluster 1
3. Take second obersavtion if far enough from 1 –>Cluster
4. > Take next observation and compare with 1 and 2 (ggf. cluster 3)

Question 19

What do k-means cluster plots show?
What are the limitations

Answer 19

whether it is possible to differentiate groups based on key variables

Limitation:
K-means requires arbitrary specification of clusters (use different values for k)
–>difficult to determine whether one solution is better than the other

Question 20

What is the problem with K-means cluster plots?

Answer 20

Difficult whether one sultion is better than another
–>Repeat analysis for several number of clusters to compare the results

Question 21

How can the outcome of the k- mean based model be tested?

Distance based

Answer 21

1. Check mean values by ussing aggregate()
2. Plot k-mean cluster to chech if it is possible to differentiate groups based on key variables
3. Alternatively plot two continous variable by segment

Question 22

How can the solution of the model based clusteirng be tested? ( mclust())
How to compare between models?

Answer 22

1. Check mean values by using aggregate()
2. PLot model based clusters
3. use other values for G, and compare the model outputs:
   - Log liklihood –>less negative
   - BIC –>lowest value
   –>also used for model comparisoon

Question 23

How can the solution of the Laten class analysis be tested?

Answer 23

1. Check mean values using aggregate()
2. Plot the LCA clusters
3. compare predicted class memberships

Question 24

How can the solution of hierarchical clustering be tested?

Distance based clustering

Answer 24

• Zooming in and focusing on certain branches of the dendogram
• use the ceophentic correlation coefficient (CPC) = measure the correlation between the original dissimilarty and the cophentic distances
• CPCC close to 1 –>strong positive correlation

Question 25

What is the output of model-based clustering ?

Answer 25

Either shows the **optimal number of clusters** if g was not predetermined - BIC - Log-Liklihood

Question 26

Key facts about model-based clustering? **(mclust)**

Answer 26

- observation come from **groups with different statistical distributions** -->algorithm try to find best set of such underlying distribution - it **clusters as being drawn from a mixture of normal distribution** - Can **only be used** with **numerical data**

Question 27

What is the Laten Class Analysis (LCA)? (Model-based Clustering)

Answer 27

differences are **attributable to unobserved groups** that one wishes to uncover (**nclass** is predetermined) -->Bayesian technique -->**Goal**: estimate **probabilities of membership** in each class and assing individual to their most likely class

Question 28

Steps in Latent-class analysis? (Model-based clustering)

Answer 28

1. Variable scores are caused by the hidden groups 2. LCA posits a latent variable that maximizes **liklihood of obserrving the scorces and the variables** 3. It creates a probability of each observation belonging to each segment 4. Segment with highest probability is the segment where most observations are placed

Question 29

Advantages of LCA?

Answer 29

* Possible for **complex data** * Provides **optimal number of clusters** * Provided **indicator for variables** * Segment **probability score**

Question 30

What are diagnostics to test for **statistical fit**? ( Latent class analysis - model based clustering)

Answer 30

- **BIC:**Bayes information criterion (lower values better) - **Error rate** (better if lower) - **Negative log likelihood** (better if less negative)

Question 31

Clustering process in R?

Answer 31

1. Transofrm to right format 2. Compute **distance matrix** 3. Apply clustering method 4. Analyze groups 5. Examine solution in the model and apply

Question 32

How does Classification work? (Supervised learning)

Answer 32

= use observations with **known status** to derive **predictors** which then can be applied to new observations

Question 33

Classification process in R

Answer 33

1. Colllcecct data (group membership is known) 2. Splitting data: Trainsing set with 50-80% of obserations and test set with 20-50% 3. Build prediction model: identify predictors from training data 4. Assess performance by applying predictors to test dataset

Question 34

How can the performance of the **Naiive Bayes Classification** be assed?

Answer 34

1. Considering **raw agreement rate** mean(raw$Segment == predition) = 0.92 -->92% correct prediction 2. compare performance against random chance using **ARI** - 1 = perfect agreement, 0 = random -1 = complete disagreement 3. asses performance for **each** different class using **Confusion Matrix** - actual segment is left (**rows**) - predicted (**columns**

Question 35

How can the performance of the **Random forrest** be assed?

Answer 35

1. compare performance against random chance using **ARI** (adjustedRandIndex) 1= perfect agreement, 0 = random -1 = complete disagreement 2. asses performance for **each** different class using **Confusion Matrix** using **test data** - actual segment is left (**rows**) - predicted (**columns**

Question 36

How can the outcome of the **Naive Bayes Classification** be tested?

Answer 36

1. use **test data** to predict() values based on trained model **using test data** 2. Revview the **segment frequencies** and compare to the inital **a-priori frequencies** based on the **training data**

Question 37

How can the outcome of the **Random Forest** be tested?

Answer 37

1. use **test data** to predict() values based on trained model **using test data** 2. plot the **clusters** based on test data

Question 38

What does Naive Bayes Classification do? (Supervised learning)

Answer 38

= Training data is used to learn **probability of class membership** as a function of each **predictor variable** considered independently -->using bayes rule -->starts with **observed probabilities** of vairbales conditiona on segments found in the training data --->**only uses one model**

Question 39

What is the Random forrest classification (compared to Naive bayes classification? (**Supervised learning**)

Answer 39

Instead of unsing a sinlge model, it **builds and ensemlbe of models** that jointly classify the data by fitting **many classification trees** (forrest) -->**not providing** class membership

Question 40

What is the **Class imbalance**? and how can it be resolved in RandomForrest models?

Answer 40

using randomForest for **prediction** the model might generate values with 90% **being in one group** -->**imbalance** -->**Resolving**: - looking at frequency table of the **training data**, to see the group **allocation** -->pick **smallest** - SSet **sampsize**= "value" in randonForest model

Question 41

What is meant by **importance** analysis in the **Random Forest model**?

Answer 41

the model uses many **predictor variables**, thus it is useful to know the **importance of different** classification variables -->randomForest( importance = **TRUE**)

Question 42

Advantage of Random forrest model?

Answer 42

- **many classification trees(format)** instead of one model - **More accurate** because more models are applied - **Useful to estimate the importance** of **predictor** variables

Question 43

Classification trees are used for?

Answer 43

used to **predict a categorical (and usually) binary** dependent varible

Question 44

What are the steps of classification trees?

Answer 44

1. Identify **most effect prediction variable** for predicting the binary dependent variable 2. Stat wit root node with all combinations and then use independent variables to split the root node to **create most improvement in class** separation

Question 45

Classification trees: What is a Pure decision node?

Answer 45

**Pure decision node:** all data points associated with that node have **same value of dependent variable**

Question 46

Classification trees: What is Impurity?

Answer 46

**Impurity (unreinheit)**: assess the **degree of impurity or heterogeneity** within a **subset of data** in the classification tree. impurity of a **split**: is the weighted average of impurities for the nodes involved in that split

Question 47

Classification trees: What is entropy?

Answer 47

-->**Entropy** is a measure of **impurity or disorder** - Entropy is calculated using conditional probability - Always between 0 and 1 -->**Lower entropy -->decreasing impurity!

Question 48

What is Collaborative filtering?

Answer 48

**filter and predict** choices based on other people behavior -->Typically using **rating matrixes** Idea: people with **similar** preferences tend to like similar items

Question 49

What are the two method of collaborative filtering?

Answer 49

1. **Neighborhood based method:** prediict unknown rating by **using the nearest neighbor approach** 2. **Model-based method:** use more complex, predictive models

Question 50

Collaborative filtering: baseline estimates:

Answer 50

A **rating matrix** exhibts string user and item biases - one can account for these systematic user and item effects: The model caputures **only the average user and item effect** but it can help to **absorb the biases and isolate** the signal that represent user item interactions

Question 51

Advantages **Collaboorative filtering**?

Answer 51

**Advantages:** - Make recommendations **without any additional information** about catalog items - helps to **produce non-trivial recommendations** - rating **captures human tastes and judgments**

Question 52

What are the disadvantes of Collaborative filtering?

Answer 52

**Disadvantages**: - **difficult to build reliable** prediction models with trustworthy rating - content filtering is **biased towards popular items** and standard choices - **cold start problem** does not work for new users - **Product standardization** is **difficult** - **Arbitrary assumptions**

Question 53

Advantages of **User-based filtering**?

Answer 53

- **easy to program** - more **attractive** when **users are personally familiar**

Question 54

Advantage of Item-based filtering?

Answer 54

- preferred when **customers are not familiar** - **item-based matrix** of correlation is **more stable** over time - matrix needs to be **updated less often**