3_2: Market Analytics: Analysing and Predicting aggregated Demand and Competiton

Question 1

What is Customer Segmentation?

Answer 1

Segmentation: slicing a pie
is the process of dividing a heterogeneous customer base into distinct groups based on shared characteristics, behaviors, or needs

–>Groups are homogenous within and **heterogenous

Question 2

Name the 3 difficulties of customer segmentation?

Answer 2

Finding actionable outcome: is the result meaningful for the business need=

Choice of method: there is no method that is priori preferable to others

Iterative approach: much time and rounds of data collection

Question 3

RFM analysis is used to predict …?

Answer 3

used to predict the response rate and profitability generated by marketing campaings

–>tool to identify a organization´s best customers

Question 4

What is the meaning behind “RFM”?

Answer 4

Recency (R): numbe rof time units that have passed since last purchased

Frequency (F): average number of purchases per time unit

Monetary value (M): total doolar amount spent per time unit

Question 5

For what is the RFM method used

Answer 5

• it is used to identify the most engaged customers based on the observation that recency, frequency, and monetary metrics are often correlated with the probability of response and lifetime value
• common to use same discrete scoring for all three metrices –>three dimensional cube
• targeting decisions by selecting a subset of segment from the RFM cube

Question 6

4 Steps of the RFM Analysis

Answer 6

1. recency: sort databse in terms of most recent transactions and score your customers
2. frequency: re-sort the dartabase on frequency
3. Monetary value: Re-sort the database on sales dollar volume (monetary value)
4. Selection use the three columns and each customers total score
   –>highest scores are the best customers

Question 7

Advantages and Disadvantages of RFM Analysis?

Answer 7

Pros:
- Valuable for short term financial orientation
- requires no marketing strategy
- Fast, simple, and easy to use, exlain and implement

Cons:
- Liimited marketing usage as iit is only about engagement
- does not measure the factors that impact customer behavior

Question 8

What is Behavioral segmentation?

Answer 8

Goal to understand customer behavior (marketing orientation)

• uses behavioral data instead of financial data
• –>Customer core behavior doen´t change

Question 9

What is Supervised learning (Classification)?

Answer 9

Group affiliation: is known
Goal: to predict outcome data from independent variables

Question 10

From RFM Analysis to Behaviroal Segmentation

Picture

Answer 10

Question 11

What is Unsupervides learning (clustering)?

Answer 11

Group affiliation: unkown
Goal: discover grouppings from data structue

Question 12

What are the two methods of distance-based clustering?

Answer 12

Hierachircal clusteirng
K-Mean-based clustering

–>minimize the discance between the group member while max. distance to members of other groups

Question 13

What are the two methods of Model-based clustering?

General description of Model-based clustering?

Answer 13

Model-based clustering
Latent class analyis

–>Model data so that the observed variance can be represented by a small group with specific distrib. characteristics

Question 14

How does Heararchical clustering work?

Answer 14

observations are group acc. to their similarity (distance matrix) clust method used complete linkage method

Question 15

What are the steps in Hierachical Clustering?
(Distance-based clustering)

Answer 15

1. Calculation of distance between the observation by Euclidean distance dissimilarity matrix
2. the model uses the complete linkage method, comparing distance between all group members
3. Output dendogram which is interpreted by height and where observations are joined

Question 16

Dendogram: Hierarchical clusteirng
(distance-based clustering)

Answer 16

At the lowest level, the groups are combined into smaller groups that are relatively similar.
–>These groups are sucessively combiine with less similar groups

Height = dissiminlarity

Question 17

What is k-Mean-based clustering?

Answer 17

(also k-mean clustering) = find groups based of sum-squares deviation from the multivariate center of the assigned group

–>centers need to be specified

Question 18

What are the steps in (k-)mean-based clustering?

Answer 18

1. Choose number of clusters and maximum distance
   –>requires numeric data
2. Find observation for cluster 1
3. Take second obersavtion if far enough from 1 –>Cluster
4. > Take next observation and compare with 1 and 2 (ggf. cluster 3)

Question 19

What do k-means cluster plots show?
What are the limitations

Answer 19

whether it is possible to differentiate groups based on key variables

Limitation:
K-means requires arbitrary specification of clusters (use different values for k)
–>difficult to determine whether one solution is better than the other

Question 20

What is the problem with K-means cluster plots?

Answer 20

Difficult whether one sultion is better than another
–>Repeat analysis for several number of clusters to compare the results

Question 21

How can the outcome of the k- mean based model be tested?

Distance based

Answer 21

1. Check mean values by ussing aggregate()
2. Plot k-mean cluster to chech if it is possible to differentiate groups based on key variables
3. Alternatively plot two continous variable by segment

Question 22

How can the solution of the model based clusteirng be tested? ( mclust())
How to compare between models?

Answer 22

1. Check mean values by using aggregate()
2. PLot model based clusters
3. use other values for G, and compare the model outputs:
   - Log liklihood –>less negative
   - BIC –>lowest value
   –>also used for model comparisoon

Question 23

How can the solution of the Laten class analysis be tested?

Answer 23

1. Check mean values using aggregate()
2. Plot the LCA clusters
3. compare predicted class memberships

Question 24

How can the solution of hierarchical clustering be tested?

Distance based clustering

Answer 24

• Zooming in and focusing on certain branches of the dendogram
• use the ceophentic correlation coefficient (CPC) = measure the correlation between the original dissimilarty and the cophentic distances
• CPCC close to 1 –>strong positive correlation

Question 25

What is the output of model-based clustering ?

Answer 25

Either shows the optimal number of clusters if g was not predetermined
- BIC
- Log-Liklihood

Question 26

Key facts about model-based clustering?
(mclust)

Answer 26

• observation come from groups with different statistical distributions –>algorithm try to find best set of such underlying distribution
• it clusters as being drawn from a mixture of normal distribution
• Can only be used with numerical data

Question 27

What is the Laten Class Analysis (LCA)?
(Model-based Clustering)

Answer 27

differences are attributable to unobserved groups that one wishes to uncover (nclass is predetermined) –>Bayesian technique

–>Goal: estimate probabilities of membership in each class and assing individual to their most likely class

Question 28

Steps in Latent-class analysis?
(Model-based clustering)

Answer 28

1. Variable scores are caused by the hidden groups
2. LCA posits a latent variable that maximizes liklihood of obserrving the scorces and the variables
3. It creates a probability of each observation belonging to each segment
4. Segment with highest probability is the segment where most observations are placed

Question 29

Advantages of LCA?

Answer 29

• Possible for complex data
• Provides optimal number of clusters
• Provided indicator for variables
• Segment probability score

Question 30

What are diagnostics to test for statistical fit? ( Latent class analysis - model based clustering)

Answer 30

• BIC:Bayes information criterion (lower values better)
• Error rate (better if lower)
• Negative log likelihood (better if less negative)

Question 31

Clustering process in R?

Answer 31

1. Transofrm to right format
2. Compute distance matrix
3. Apply clustering method
4. Analyze groups
5. Examine solution in the model and apply

Question 32

How does Classification work?
(Supervised learning)

Answer 32

= use observations with known status to derive predictors which then can be applied to new observations

Question 33

Classification process in R

Answer 33

1. Colllcecct data (group membership is known)
2. Splitting data: Trainsing set with 50-80% of obserations and test set with 20-50%
3. Build prediction model: identify predictors from training data
4. Assess performance by applying predictors to test dataset

Question 34

How can the performance of the Naiive Bayes Classification be assed?

Answer 34

1. Considering raw agreement rate
   mean(raw$Segment == predition) = 0.92 –>92% correct prediction
2. compare performance against random chance using ARI
   - 1 = perfect agreement, 0 = random -1 = complete disagreement
3. asses performance for each different class using Confusion Matrix
   - actual segment is left (rows)
   - predicted (columns

Question 35

How can the performance of the Random forrest be assed?

Answer 35

1. compare performance against random chance using ARI (adjustedRandIndex)
   1= perfect agreement, 0 = random -1 = complete disagreement
2. asses performance for each different class using Confusion Matrix using test data
   - actual segment is left (rows)
   - predicted (columns

Question 36

How can the outcome of the Naive Bayes Classification be tested?

Answer 36

1. use test data to predict() values based on trained model using test data
2. Revview the segment frequencies and compare to the inital a-priori frequencies based on the training data

Question 37

How can the outcome of the Random Forest be tested?

Answer 37

1. use test data to predict() values based on trained model using test data
2. plot the clusters based on test data

Question 38

What does Naive Bayes Classification do? (Supervised learning)

Answer 38

= Training data is used to learn probability of class membership as a function of each predictor variable considered independently –>using bayes rule

–>starts with observed probabilities of vairbales conditiona on segments found in the training data
—>only uses one model

Question 39

What is the Random forrest classification (compared to Naive bayes classification? (Supervised learning)

Answer 39

Instead of unsing a sinlge model, it builds and ensemlbe of models that jointly classify the data by fitting many classification trees (forrest)

–>not providing class membership

Question 40

What is the Class imbalance? and how can it be resolved in RandomForrest models?

Answer 40

using randomForest for prediction the model might generate values with 90% being in one group –>imbalance

–>Resolving:
- looking at frequency table of the training data, to see the group allocation –>pick smallest
- SSet sampsize= “value” in randonForest model

Question 41

What is meant by importance analysis in the Random Forest model?

Answer 41

the model uses many predictor variables, thus it is useful to know the importance of different classification variables

–>randomForest( importance = TRUE)

Question 42

Advantage of Random forrest model?

Answer 42

• many classification trees(format) instead of one model
• More accurate because more models are applied
• Useful to estimate the importance of predictor variables

Question 43

Classification trees are used for?

Answer 43

used to predict a categorical (and usually) binary dependent varible

Question 44

What are the steps of classification trees?

Answer 44

1. Identify most effect prediction variable for predicting the binary dependent variable
2. Stat wit root node with all combinations and then use independent variables to split the root node to create most improvement in class separation

Question 45

Classification trees:
What is a Pure decision node?

Answer 45

Pure decision node: all data points associated with that node have same value of dependent variable

Question 46

Classification trees:
What is Impurity?

Answer 46

Impurity (unreinheit): assess the degree of impurity or heterogeneity within a subset of data in the classification tree.

impurity of a split: is the weighted average of impurities for the nodes involved in that split

Question 47

Classification trees:
What is entropy?

Answer 47

–>Entropy is a measure of impurity or disorder

• Entropy is calculated using conditional probability
• Always between 0 and 1
  –>**Lower entropy –>decreasing impurity!

Question 48

What is Collaborative filtering?

Answer 48

filter and predict choices based on other people behavior
–>Typically using rating matrixes

Idea: people with similar preferences tend to like similar items

Question 49

What are the two method of collaborative filtering?

Answer 49

1. Neighborhood based method: prediict unknown rating by using the nearest neighbor approach
2. Model-based method: use more complex, predictive models

Question 50

Collaborative filtering: baseline estimates:

Answer 50

A rating matrix exhibts string user and item biases

• one can account for these systematic user and item effects:

The model caputures only the average user and item effect but it can help to absorb the biases and isolate the signal that represent user item interactions

Question 51

Advantages Collaboorative filtering?

Answer 51

Advantages:
- Make recommendations without any additional information about catalog items
- helps to produce non-trivial recommendations
- rating captures human tastes and judgments

Question 52

What are the disadvantes of Collaborative filtering?

Answer 52

Disadvantages:
- difficult to build reliable prediction models with trustworthy rating
- content filtering is biased towards popular items and standard choices
- cold start problem does not work for new users
- Product standardization is difficult
- Arbitrary assumptions

Question 53

Advantages of User-based filtering?

Answer 53

• easy to program
• more attractive when users are personally familiar

Question 54

Advantage of Item-based filtering?

Answer 54

• preferred when customers are not familiar
• item-based matrix of correlation is more stable over time
• matrix needs to be updated less often