Week 7: Missing Data and Clustering Flashcards
What is a way to display ggplots together in R?
library(patchwork) #allows you to display ggplots together using plot1 + plot2
How do you print the number of missing values in a model?
naprint(na.action(model))
What is the code for creating a table display of the missingness patterns?
md.pattern(data)
How do you display the number of NAs in each variable of a dataset?
colSums(is.na(data))
What are embedded or model based methods for missing data?
Don’t impute, deal with missing values in the prediction model itself
What are the advantages of multiple imputation? (3)
- Solves the problem of too small standard errors
- Our level of confidence in a particular imputed value is expressed as the variation across the m completed datasets
- Under the right conditions, the pooled estimates are unbiased and have the correct statistical properties
How do you perform LOCF imputation in R?
tidyr::fill(data, variable)
Under what conditions is listwise deletion unbiased? What happens to the standard error?
Mean, regression weight and correlation are unbiased only under NDD. Standard error is too large
What is linkage?
The dissimilarity between two clusters if one or both contains multiple observations
What are the disadvantages of single and centroid and complete linkage?
- Single linkage can result in extended, trailing clusters in which single observations are fused one at a time. Can’t separate clusters properly if their is noise between clusters
- Centroid linkage can result in undesirable inversions, where two clusters are fused at a height below either of the individual clusters in the dendrogram.
- Complete linkage tends to break large clusters.
How do you perform mean imputation in R? (2 ways)
library(“mice”)
imp
What are the advantages of the indicator method? (2)
- Retains the full dataset and allows for systematic differences between the observed and unobserved data by the inclusion of the response indicator
- Can be useful to estimate the treatment effect in randomised trials when a baseline covariance is partially observed
How do you perform k-means clustering in R? and what does the output consist of?
means_cluster
Under what conditions is stochastic regression imputation unbiased? What happens to the standard error?
Mean, regression weights and correlation are unbiased under SDD
Standard error is too small
What is regression imputation?
First builds a model from the observed data
Predictions for the incomplete cases are then calculated under the fitted model and serve as replacements for the missing data
What is mean imputation?
Replace missing data by the mean or the mode for categorical data
What are the forumlas for NDD, SDD and UDD? Where M indicates whether variable 2 is missing (1) or not (0)
NDD: Pr(M=1 | var1, var2) = Pr(M=1)
SDD: Pr(M=1 | var1, var2) = Pr(M=1 | var1)
UDD: Pr(M=1 | var1, var2) can’t be reduced
What is the k-medoids clustering algorithm?
- Initialise: select k random points as the medoids
- Assign each data point to the closest medoid by using any distance method (e.g. euclidean)
- For each data point of cluster i, its distance from all other data points is computed and added. The point of ith cluster for which the computed sum of distances from other points is minimal is assigned as the medoid for that cluster
- Repeat steps 2 and 3 until the medoids stop moving
How do you perform multiple imputation and fit a model in R?
imp
What are internal validation indices and what are some popular methods?
- Only look at unsupervised bit: data and clustering and quantify how successful clustering is
- Popular measures: average silhouette width (ASW) - how close points are to other clusters, gap statistic
Under what conditions is regression imputation unbiased? What happens to the standard error?
Mean and regression weights are unbiased under SDD
-for regression weights under SDD, only if the factors that influence the missingness are part of the regression model
Standard error is too small
What is a “good” k-means clustering?
One for which the within-cluster variation (W(Ck)) is as small as possible
When does hierarchical clustering give worse results than k-means clustering?
When the data doesn’t have a hierarchical structure. e.g. when the best division into 2 groups is by gender but the best division into 3 groups is by nationality
How do you differentiate between SDD and UDD?
You can’t
What are the disadvantages of pairwise deletion?
- The covariance matrix may not be positive-definite
- Problems are more severe for highly correlated variables
- Requires numerical data that follows approximate normal distribution
What is the general code for imputing data and fitting a model with mice in R?
imp
How do you perform listwise deletion in R?
Use na.omit()
What is the algorithm for k-means clustering?
- Randomly assign a number, from 1 to K, to each of the observations. These serve as initial cluster assignments for the observations
- Iterate until the cluster assignments stop changing:
a) For each of the K clusters, compute the cluster centroid. The kth cluster centroid is the vector of the p feature means for the observations in the kth cluster
b) Assign each observation to the cluster whose centroid is closest (using Euclidean distance)
* When the result no longer changes, the local optimum has been reached
What are the advantages of k-medoids over k-means clustering?
- In k-means, squared Euclidean distance places the highest influence on the largest distances
- K-means lacks robustness against outliers that produce very large distances
- K-medoids is less sensitive to outliers
What does the vertical axis/height of a dendrogram show?
Mean intercluster dissimilarity: for any two observations, look at the point in the tree where branches containing those observations are first fused. The height of this fusion indicates how different the observations are. Higher = less similar
How do you perform hierarchical clustering in R and plot the dendrogram and select the number of clusters?
distances
Should you cut the dendrogram higher or lower for more clusters?
Lower cut = more clusters
What is within cluster variation?
Can be defined in many ways, common approach is euclidean distance
What are two ways of hierarchical clustering?
- Bottom-up or agglomerative clustering: the dendrogram is built starting from the leaves and combining clusters up to the trunk
- Divisive clustering: starting with one cluster and keep splitting the most different
What is the indicator method of imputation?
The indicator method replaces each missing value by a zero and extends the regression model by the response indicator. This is applied to each incomplete variable. Then analyse the extended model
What does summarise_all() do?
used to do a summarise transformation across all variables
How do you perform regression imputation in R? (2 ways)
imp
What is the time complexity of hierarchical clustering?
O(n*3)
What is Rubins theory of classifying missing data?
Every data point has some likelihood of being missing. The process that governs these probabilities is called the missing data mechanism or response mechanism. There are three categories of missing data: MCAR, MAR and MNAR
How do you do a mean calculation, removing missing values first?
mean(y, na.rm=TRUE)
How do you show the indices of NAs in a model?
na.action(model)
What is vector quantisation?
- K-means clustering applied to images
- Goal is image compression -> less storage. Cluster pixels and replace them by their cluster centroid
- File size increases with number of clusters
- Image loss decreases with number of clusters
Once you have performed multiple imputation and stored it as “imp”, what codes are there to access different parts of the imputation?
imp$data #shows original data
imp$imp #shows imputed data
complete(imp, 3) #extracts the 3rd completed dataset of the m imputations
What are LOCF and BOCF imputation methods?
Last observation carried forward (LOCF) and baseline observation carried forward (BOCF) are ad-hoc imputation methods for longitudinal data
Take the previous observed value as a replacement for the missing data
What type of data does multiple imputation assume?
SDD assumption (or NDD)
Under what conditions is pairwise deletion unbiased?
Mean, regression weight and correlation are unbiased only under NDD.
How many possible re-orderings of the dendrogram are there without changing its meaning?
2^(n-1)
What is the time complexity of k-medoids clustering?
O(k*(n-k)^2)
What does the variance of the parameter estimates from multiple imputation consist of?
- Within-dataset variance: the conventional sampling variance caused from taking a sample rather than entire population, the uncorrected standard error
- Between-dataset variance: extra variance cause by the missing data
- Simulation error: extra variance caused by the fact that the estimator is based on a finite amount of datasets m. Less of a problem with machines as m can be large
How do you use external information to evaluate clustering results?
Are the clusters associated with an external feature Y? Find data for Y to evaluate
What is pairwise deletion/available case analysis?
Calculates the mean and covariances of all available data. The matrix summary of statistics is then used for analysis and modelling
Under what conditions is LOCF imputation unbiased? What happens to the standard error?
LOCF is always biased
Standard error is too small
What is stochastic regression imputation?
A refinement of regression imputation that attempts to address correlation bias by adding noise to the predictions. This method first estimates the intercept, slope and residual variance under the linear model, then calculates the predicted value for each missing value and adds a random draw from the residual to the prediction
How do you change the settings to always omit NAs?
options(na.action = na.omit)
How do you use visual exploration to evaluate clustering results when there are many variables?
- Reduce variables into 2D “manifold” for visualisation
- Popular techniques: UMAP, t-SNE, MDS, Discriminant coordinates, PCA
What is the hierarchical clustering algorithm?
- Begin with n observations and a measure (such as Euclidean distance) of all the n(n-1)/2 pairwise dissimilarities. Treat each observation as its own cluster.
- For i=n, n-1, … 2:
a) Examine all pairwise inter-cluster dissimilarities among the i clusters and identify the pair of clusters that are the least dissimilar. Fuse these two clusters. The dissimilarity between these two clusters indicates the height in the dendrogram at which the fusion should be placed
b) Compute the new pair-wise inter-cluster dissimilarities among the i-1 remaining clusters
What does mean(y) return when y has missing values?
NA
List the 4 ways of evaluating clustering results
- Use of external information
- Visual exploration
- Stability assessment
- Internal validation indices
How do you fit a linear model, removing missing values first?
lm(y~x, data, na.action = na.omit)
When and why should variables be scaled before computing dissimilarity?
- Scaling to standard deviation 1 gives equal importance to each variable in the clustering
- Useful when variables are measured on different scales
What is imputation?
Replacing missing values with guessed values
Under what conditions is mean imputation unbiased? What happens to the standard error?
Only mean is unbiased under NDD
Standard error is too small
Disturbs relations between variables
How do you differentiate between NDD and SDD?
Look at the {0,1} missingness indicator M versus other features. If you can classify M from other features thence do not have NDD
What is the code for getting the number of na’s for each variable grouped by a variable
data %>% group_by(variable) %>% summarise_all(function(x) sum(is.na(x))
How do you perform average silhouette width analysis in R? what does the result tell us?
distances
What is stability assessment of clustering results?
How much does the clustering change when: 1. Changing some hyperparameters, 2. Changing some observations (bootstrapping), 3. Changing some features
Check if observations are classified into same cluster across choices
What are strategies to handle MNAR data?
Find more data about the causes for the missingness, or to perform what-if analyses to see how sensitive the results are under various scenarios.
What is deductive imputation?
Deduce the missing data from the data you have e.g. from height and weight can calculate BMI
Describe the three categories of missing data
Missing completely at random (MCAR) / Not data dependent (NDD): The probability of being missing is the same for all cases. The causes of the missing data are unrelated to the data.
Missing at random (MAR) / Seen data dependence (SDD): the probability of being missing is the same only within groups defined by the observed data
Missing not at random (MNAR) / Unseen data dependence (UDD): the probability of being missing varies for reasons that are unknown to us. It is missing because of the value you would have obtained.
What is listwise deletion/complete case analysis?
Eliminate all cases with one or more missing values
How do you perform stability assessment in R?
library(fpc)
clusterboot(data, clustermethod = hclustCBI, method = “complete“, k = 3) #for kmeans use kmeansCBI
Gives Jaccard bootstrap mean for each cluster. Generally stability less than 0.6 is considered unstable. Clusters with stability above 8.5 are highly stable (likely to be real clusters)
What is the procedure for multiple imputation?
- Create several (m) complete versions of the data (imputed datasets) by replacing the missing values by plausible data values using stochastic imputation
- Estimate the parameters of interest from each imputed dataset. Typically done by applying the method that we would have used if the data was complete
- Last step is to pool the m parameter estimates into one estimate by getting the average of the estimates Q*, and to estimate its variance
What is k-medoids clustering?
The idea of k-medoids is to make the final centroids as actual data points, making them interpretable
What is the disadvantage of a k-medoids clustering?
It is not suitable for clustering non-spherical groups of objects
How do you perform stochastic regression imputation in R?
fit
What is the goal of clustering?
Clustering looks to find homogeneous subgroups among observations
What are the 4 types of linkage and how they work?
- Complete: Maximal intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the largest of these dissimilarities.
- Single: Minimal intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the smallest of these dissimilarities.
- Average: Mean intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the average of these dissimilarities.
- Centroid: Dissimilarity between the centroid for cluster A (a mean vector of length p) and the centroid for cluster B.
What are the disadvantages of regression imputation? (4)
- Can increase correlation between variables
- Correlations are biased upwards
- P-values are too optimistic
- Variability systematically underestimated
What is the code for returning the mean of each variable in a dataset? (2 ways)
map_dbl(data, mean) #use map_int if data is int etc.
summarise_all(data, mean)
What are three practical issues with clustering?
- Small decisions such as k and dissimilarity measure have big impacts on the clusters
- Validating the clusters obtained: clustering will always result in clusters, but do they represent true subgroups in the data or are they simply clustering the noise
- Since k-means and hierarchical clustering force every observation into a cluster, the clusters found may be heavily distorted due to the presence of outliers that do not belong to any cluster
What is k-means clustering?
Specify the desired number of clusters K, then the K-means algorithm will assign each observation to exactly one cluster
What are the disadvantages of listwise deletion? (4)
- Large loss of information
- Hopeless with many features
- Inconsistencies in reporting as analysis on the same data often uses different sub-samples
- Can lead to non-sensical sub-samples e.g. deleting data in time series analysis
What is correlation-based distance?
Considers two observations to be similar if their features are highly correlated, even though the observed values may be far apart in terms of Euclidean distance
What are the advantages of single and complete linkage?
- Single linkage can differentiate between non-eliptical clusters
- Complete linkage gives well-separated clusters if their is noise between the clusters
