L9: Segmentation Flashcards
What is the difference between segmentation and object detection?
object detection → makes bounding box (nothing of the shape of the object)
segentation → Gives information of the shape, uses pixel-wise mask for each object in the image)
Clusteirng
Why would you do color space clustering?
A way of doing image segmentation as objects with similar colors distributed in it will describe the same object. Grouping similar colors describe separation of dirrent objects.
- Each pixels gets a label according to the nearest cluster
Clustering
What is clustering
Given a bunch of data points in some space, clustering is about discovering dense regions of that space, and separating these from each other.
Clustering
K-means clustering
This can be done in pixel space, k cluster should be defined in advance.
- K-means has hard limitations (black or white). If the clusters overlap it will be a problem, which GMM can solve.
Clustering - GMM
❗️❗️❗️Know the GMM model
Initialize it with mahalnobis (elipse) distance instead of euclidean (spherical).
- GMM is k-dimensional ellipse
Goal is to estimate k means (μ), covariance (Σ) and k mixing coefficients (π)
- π is weights summing to 1. The higher the cluster is, the higher π is.
Softly assigned → e.g. 67% to k_1, 12% to k_2 and 21% to k_3.
- probability of each data point for one of the the k Gaussians is really the “distance” measure used during GMM optimization.
Iteratively compute (a local) maximum likely estimate for the unknown mixture parameters (πk, μk, Σk)
- Expectation step:
- Maximization step:
Clustering: GMM
❗️❗️❗️Know the EM algorithm for GMM optimization
Step 1: Initialize the means (using k-means often) as a starting step and then proceed with EM afterwards.
E first, then M.
- E: calculate the “probability” that the i’th data point belongs to the k’th cluster
-M: Estimate the number of points assigned to each cluster k.
Updates (πk, μk, Σk).
Alternates between the E and M steps, and when done the likelihood of all of the data points given the K cluster will be as high as possible.
GMM reaches local optima and thus depends on a good initialization.
Clustering: Mean shift
❗️❗️❗️Know the update method for the mean shift algorithm
Updates iteratively by shifting each point uphill intil it reaches peak.
- For color segmentation; when all the points have been shifted to their respective mode, we can assign them the value of that mode
Clustering: Mean shift
❗️❗️❗️Know the role of the kernel and the bandwidth for mean shift
Smaller bandwidth results in more clusters, while larger results in fewer.
Kernel Density Estimation (KDE)
Clustering: Mean Shift
❗️❗️❗️Know the differences between and pros/cons of k-means, GMMs, and mean shift
K-means:
- hard assigned to a cluster
- dependent on number of k
- hope we separate enough and do not overlap
- parametric (k)
GMM:
- softly assigned to a cluster (probability of belonging to one cluster and a probability of belonging too another)
- great at overlaps
- parametric (πk, μk, Σk)
MS:
- mode finding method
- non parametric, has internal parameter tuning (kernal size and bandwidth)
Mean Shift (MS)
U-Net
❗️❗️❗️The structure of the network
U-net is a learning-based neural segmentation algorithm. Predicts correct class labels for each pixel in the input image.
- NEEDS LABELED TRAININF SET!
Upsamples bottom high-level features back to its original dimensions.
U-Net
❗️❗️❗️The modified loss weighting terms
1- w_c is a class-balancing term
Associates a smaller weight (and thereby a smaller gradient during training) to big segments to make room for also learning labels for small segments in the image
2. w_0 is a border term between touching segments
Focus more on the difficult borders between touching segments.
U-Net
❗️❗️❗️What it takes to set up a training/testing pipeline on a new segmentation task using U-Net
Utilizes augmentation during training, size of training set is not needed to be big.
U-Net
❗️❗️❗️The pros and cons of using U-Net vs. clustering methods for segmentation
U-Net:
- Separates colors with apperance variation and many similar structure across objects. Cant be done with color alone.
- defines a clear border (color segmentation does not do that)
- Uses small training set
Clustering
What is clustering
Given a bunch of data points in some space, clustering is about discovering dense regions of that space, and separating these from each other.
Clustering
What is clustering
Given a bunch of data points in some space, clustering is about discovering dense regions of that space, and separating these from each other.
