12 + 13 Flashcards
What is unsupervised learning?
When a machine learning algorithm is used to organise/make sense of data with no special output dimension.
What is PCA? What is a good example?
Principal component analysis is a technique which finds new axes for representing a set of input data points. A good example is a dataset with strong correlation between the two dimensions, really there is only two dimensions. We could represent it with a 1D graph(dimension reduction).
What does PCA create?
For a dataset with d dimensions, PCA creates a new representation in dimensions D1… Dd where D1 is the main component of variation, D2 the second biggest component of variation, D3 the third etc.
The first couple of components often account for nearly all variance, small components can typically be ignored.
How do we do PCA?
Compute covariance matrix(defines transformation that best maps cloud of gaussian nodes onto data) of input data. Compute eigenvectors(the vectors whose direction doesnt change under the transformation, these extract information from data) and associated eigenvalues(how much it is stretched by transformation, how much information an eigenvector extracts). Then project data vectors onto component vectors to get PCA representation.
How does PCA do compression?
By using less eigenvectors the output is reduced as some information is ignored. Therefore causing compression when the data is reconstructed.
What do clustering techniques do?
seperate datasets into discrete categories.
What is K-means clustering?
Partitions dataset into K clusters(K is chosen in advance). Clustering is done based on measure of similarity, commonly, Euclidean distance is used.
How does K-means clustering work?
Randomly choose Kpoints from input space to be starting cluster centroids. For each point in dataset, find centroid thats closest to that point and move the centroid towards the point. Repeat this until a stopping criterion is achieved.
Evaluate K-means clustering
Desired partition is one such that sum of distances between each point and nearest cluster is minimised. K-means isn’t guaranteed to minimise this as its prone to finding local minima, it can be run repeatedly to find different minima.
What is an autoencoder?
A single hidden layer network, trained to reproduce the input. Typically this will produce a more compact representation of the data.
The first layer weights encode input, the second layer weights decode it.
What is a restricted Boltzmann machine?
This is like an autoencoder, but decoding weights are the same as encoding. Will minimise cost the same way as autoencoder.
What is a deep belief net?
An architecture of stacked RBMs. First train hidden layer to encode input data. Then treat encoded vector as new input and create a second layer to autoencode it. This is encoded as often as you want. In end, add output layer with a number of neurons corresponding to number of classes. If doing supervised proble, train with backpropagation.
What makes the deep belief net a generative model?
It is possible to compute data in a reverse direction, allowing it to generate imagined input for a given label.
