M9

Question 1

Autoencoder

Answer 1

• Type of ANN
• learns to represent data in compressed form, then reconstruct the data to resemble initial input as close as possible
• 2 components:
  
  • encoder (compresses input data, maintains original features)
  • decoder (reconstructs initial input data from the compressed data)
• architecture has 3 components: encoder, bottleneck/latent space, and decoder
• can learn nonlinear dependencies, use convolutional layers, and use transfer learning

Question 2

Autoencoder properties

Answer 2

• data-specific (can only do stuff similar to what they were trained on)
• lossy (“decompressed outputs will be degraded compared to the original inputs”)
• “learns automatically from examples”

Question 3

How autoencoders deal with overfitting when “capacity is too large for the data”

Answer 3

• bottleneck layer
• train to denoise
• sparsity through regularization
• contractive penalty

Question 4

Overcomplete autoencoder

Answer 4

Autoencoder where the dimension of h is greater than or equal to the dimension of xi