Misc

Question 1

Beyond compute budget, model size, and number of training tokens, what affects performance and efficiency?

Answer 1

Hyperparameter selection

Question 2

VQGAN does what

Answer 2

a generative adversarial neural network that is good at generating images that look similar to others (but not from a prompt)

Question 3

CLIP

Answer 3

neural network that is able to determine how well a caption (or prompt) matches an image.

Question 4

LDM stands for

Answer 4

Latent diffusion model

Question 5

Two main types of supervised learning tasks

Answer 5

Classification and regression

Question 6

Three main types of unsupervised learning tasks

Answer 6

Clustering, association, dimensionality reduction

Question 7

Difference between unsupervised and supervised learning

Answer 7

Supervised learning uses labeled data. Unsupervised learning doesn’t use labeled data

Question 8

Benefit of sparse activation?

Answer 8

It is better to achieve the same result with a simpler data representation than a complex one

Question 9

Autoregressive

Answer 9

predicting a future outcome of a sequence from the previously observed outcomes of that sequence

Question 10

What does VQGAN stand for?

Answer 10

Vector Quantized Generative Adversarial Networks

Question 11

What are logits?

Answer 11

The unormalized probabilities outputted by a network, before being passed thru [softmax I assume]