AI Pract Flashcards
LLM
Transformer architecture most common - LLMs genereate human text. Trained on vasts amounts of text data
How LLMs work
Tokens, embeddings and vectors
Token
basic unit of text that model processes
embeddings
numerical representation of tokens
vectors
a list of numbers that capture it’s meaning in relationship to other tokens
diffusion models
deep learning architecture system
how does diffusion work
Starts with pure noise or random data and gradually adds more information to create clear output. Two step process noisy to clean, clean to noisy
Forward Diffusion
Clean image to start and introduces noise until just noise is left
Reverse diffusion
noisy image gradually denoised until new image generated
multimodal models
can generate multiple modes of data simultaneously
when to use multimodal
automating video captions, creating graphics from text input, answering questions beter by combining text and visuals, translating content while keeping visuals
Generative adversarial networks (GANs)
Two neural networks competing in a zero-sum game framework.
GAN - Generator
network that generates new synthetic data (ex. image, audio, text) taking random noise as input. tries to trick the discriminator
Discriminator
Takes real data from training set and syntheticly generated data as input trying to distinguish between the two.
Variational autoencoders (VAEs)
generative model that combines ideas from autoencoders and variational inference. Two parts, encoder and decoder
