w8l1 sequence to sequence Flashcards
what are sequence to sequence models
input is any sequence
output is any sequence
neural machine translation?
machine translaiton using neural networks
use sequence to seuqnece models
end-to-end differentiable
what would the formula look like for if we want to find the beset spanish setnence y, given english sentence x
y’ = max p(y|x,0)
whats a diff between fnns and rnns
rnns loop back to the previous state to check what that was before sumbiting output
what do we put into our rnns
embeddings
the interal representation, the -> between the square
represents the sentence up to this point
what differens conditional language models to regular language models
P(y1…yn) = product P(y|y<t)
P(y1…yn |x ) = product P(y|y<t, x)
conditioned on x
if i have a question and an image, they can both be in the input and we can condition on that input
explain how an rnn encoder goes to classifcation
once you feed in all the input, take the hiden layer (vector H) and then map it using a linear layer to the number of labels
then push it through a softmax ill get a probabibility distbrution over that
what is an autoregressive model
models where information from pervious time steps is used to predict the current time step output
how is seq2seq optimized
simutaenously
why does back propagration called end to end
simutaenously teachers this part to generate what i want while encoding the other part so it becoems useful in the decoding process