Hoorcollege 9 Translating ||: Neural methods

Question 1

Attention

Answer 1

Attention gives weights to each context word before putting into vector
* Fixed content without attention puts all the words of context in one vector

Question 2

RNN encoder

Answer 2

• Often bidirectional, which can be contextualize bettter

Transfomers:
* More efficient computing T(x1, x,) in one step via attention

Question 3

Translating with transformers

Answer 3

Encoder uses transformer block and decoder more powerful blocks with an extra encoder-decoder attention layer which is undirectional via MASK
* Row of self-attention is normalized by softmax

Question 4

Neural generation

Answer 4

Needs as many output vector D’s as words and highest score gets looked up

Question 5

Out of vocabulary

Answer 5

Approaches:
* <unk>
* copying, because it is probably a name
* Subword segmentation, such as BPE</unk>

Question 6

BPE (Byte pair encoding)

Answer 6

We try to find subwords using BPE and translate them

BPE finds most common letters co-occurring in a corpus and then treats those as 1 item
and goes on

OPUS: massive open sources parallel (meaning in multiple languages for 1 text) texts

Question 7

Sentence aligment

Answer 7

finding correspondence between source sentences and their
equivalent translations in the target text

Question 8

Backtranslation

Answer 8

If there is a small corpus for language x to y then train a model on it and
expand corpus with y to x translations and train it again

• You can align word vector spaces of the two languages and translate it word by word and
  use thee translations to train the model as well

Question 9

MT evaluation

Answer 9

• precision: % of ngrams of candidate translation in the reference &
• recall= % of ngrams of reference are in candidate translation

chrFB = (1 + B^2) * ((chrP * chrR) / (B^2 * chrP + chrR))
calculates the P of word sequences