Article: Will Transformers Take Over Artificial Intelligence?

Question 1

Source

Answer 1

Quanta: Stephen Ornes

Question 2

Compare and contrast other AI approaches with transformers

Answer 2

1) In other approaches to AI, the system would first focus on local patches of input data and then build up to the whole. In a language model, for example, nearby words would first get grouped together. 2) The transformer, by contrast, runs processes so that every element in the input data connects, or pays attention, to every other element. Researchers refer to this as “self-attention.” 3) This means that as soon as it starts training, the transformer can see traces of the entire data set.