TNNVSRNNWESLEY

Question 1

Which type of neural network (RNN or TNN) is the insurance company, RAKT, currently using for its chatbot?

Answer 1

RNN

Question 2

What is a key advantage of Transformer Neural Networks (TNNs) over Recurrent Neural Networks (RNNs) in processing language?

Answer 2

TNNs use a self-attention mechanism to capture relationships between words, while RNNs process sequential data.

Question 3

What is the purpose of the hidden layers in a Recurrent Neural Network (RNN)?

Answer 3

Maintaining the memory of the network.

Question 4

What mechanism do Transformer Neural Networks (TNNs) use to capture the relationships between different words in the input sequence?

Answer 4

Self-attention mechanism

Question 5

What is a potential problem when training RNNs (Recurrent Neural Networks) using backpropagation through time (BPTT)?

Answer 5

The vanishing gradient problem, which makes it difficult to learn long-term dependencies in data.

Question 6

How do Long Short-Term Memory (LSTM) networks address the vanishing gradient problem in RNNs?

Answer 6

With a three-gate mechanism (input, forget, output gates) that controls the flow of information.

Question 7

Give an example of a large language model that uses the self-attention mechanism found in Transformer Neural Networks (TNNs).

Answer 7

Generative Pre-trained Transformer 3 (GPT-3)

Question 8

What type of data are RNNs designed to process?

Answer 8

Sequential data

Question 9

In an RNN, what is the role of the input layer?

Answer 9

Processes the sequence of data, with each element in the sequence fed as an input to the network.

Question 10

What is the full name of TNN, and what is its key innovation?

Answer 10

Transformer Neural Network, the key innovation being the application of a self-attention mechanism.