DL test 2023

Question 1

You have to choose an activation function for a fully connected nn. Which of the following is most likely to lead to dead neurons?

Answer 1

Rectified Linear Unit

Question 2

At what stage of the life of a nn model is backpropagation used?

Answer 2

Training

Question 3

Which one is true? 1. The final activation in CBOW is the logistic sigmoid function since we only predict one word, 2. CBOW predicts words based only on the previous words, 3. CBOW and Skipgram ignore the order of words, 4. Skipgram predicts the context based on a word

Answer 3

CBOW and Skipgram ignore the order of words, Skipgram predicts the context based on a word

Question 4

A model for element classification of set data (i.e. assign a class for each element in the set) needs to be:

Answer 4

equivariant to permutation

Question 5

A fully connected (dense) nn is used to model data that resides on a grid domain (such as a 2D image). This model is:

Answer 5

Not translationally invariant nor equivariant

Question 6

Choose the best structure for a data efficient model that operates on graph data and assigns a class to a graph

Answer 6

Permutationally equivariant layer(s) followed by a global pooling layer(s) and a softmax layer

Question 7

Select the layer(s) of a CNN that inputs a tensor with dimensionality 64x64x28 and outputs an activation map with the dimensionality of 60x60x32

Answer 7

2D Convolutional layer, with 32 filters, each with a kernel of size (5x5), stride 1 and VALID padding

Question 8

The number of times a parameter is re-used in an RNN cell is proportional to:

Answer 8

The length of the sequence

Question 9

Which of the following activation functions have the right properties to be suitable for serving as a gating mechanism?

Answer 9

The cumulative distribution function of the standard normal distribution, hard sigmoid, logistic sigmoid

Question 10

What are the required characteristic(s) of the aggregation function of message passing graph nn?

Answer 10

Produce the average value of all input values, produce the same result for any permutation of the input values, deal with different number of input elements

Question 11

Select which statements are true for the message passing graph nn model

Answer 11

the model can learn a representation of a graph with a variable number of unordered edges and nodes, the model can learn a representation of a graph with a fixed number of nodes and bidirectional edges

Question 12

What is the number of iterations that a message passing graph nn needs to implement to guarantee that information from each node will reach each other node in a fully connected graph?

Answer 12

1

Question 13

The depth of the message passing graph nn model is proportional to the:

Answer 13

number of iterations of message passing

Question 14

When choosing an activation function for a fully connected neural network. Which of the following is more likely to cause vanishing gradients during training?

Answer 14

Logistic Sigmoid

Question 15

When choosing an activation function for a fully connected neural network. Which of the following is most likely to lead to “dead” neurons?

Answer 15

Rectified Linear Unit

Question 16

At what stage of the life of a neural network model is backpropagation used?

Answer 16

Training

Question 17

Word Embedding
What probability distribution does the model in diagram (a) above estimate? Give a formula for the correct probability distribution, e.g

Answer 17

P

Question 18

What probability distribution does the model in diagram (b) above estimate? Give a formula for the correct probability distribution, e.g.

Answer 18

p

Question 19

Word embeddings (e.g. CBOW or Skipgram) can be used to train embeddings of other types of data than just words. Which of the following input data types are suitable for the use of these techniques?

Answer 19

DNA sequences of genes in terms of their four bases (Adenine A, Cytosine C, Guanine G, Thymine T), Tap dance choreography in “Kahnotation” (see figure below),
Time series representing the values of stocks over time

Question 20

A model consists of 5 convolutional layers operating on a grid. This model is:

Answer 20

Equivariant to translation

Question 21

Select the layer(s) of a CNN that inputs a tensor with dimensionality 64x64x128 and outputs an activation map with the dimensionality of 60x60x32.

Answer 21

2D Convolutional layer, with 32 filters, each with a kernel of size (5x5), stride 1 and VALID padding.

Question 22

The task at hand is signature verification. You have access to a dataset of 50000 images of signatures. You only have 2-3 signatures per person.
During the operation of the system, a person provides identification and they sign a document. The task of the model is to verify that the provided signature corresponds to the one in the database.
Describe:
1. The data domain and its symmetries
2. The type of model that is motivated well for the given data domain and problem formulation
2. The loss function
3. How your model computes a verification score during the operation of the system

Answer 22

The task is metric learning since we have many images of signatures and not many classes. We use the siamese network to learn the signatures where we learn classes by taking one image which belongs to the class and another that does not belong to the class. We compare similarities between signatures. We have translation symmetry where the position of the signature with the image does not affect the model in verification and rotational symmetry where the orientation of the signature does not affect the model. The siamese network works outputs the similarity between images. The loss function is contrastive loss. The loss function aims to maximize the similarity between genuine signatures while minimizing the similarity between genuine and imposter signatures. We use the distance between images as a verification score where the lower distance means a higher score.

Question 23

Suppose the recurring weights W in vanilla RNN (so without gates) has matrix-norm
How fast does the sensitivity of an output o to an input x increase or decrease in terms of l?

Answer 23

It decreases exponentially in l

Question 24

What is the number of iterations that a Message Passing Graph Neural network needs to implement to guarantee that information from each node will reach each other node in a fully connected graph?

Answer 24

1

Question 25

To which of the following is the depth of a Message Passing Graph Neural Network model proportional?

Answer 25

Number of iterations of message passing

Question 26

The task is molecule classification. The molecules are represented as graphs, where the nodes are the atoms of the molecule and the edges are the bonds of the molecules. Each molecule in the training data is assigned a class label out of a fixed set of 10 labels.
Describe the components of a Message Passing Graph Neural network that is suitable for this task.

Answer 26

We have a graph structure representation that fits with a GNN. The message passing can describe how information is passed in the graph. At each iteration, a node is aggregated and information about the node is updated. The information on the neighborhood nodes and the node itself is updated and combined to capture both local and global information. The graph readout function summarizes the node representations to the final size. This is necessary for the final classification. Layers on the model will learn the high level features and patterns of the molecule and perform classification. The final layer produces the probability distribution over the class labels. The selected predicted class has the highest probability.

Question 27

Why do we need the reparameterizaton trick to train the VAE?

Answer 27

Without it, we cannot backpropagate through a stochastic node

Question 28

why do we need the reparameterization trick to train the VAE?

Answer 28

Without it we cannot backpropagate through a stochastic node

Question 29

Model collapse is an effect that occurs when training GAN models where

Answer 29

The model learns to represent only a small part of the distribution of the data

Question 30

The task at hand is signature verification. You have access to a dataset of 50000 images of signatures. You only have 2-3 signatures per person.
During the operation of the system, a person provides identification and signs a document. The task of the model is to verify that the provided signature corresponds to the one in the database.

Describe:
1. The problem/model formulation that you will use for this task
2. The model architecture and loss function
3. How you set up the training data and the training process
4. How you use your model (describe steps of the algorithm)

Answer 30

metric learning, few shot learning

Question 31

Describe a method for translating sentences from one language to another using a sequential ml model. You have access to a training set of pairs of sentences from the source and target language. The training sentences have a variable number of words. The model should be capable of producing more than one correct translation for a given input sentence. (You should not assume conditional independence of output words)

Describe:
1. The problem/model formulation that you will use for this task
2. The model architecture and loss function
3. How you set up the training data and the training process
4. How you use your model

Answer 31

sequential prediction problem

Question 32

The task is predicting the level of bioactivity of molecular compounds. In the dataset, the molecules are represented as graphs, where the nodes are atoms of the molecule and the edges are the bonds between the atoms. Each molecule is assigned a continuous bioactivity between [0,1]

Describe:
1. The problem/model formulation that you will use for this task
2. The model architecture and loss function
3. How you set up the training data and the training process
4. How you use your model

Answer 32

Graph structure with GAN

Question 33

Your task is to repair music recordings. The music recordings are 10 wave signal. Some of these music recordings are injected with very short interruptions (no sound) during transmission. The interruptions can be easily identified with a simple thresholding step and distinguished from quiet music parts. The solutions should fill in the gaps and restore the full music recordings. You are given a training set of music recordings without interruptions.

Specify:
1. The problem/model formulation that you will use for this task
2. The model architecture and loss function
3. How you set up the training data and the training process
4. How you use your model

Answer 33

sequence