Tutorial 2

Question 1

What are the two optimization steps and loss step you need to perform in the training loop?

Answer 1

optimizer. zero_grad() - To zero out accumulated gradients
loss. backward() - To backpropagate and accumulate gradients for all parameters.
optimizer. step() - To update all parameters in the model according to the optimizer.

Question 2

what does model(x).detach() do?

Answer 2

It constructs a new view on the tensor that is not attached to the computational graph.

Question 3

what is tf.nn.CrossEntropyLoss()

Answer 3

It is a loss function in pytorch that calculates the cross entropy loss. It also internally applies the softmax function so all inputs should be raw and unnormalized.

Question 4

What does torch.max(tensor, dim) give you?

Answer 4

It gives you the maximum values along the dimensions. It also returns the indices of the maximum values if multidimensional array is being called.

Question 5

What does model.eval() do?

Answer 5

This tells all the layers that you are in eval() mode, meaning batchnorm or dropout layers will work in eval mode rather than training mode.

Question 6

What does with torch.no_grad(): do?

Answer 6

Deactivates the autograd engine, resulting in less memory use and performance speedups, but no backpropagation.

Question 7

What does tensor.size(d) do

Answer 7

Gives you size along the dth dimension.

Question 8

How do you reshape a tensor

Answer 8

tensor.reshape()

Question 9

How do you some up all the elements of a tensor in a given dimension?

Answer 9

tensor.sum(d)