Tutorial 1 Flashcards
torch.tensor()
A torch.Tensor is a multi-dimensional matrix containing elements of a single data type
How CPU and GPU torch.tensor() types exist?
8 CPU and 8 GPU
Can a torch.tensor() be created from a Python list?
Yes, like torch.tensor(np.array([[1, 2, 3], [4, 5, 6]]))
Does torch.tensor() copy data?
Yes. If you want to change a numpy array to a tensor, use as_tensor()
How do you use slice notation with torch.tensor()?
Like you would a numpy array.
what is requires_grad argument in torch.tensor()?
It allows Pytorch to record operations including that tensor for automatic differentiation.
How do you see the value of a torch.tensor()?
.item()
Where do gradients accumulate when calling .backward()?
In the leaves
What does a .backward() call do?
Computes the dloss/dx for every gradient enabled variable, and accumulates into the x.grad for every parameter x
torch.randn(size)?
provides a pytorch tensor with each element sampled from the standard normal distribution
torch.nn.Linear(in_features, out_features, bias=True)?
Applies a linear transformation to the incoming data: y = xW^T + b. W and b are randomly initialized
what does nn.loss() create?
Creates a criterion, which you can then supply an input and target for that loss function.
what does optimizer.step() do?
Updates the parameters based on current gradients stored in the .grad attribute of each paramter.
What is a Pytorch parameter?
A parameter is still a Tensor, much like a normal variable, however when associated with a model, it becomes part of the model attributes. This means we can easily feed model parameters to the optimizer. Parameters automatically has requires_grad() set to true.
What does sub() and add() do?
Add the argument to the tensor. If multiple elements, the argument is added to each element.
What is the difference between sub() and sub_()?
sub_() is an in-place tensor operation.
How do you convert from Pytorch tensor to a numpy array?
.numpy()
What methods do you have to override to create a custom python dataset?
__len__(self) and __getitem__(self, index)
What should __len__() do in a dataset?
Give the length of the dataset
What should __getitem__(index) do in a dataset?
Give you the value in the dataset at index, after loading, transforming, etc.
What should __init__(self) do in a dataset?
Initialize filepaths or list of filenames
What does .state_dict() return
A dictionary containing the whole state of the module.
How do you save a pytorch model?
You can do torch.save(model, ‘model.ckpt’), or torch.save(model.state_dict, ‘model.ckpt’). The latter is preferred as then intermediates are not saved.
How do you load from state_dict?
model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH)). Note you have to load the model first before you need to load_state_dict
