New Flashcards
What is the purpose of the BigramLanguageModel class in PyTorch?
It is designed to predict the next token based on the current token using a lookup table for embeddings.
The model uses a vocabulary size to create a token embedding table.
What does the forward method in the BigramLanguageModel class do?
It takes input indices and targets, and returns logits from the token embedding table.
The logits shape is (B, T, C), where B is batch size, T is sequence length, and C is vocabulary size.
What does the torch.allclose function do?
It returns true if two tensors are close in value, within a specified tolerance.
What is the purpose of the nn.Sequential class in PyTorch?
It allows you to create a neural network by stacking layers in a sequential manner.
How can layers be passed to nn.Sequential?
Layers can be passed as:
* Direct arguments
* List
* Tuple with names
* OrderedDict
Each method provides a different way to organize and name the layers.
What does the torch.clamp function do?
It clamps the values of a tensor within a specified range.
What is the structure of a tensor created with requires_grad=True?
It allows for gradient accumulation during backpropagation.
Fill in the blank: In PyTorch, you can use _______ to apply multiple transformations to a dataset.
transforms.Compose
What is the function of the TransformedDataset class?
Make an example
It wraps an original dataset and applies transformations to its items when accessed.
from torch.utils.data import Dataset
class TransformedDataset(Dataset):
def \_\_init\_\_(self, dataset, transform=None):
self.dataset = dataset
self.transform = transform
def \_\_getitem\_\_(self, idx):
data = self.dataset[idx]
if self.transform is not None:
data = self.transform(data)
return data
def \_\_len\_\_(self):
return len(self.dataset)
What does the __getitem__ method do in the TransformedDataset class?
It retrieves an item from the original dataset and applies the transformation if provided.
What is TMUX used for in Linux?
It is a terminal multiplexer that allows users to create and manage multiple terminal sessions.
True or False: The command ‘ctrl b + %’ in TMUX splits the window vertically.
True
What command is used to detach from a TMUX session?
ctrl b + d
What does the command ‘grep [pattern] [file_directory]’ do?
It searches for a specified pattern within files in the given directory.
What is the purpose of the manual_seed function in PyTorch?
It sets the seed for generating random numbers for reproducibility.
What is the output of accessing y.grad_fn in the context of PyTorch tensors?
It provides the gradient function that generated the tensor y.
Fill in the blank: In terminal, to create a new horizontal pane in TMUX, you use the command _______.
ctrl + b + “
What does the wei tensor represent in the context of matrix multiplication?
It represents a weighted aggregation used for causal sums.
