Hugging Face ecosystem | HF NLP course | 2. Using Hugging Face Transformers | Other

Question 1

[page] Models: [page section] Creating a Transformer: [q] To initialize a BERT model from scratch, you first need to initialize a ? and a ?

Answer 1

from transformers import BertConfig, BertModel
					
# Building the config
config = BertConfig()
					
# Building the model from the config
model = BertModel(config)

Source

Question 2

[page] Models: [page section] Creating a Transformer: [section] Different loading methods: [q] Code to load a Transformer model with a specific architecture that is already trained.

Answer 2

from transformers import BertModel
					
model = BertModel.from_pretrained("bert-base-cased")

Source

Question 3

[page] Models: [page section] Creating a Transformer: [section] Different loading methods: [q] How can you can customize your cache folder?

Answer 3

HF_HOME environment variable

Source

Question 4

[page] Models: [page section] Creating a Transformer: [section] Saving methods: [q] How do you save a model?

Answer 4

model.save_pretrained(“directory_on_my_computer”)

Source

Question 5

[page] Models: [page section] Using a Transformer model for inference: [q] What does the model accept as inputs? What creates these inputs?

Answer 5

Tensors (of rectangular shape). Tokenizers.

Source

Question 6

[page] Models: [video] Instantiate a transformers model: [q] To load a model configuration from a checkpoint?

Answer 6

bert_config = AutoConfig.from_pretrained("bert-base-cased")

Source

Question 7

[page] Tokenizers: [page section] Loading and saving: [q] Code to load the BERT tokenizer trained with the same checkpoint as BERT.

Answer 7

from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")

Source

Question 8

[page] Handling multiple sequences: [page section] Models expect a batch of inputs: [q] For a single sequence, initialize a model for classification and show the intermediate methods of the tokenizer to generate ids for the example (3 steps).

Answer 8

"import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
					
checkpoint = ""distilbert-base-uncased-finetuned-sst-2-english""
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSequenceClassification.from_pretrained(checkpoint)
					
sequence = ""I've been waiting for a HuggingFace course my whole life.""
					
tokens = tokenizer.tokenize(sequence)
ids = tokenizer.convert_tokens_to_ids(tokens)
					
input_ids = torch.tensor([ids])
print(""Input IDs:"", input_ids)
					
output = model(input_ids)
print(""Logits:"", output.logits)"

Source