Lecture 5

Question 1

What is Semantics in NLP?

Answer 1

Semantics is the study of meaning in language, focusing on understanding references or truth within text and speech.

Question 2

Why are Word Representations important in NLP?

Answer 2

Word representations allow us to find documents with similar meaning rather than exact word matches, improving search and retrieval.

Question 3

What does “You shall know a word by the company it keeps” mean?

Answer 3

It implies that words used in similar contexts tend to have similar meanings, a foundation for distributional semantics.

Question 4

What are Word Embeddings?

Answer 4

Word embeddings are vector representations of words, capturing semantic similarity by placing similar words close in vector space.

Question 5

Describe Word2Vec and its two main methods.

Answer 5

Word2Vec creates word embeddings through Skip-gram (predicting context from target word) and CBOW (predicting target word from context).

Question 6

What is a limitation of Word2Vec?

Answer 6

Word2Vec generates a single representation per word, which doesn’t account for polysemy (words with multiple meanings).

Question 7

What is Cosine Similarity in the context of word embeddings?

Answer 7

Cosine similarity measures the similarity between two word vectors as the normalized dot product of those vectors.

Question 8

What are Contextualized Embeddings?

Answer 8

Contextualized embeddings, like those from BERT, create word representations that vary depending on the word’s context within a sentence.

Question 9

Name two examples of Contextualized Embedding Models.

Answer 9

ELMo and BERT are examples of models that create context-dependent embeddings.

Question 10

What is Sentence-BERT used for?

Answer 10

Sentence-BERT is used to create sentence embeddings, allowing for efficient comparison of sentence meanings.

Question 11

Define TF-IDF and its purpose.

Answer 11

TF-IDF (Term Frequency-Inverse Document Frequency) is a method to weight terms in a document based on their frequency and importance, improving information retrieval.

Question 12

How does TF-IDF work?

Answer 12

TF-IDF assigns higher weights to words that are frequent in a document but rare in the entire corpus, reducing the impact of common words.

Question 13

What is Intrinsic Evaluation in evaluating embeddings?

Answer 13

Intrinsic evaluation assesses embeddings by comparing algorithm-generated word similarity scores to human-annotated scores.

Question 14

What is Extrinsic Evaluation in evaluating embeddings?

Answer 14

Extrinsic evaluation tests embeddings in real NLP tasks (e.g., information retrieval) to measure their practical effectiveness.

Question 15

What are Bilingual Embeddings?

Answer 15

Bilingual embeddings align words from two languages in the same vector space, enabling cross-lingual tasks.

Question 16

What is the goal of Cross-Lingual Embedding?

Answer 16

Cross-lingual embedding aims to position words with similar meanings from different languages close together in vector space.

Question 17

Describe the Direct Transfer approach in bilingual embeddings.

Answer 17

Direct Transfer trains a model in a resource-rich language (e.g., English) and applies it to a low-resource language by mapping embeddings.

Question 18

What is the Naive Approach to bilingual embeddings?

Answer 18

The naive approach uses a bilingual dictionary to align words between languages, assigning similar embeddings to translation pairs.

Question 19

What is a Transformation Matrix in bilingual embeddings?

Answer 19

A transformation matrix aligns monolingual embeddings by learning a linear mapping between two languages based on word pairs.

Question 20

What are Multilingual Embeddings?

Answer 20

Multilingual embeddings map more than two languages into a shared vector space, supporting tasks across multiple languages.