Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

CSE 476 > Midterm Review > Flashcards

Midterm Review Flashcards

(104 cards)

Start Studying
1
Q

What does NLP stand for?

A

Natural Language Processing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the two main categories of tasks in NLP?

A
  • Pre-neural
  • Post-neural and pre-LLM
  • Post-LLM
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the purpose of sequence tagging/classification tasks?

A

They mostly serve as features for downstream applications

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the role of Part of Speech Tagging in NLP?

A

Widely adopted as features in ML models

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What does morphology tagging involve?

A

Lemmatization and reducing words to their simplest form

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is Dependency Parsing used for?

A

Understanding sentence structure and semantic disambiguation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does Semantic Role Labeling (SRL) identify?

A

Identifies predicates and roles of elements in sentences

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is Named Entity Recognition (NER)?

A

Identifies all named entities in text that belong to certain types

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What type of models are Markov Chains?

A

Chains of states with transition probabilities

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is a language model?

A

A model that predicts the next word/token in a sequence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is the objective of a language model?

A

Compute the probability of a sentence or sequence of words

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Define perplexity in the context of language models.

A

Inverse probability of test text normalized by the number of words

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are some common classification tasks in NLP?

A
  • Document Classification
  • Spam Detection
  • Sentiment Analysis
  • Textual Entailment
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the goal of generative classifiers?

A

To model P(x, y) = P(x | y) P(y)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the main advantage of discriminative classifiers?

A

Few assumptions and more direct modeling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is the function of the softmax function in multi-class logistic regression?

A

Converts logits into probabilities that sum to 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Fill in the blank: In Naive Bayes, you estimate P(x|c) and P(c) for classification based on _______.

A

features

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is the purpose of TF-IDF in text representation?

A

To weigh the importance of a term in a document relative to the corpus

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is Word2Vec used for?

A

To train a classifier to predict word embeddings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is the concept behind the skip-gram architecture in Word2Vec?

A

Uses nearby words to predict the target word

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is the primary function of a neural unit in a neural network?

A

To carry information and produce a non-linear activation value

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is the significance of using non-linear activation functions in neural networks?

A

They enable the network to approximate complex functions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What does RNN stand for?

A

Recurrent Neural Network

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is the purpose of training an RNN?

A

To handle sequences and maintain context from previous inputs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
What is the primary limitation of feed-forward neural networks?
They assume a fixed input size and struggle with temporal orders
26
What is cross-entropy loss used for in classification tasks?
To measure the performance of a model whose output is a probability value
27
True or False: Stochastic Gradient Descent (SGD) typically converges slower than Gradient Descent (GD).
False
28
What is the goal of optimization in machine learning?
To minimize the loss function
29
What do we call the process of updating weights in a neural network based on the loss?
Backpropagation
30
What is the purpose of training a RNN?
To handle sequences and generate predictions based on previous input.
31
What does 'unrolling' a RNN entail?
Generating a feedforward network specific to the input sequence.
32
What is a common loss function used in RNN for language modeling?
Cross entropy loss.
33
What is the size of the input matrix representing words in RNN LM?
|V| X H.
34
What matrix is used for output word probability distributions in RNN?
H X |V|.
35
What is a drawback of basic RNNs?
Early tokens receive very little attention in later context.
36
What does LSTM stand for?
Long Short-Term Memory Network.
37
What are the two subproblems addressed by LSTMs?
* Removing unnecessary info from context * Adding relevant info.
38
What is the function of the Forget Gate in an LSTM?
Decides on what info to forget from the current cell state.
39
What is the purpose of the Add Gate in an LSTM?
To decide which parts of new info should be added to the cell state.
40
What does the Output Gate in an LSTM do?
Decides which parts of cell state should be output for predictions.
41
What is a variant of LSTMs that allows gates to peek at cell states?
Peephole Connections.
42
What is a simpler model that merges cell states and hidden states?
Gated Recurrent Unit (GRU).
43
What is the main challenge when using Bidirectional LSTMs?
The backward layer has already seen next words that are being predicted.
44
What is ELMo used for?
To compute contextualized embeddings for entity typing.
45
What is the first step in using ELMo for entity typing?
Find all common contexts for each Wikipedia entity.
46
What are common tasks RNNs struggle with?
* Machine translation * Summarization * Question answering.
47
What is the role of an encoder in encoder-decoder models?
To generate contextualized representations of the input sequence.
48
What does Beam Search do during inference?
Keeps top-k most probable sequences.
49
What is the purpose of Top-K Sampling?
To improve diversity and creativity in generated sequences.
50
What does the term 'temperature' refer to in sampling methods?
It scales logits before softmax to control randomness.
51
What does BLEU measure?
N-gram precision in translations.
52
What does ROUGE-L measure?
Longest common subsequence in translations.
53
What is the BERT Score based on?
Cosine similarity between contextualized word embeddings.
54
What is the main advantage of self-attention in transformers?
It helps compute latent representations by attending to surrounding tokens.
55
What are the three roles in the actual attention mechanism?
* Query * Key * Value.
56
What is Multi-head Attention used for?
To capture more comprehensive and diverse information.
57
What is the purpose of positional embeddings in transformers?
To represent the temporal order of words.
58
What is a common type of positional encoding?
Sinusoidal Positional Encoding.
59
What is the advantage of learned positional embeddings?
They treat positional embeddings as learnable parameters.
60
What is the main function of T5 in NLP?
To unify different tasks into input-output sequences.
61
What is the goal of instruction tuning in models like Chat-GPT?
To train the model with human-labeled data for specific tasks.
62
What is a key characteristic of autoregressive models?
They generate tokens sequentially.
63
What is the Byte Pair Encoding technique used for?
To handle out-of-vocabulary words efficiently.
64
What is one of the benefits of Mixture of Experts (MoE)?
Computation efficiency at inference time.
65
What does in-context learning refer to in LLMs?
Learning tasks by examples provided in the input context.
66
What is zero-shot prompting?
Providing only the task description for the model to generate an answer.
67
What is the limitation of GPT-3 in instruction-following?
It often fails when the task is unclear or lacks proper context.
68
What kind of data did BERT use for training?
Wikipedia and books.
69
What is the significance of larger model sizes in LLMs?
They often lead to better performance and capabilities.
70
What is the first real LLM developed by OpenAI?
GPT-3.
71
What is the main limitation of pretrained LLMs?
They don't work well towards tasks.
72
What is instruction tuning?
Training the model with human-labeled data on how specific instructions or tasks should be handled.
73
What does GPT-3 primarily focus on during its training?
Next word prediction.
74
What is Chat-GPT?
GPT-3 further trained on human-label supervised data to acquire its 'assistant' behavior.
75
What type of responses can instruction tuning target?
Natural language explanations, not just answers.
76
What is UnifiedQA primarily focused on?
Question answering.
77
What does Flan-T5 build upon?
UnifiedQA but with more diverse data.
78
What is InstructGPT?
GPT-3 trained on instruction data from human labeling.
79
What does ChatGPT's instruction-tuning data contain more of?
Multi-turn conversational instances.
80
What is the purpose of Alpaca?
To replicate instruction following capabilities by letting GPT-3.5 generate instruction examples.
81
What is a characteristic of generative QA?
Asks models to provide a short answer.
82
What is SQuAD?
Standard Question Answering Dataset.
83
What does selective QA allow for?
Flexibility with deterministic evaluation metric (accuracy).
84
What does GSM8K refer to?
Grade school math problems.
85
What is DROP?
Discrete Reasoning Over the content of Paragraphs.
86
What is the purpose of the Putnam Bench?
To evaluate mathematical proof correctness.
87
What is a hallucination in the context of LLMs?
When LLMs use incorrect facts, knowledge, or reasoning paths.
88
True or False: Factual hallucinations are when LLMs provide incorrect historical facts.
True.
89
True or False: Commonsense hallucinations involve LLMs making logical errors based on everyday reasoning.
True.
90
What is the inverse scaling evaluation?
Evaluates if larger LMs perform better on tasks.
91
What is alignment in the context of LLMs?
The process of aligning model behavior to human expectations.
92
What is safety alignment?
Providing instructions as supervision to ensure models do not generate harmful content.
93
What is the challenge with labeling 'good' and 'bad' responses?
Good and bad are often relative and hard to quantify.
94
What is Reinforcement Learning (RL)?
A machine learning technique that teaches models to learn by trial and error based on reward signals.
95
What is PPO?
Proximal Policy Optimization, a standard RL objective.
96
What does Direct Preference Optimization (DPO) aim to achieve?
Train models without needing a reward model.
97
What is 'jailbreaking' in the context of LLMs?
Bypassing model's alignment.
98
What is cognitive overload?
When a model is overwhelmed by too much information, leading to incorrect outputs.
99
What is RLAIF?
Uses AI feedback for alignment.
100
What is the purpose of inference-time methods?
To acquire desired behaviors from models without changing model weights.
101
What technique involves sampling multiple chain-of-thought?
Self-Consistency.
102
What is the goal of multi-LLM debating?
To reach an agreement between multiple models.
103
What does RAP stand for?
Reasoning via Planning.
104
What is RAG?
Retrieval Augmented Generation.

CSE 476 (1 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions