lecture 11 Flashcards
limitations of rule-based systems
- high precision but low recall
- tedious
why end-to-end supervised learning
= perform a task directly from raw input data to the desired output
if a phenomenon is systematic, it should arise from examples
the learning algorithm will infer relevant features and tendencies itself if shown through examples
black box
we don’t know what the system learned
Internally, the system applies learned patterns to new data, but we, as the users, can’t always see or understand the specific logic it employed to reach its conclusions. This opacity requires us to conduct selective and detailed error analysis to better comprehend where and why the algorithm makes mistakes.
bias
- systematic error
- evaluation metrics do not reveal bias
- systems may perform well on the test set, but this is not indicative of how they will perform in the real world
–> better performance does not necessarily mean better understanding
consequences of bias
- behavior on new data in the real word might be worse (doesnt generalize)
- ethical issues
- scientific questions/hypothesis testing: better performance does not necessarily mean better understanding
natural language inference (NLI)
- premise and hypothesis
- does the hypothesis follow from the premise
- entailment, contradiction, neutral
- general means of assessing how well a system can understand natural language
NLI architecture
- sentences representing the premise and hypothesis are separately processed by their respective sentence encoders.
–> the output of the premise encoder is labeled ‘u’ and the output of the hypothesis encoder is labeled ‘v’. - These encoded vectors, ‘u’ and ‘v’, are then combined and passed through a fully connected layer which aims to capture the relationship between the two sentences.
- a softmax function is applied to produce the final classification, which could be ‘entailment’, ‘contradiction’, or ‘neutral’ based on the combined encoded information
possible shortcuts for NLI
- standard majority class
- superficial similarities between premise and hypothesis
–> e.g., exact word matches, similar phrases
–> can a model perform well using only BOW representations - accidental characteristics of the hypotheses.
–> certain unintended patterns or features in the hypotheses could skew predictions, leading models to latch onto these incidental cues instead of performing a true semantic analysis.
–> solution: use hypothesis-only baselines
hypothesis-only baselines
train state-of-the-art model on hypotheses only
- sentence is processed by a sentence encoder, which transforms it into a vector representation, labeled as (v) and passed through fully connected layer
- analyze characteristics in the hypotheses
lessons about shortcuts
- performance does not tell the full story
- a majority baseline is not a sufficiently high bar
- the process of dataset creation can introduce unwanted patterns that models may exploit. For instance, if certain words or phrases are more commonly associated with specific outcomes due to the way data was labeled, models will pick up on these regularities.
- it’s often simpler for models to exploit these regularities than to solve the actual task. This shortcut undermines the purpose of training these models to understand and process natural language accurately.
model interpretability
- structural analysis
- challenge datasets
structural analysis of a network
- what are the main components in a network
–> input layer, hidden layers, output layer - diagnostic classification or probing: test if weights carry information
analyze representations of a neural network
train a classifier to predict specific information from the representation
diagnostic classification/probing
- select a linguistic property/feature (e.g., POS, vector)
- extract the weights from a specific layer of the neural network.
- To test whether these weights carry useful information, we train a classifier using these weights as inputs and the labels of the linguistic property (e.g., noun, verb, etc.) as outputs.
- evaluate its performance.
–>If the classifier performs well, it suggests that the weights from that layer encapsulate significant information about the selected linguistic property.
amnesic probing
- use a diagnostic classifier to identify information
- remove the information
- compare model performance on a task before and after removal
–> e.g., remove syntactic information and check if the model can still perform on the language modeling task (masked word prediction)
–> addresses limitations
challenge sets
gaining insights into the internal workings of a neural network
- select carefully chosen input
- observe output
- gain insight about what happens in black box by systematically analyzing how different inputs affect the outputs
