Lecture 1

Question 1

norbert wiener:

what distinguished human communication from the communication of most other animals

Answer 1

1. the delicacy and complexity of the code used
2. the high degree of arbitrariness of this code

Question 2

syntax

Answer 2

rules and principles that govern the structure of sentences in a language

Question 3

part of speech (POS)

Answer 3

labels that indicate the grammatical category of a word

Question 4

morphology

Answer 4

form and structure of words

• how words are created, how they can change form, and how these forms contribute to their meaning
• ‘is’ is a form of ‘be’ in the 3rd person singular, present tense

Question 5

semantics

Answer 5

meaning of words and the sentence as a whole

Question 6

discourse

Answer 6

considers the sentence within the context of larger communication

Question 7

natural language processing

definition + process

Answer 7

giving computers the ability to understand text and spoken words in much the same way humans can

1. representing input
2. generating output
3. computational modeling

Question 8

NLP: representing input

Answer 8

represent language in a way that a computer can process it

e.g., tokenization, stemming, lemmatization

Question 9

NLP: generating output

Answer 9

process language in a way that is useful for humans

Question 10

NLP: computational modeling

Answer 10

understanding language structure and language use

Question 11

gap of 4-5 orders of magnitude between the amount of inputs LLMs receive compared to humans

Answer 11

LLMs are trained on datasets that are much larger than the amount of language input humans receive over a similar timespan

Question 12

different scenarios for understanding and developing human-scale and LLM-scale intelligence

Answer 12

1. innate (nativist) - humans
2. grounding (multimodal) - both
3. active/social (constructivist) - LLMs
4. evaluation (humans and LLMs are fundamentally misaligned) - LLMs

Question 13

stochastic parrot

Answer 13

The term “stochastic parrot” is used metaphorically to describe a model that generates text by probabilistically selecting words based on learned patterns, much like how a parrot might repeat phrases without understanding their meaning.

In NLP, probabilistic models use statistical methods to predict the likelihood of word sequences. They rely on large amounts of data to learn these probabilities and generate coherent text.