CHAPTER 1 Flashcards
………………is a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human (natural) languages.
Natural language processing (NLP)
NLP devoted to creating computers models that use natural language as…………and/or …………..
input and/or output.
T/F NLP is about trying to get a machine brain to work the same way as a human brain when it comes to language.
T
Why NLP is important?
To interact with computing devices using human (natural) languages.
To access (large amount of) information and knowledge stored in the form of human languages quickly.
Practical examples:
Building intelligent robots (AI).
Enabling voice-controlled operation.
What are the challenges?
-Ambiguity in language
-Human language is astoundingly complex and diverse
-Errors relevant to Speech and Text
-Apathy towards Vertical-Specific Lingo
Ambiguity can be of 2 types:
-Lexical – multiple meanings of words: It is dealt with in “lexical semantics
2-Structural: It is dealt with in parsing.
Language Processing Levels
L1: Speech sound(Phonetics & Phonology )
L2: Words & their forms(Morphology, Lexicon)
L3: Structure of sentences(Syntax , Parsing)
L4: Meaning of sentences(semantics)
L5:Meaning in context & for a purpose (pragmatics)
L6:Connected sentence processing in a larger body of text(Discourse)
Examples of Levels:
L1 : sound
L2 : Dog - Dog(s), Dog(ged)
Lady – Lad(ies)
L3 : Ram goes to market (right)
goes Ram to the market (wrong)
L4 : translation from unstructured to structured representation
L5 : User situation & context
“Is that water?” – the action to be performed is different in a chemistry lab and on a dining table.
L6 : Backward & forward references –
Coreference resolution
“The man went near the dog. It bit him.”
Often co reference & ambiguity go together as in
“The dog went near the cat. It bit it.”
Three prominent application areas:(Some NLP Applications)
-Text analytics/mining (from “unstructured data”):
Sentiment analysis
Topic identification
-Conversational agents :Siri, Cortana, Amazon Alexa, Google Assistant
Chatbot
Machine translation
Some NLP Applications (cont.)
-Text Analytics
-Conversational Agents
what is text analytics ?
Data-mining of weblogs, microblogs, discussion forums, user reviews, and other forms of user-generated media.
Typically this involves the extraction of limited kinds of semantic and pragmatic information from texts
-Entity mentions
-Concept identification
-Sentiment
what is Sentiment Analysis
The field of sentiment analysis deals with categorization (or classification) of opinions expressed in textual documents
what is Conversational Agents?
Combine
-Speech recognition
-Question answering
what is Machine Translation
The automatic translation of texts between languages is one of the oldest non-numerical applications in Computer Science.
Translating documents from one language to another (for example, producing automobile repair manuals in many different languages)
How does NLP work?
NLP is similar to how humans think and learn. Humans start by learning the basics. We first learn words, then sentences. From there, a level of complexity is built. The same can be said for NLP.
This type of technology is all about statistics and probability. From the meaning in the language of the text, we extract the value for each word and we extract the value between the relationship of the words.
NLP applications require several NLP analysis:
-Sentence and/or Word tokenization
-Part-of-speech tagging (abbreviated as PoS tagging)
-Named Entity (NE) recognition
-Parsing
-Stemming
-Lemmatization
-Stemming &Lemmatization
-Stopword Removal
-Semantic analysis
…………………….taggers with high level of accuracy can solve Word’s syntactic ambiguity. On the other hand, the problem of resolving semantic ambiguity is called WSD (word sense disambiguation).
Part-of-speech (POS) taggers
…………is useful for identifying relationships between words and, therefore, understand the meaning of sentences.
PoS tagging
…………….. is the process of determining the syntactic structure of a text by analyzing its constituent words based on an underlying grammar (of the language).
Parsing
about parsing?
Breaking a sentence as per grammar:
Like a Sentence is broken into Noun Phrase and Verb Phrase.
Noun Phrase could be again Article + Noun
example of parsing
Parsing (cont.)
Example:
The boy ate the pancakes.
This sentence has the following structure:
The boy: Noun Phrase
ate: Verb
the pancakes: Noun Phrase (Determiner + Noun)
what is stemming
consists of extracting the root of the word.
wha is Lemmatization
allows reducing a word to its canonical form (the lemma). It is also called the “dictionary” form.
