Lecture 1 Flashcards
Introduction to NLP & Levels of Language
NLP - Linguistics
Formal, structural models of language
NLP - Computer Science
Internal representations of data and algorithms for efficient processing
NLP - Artificial Intelligence
Computational theory of human language processing
NLP - Cognitive Psychology
Human cognition in language
NLP - Statistics
Frequencies and probabilities of linguistic patterns
Computational Linguistics
doing linguistics on computers
Natural Language Processing (NLP)
A range of computational
techniques:
* for analyzing and representing
naturally occurring texts at various
levels of linguistic analysis
* for the purpose of achieving
human like language processing
* for a range of particular tasks or
applications
NLP Application Areas:
Machine Translation
conversion of text from one language to another
NLP Application Areas:
Information Extraction
populating a database with specific data found in text
NLP Application Areas:
Human computer Interfaces
NLP assistants, chatbots , interactive querying of databases
NLP Application Areas:
Summarization
abstraction and condensation of major points
NLP Application Areas:
Question Answering Systems
provision of best answer to the given question
NLP Application Areas:
Information Retrieval / Search Engines
provision of documents
NLP Application Areas:
Image to Text, Speech to Text and
Text to speech solutions
Support
Data:
Unstructured
Data that has no inherent structure and is usually stored as different types of files
eg txt, pdf, images, videos
Data:
Quasi-Structured
Textual Data with erratic formats that can be formatted with effort and software tools
eg JSON (JavaScript Object Notation)
Data:
Semi-Structured
Textual data files with an apparent pattern, enabling analysis
eg spreadsheets and XML files
Data:
Structured
Data having a defined data model, format, structure
eg database
Why is NLP difficult?
Usually people are largely
unaware of the complexity of the
language tasks they perform so
effortlessly (exception: adult
Why is NLP difficult?
- Subtleties of meaning
- Irony, sarcasm, humor, metaphor
- Ambiguity
- Ambiguity is a fundamental
problem of computational
linguistics - Resolving ambiguity is a crucial goal
Levels of Language Analysis (1)
Phonetic
Levels of Language Analysis (2)
Morphological
Levels of Language Analysis (3)
Lexical
Levels of Language Analysis (4)
Syntactic