Linguistic Nuances Flashcards
5 stages of NLP (natural language processing)
- Lexical analysis
- Syntactic analysis
- Semantic analysis
- Discourse integration
- Pragmatic analysis
Lexical analysis
This is breaking down the input text into individual words and putting them into an array, a process known as tokenization.
Syntactic Analysis
Interprets the grammar and structure of a sentence. Specifically, it identifies the relationships between words and how they form grammatical structures.
Semantic analysis
This part of NLP focuses on understanding the meaning of the sentence and its individual words within the context it seems to present.
Discourse integration
This involves integrating the meaning of the sentence with the larger context of the conversation. It relates the sentence being processed to what has been previously said.
Pragmatic analysis
It adds to understanding the inferred meaning of the sentence by taking into account the social, legal, and cultural context of the sentence by relating to real-world knowledge.