Natural Language Processing (NLP) & Linguistic Nuances Flashcards
What are the five major stages of Natural Language Processing (NLP) in chatbots?
The five stages are Lexical Analysis, Syntactic Analysis (Parsing), Semantic Analysis, Discourse Integration, and Pragmatic Analysis.
What is Lexical Analysis in NLP?
Lexical Analysis is the first stage where raw text is broken into words and sentences (tokenization), parts of speech are tagged, and processes such as stemming, lemmatization, and noise removal are applied to prepare the text for further analysis.
What is Syntactic Analysis (Parsing) in NLP?
Syntactic Analysis, or Parsing, focuses on grammar and structure by analyzing how words relate to each other, checking sentence well-formedness, and identifying structures like noun phrases, verb phrases, subjects, and objects, often resulting in a parse tree representation.
What is Semantic Analysis in NLP?
Semantic Analysis extracts the literal meaning of words and phrases, interprets definitions, resolves word disambiguation (choosing the meaning that fits context), and can flag meaningless or contradictory sentences.
What is Discourse Integration in NLP?
Discourse Integration deals with context across multiple sentences by linking current input with previous dialogue, tracking pronouns and references, and ensuring consistency throughout the conversation.
What is Pragmatic Analysis in NLP?
Pragmatic Analysis interprets sentences in their intended context using real-world knowledge, understanding implied meanings, idioms, cultural nuances, sarcasm, and inferring the user’s true intent beyond literal text.
How does NLP handle ambiguity in user statements?
Ambiguity is handled by applying semantic, discourse, and pragmatic analysis to resolve uncertainties—using context to determine meanings (e.g., whether ‘duck’ is a noun or verb) and resolving ellipsis and anaphora.
What is sentiment analysis and why is it important for chatbots?
Sentiment analysis determines the emotional tone (positive, negative, or neutral) of user messages, enabling the chatbot to tailor its responses (e.g., apologizing or being reassuring) to improve user satisfaction.
What is context tracking (dialogue state tracking) in chatbots?
Context tracking involves remembering details from earlier in the conversation, such as user-provided information or previous dialogue, allowing the chatbot to reference earlier points and manage multi-turn interactions naturally.
What additional enhancements are often integrated into the basic NLP pipeline of chatbots?
Enhancements include Entity Recognition (extracting specific data), Intent Recognition (classifying user goals), Dialogue Management (deciding subsequent actions), strategies for handling ambiguity (follow-up questions), and Natural Language Generation (producing fluent, contextually appropriate responses).
