Lecture 6 - Dialog Systems and Chatbots - Conversational Agents Flashcards
What are conversational agents? give two types of them as well
Conversational agents communicate with users in natural language (text, speech)
- chatbots can mimic conversations with people
- dialog systems are designed for a particular task (Siri, Alexa)
True or False. There are two types of chatbots (that we have learned of at least): rule-based chatbots and IR-based chatbots
True. Eliza and Parry are examples of rule-based chatbots
What do rule-based chatbots use? What can they not do?
They use a set of rules
but, cannot answer question outside the defined rules, and cannot learn from interactions
How do rule-based chatbots work?
Rule-based chatbots depend on keywords in the questions they receive to understand the queries. Then, the chatbot will research predefined answers to provide a relevant response.
All those keywords are associated with a transformation that is ranked, and the chatbots are trying to reply with the highest rank
True or False. In a situation where a rule-based chatbot cannot find a matching keyword to reply, they do not say anything.
False. When no keyword matches, it chooses a noncommittal response
“go on”
“I see”
“interesting”
What are the exact steps of a rule-based chatbot?
Find the word w in sentence that has the highest keyword rank
If w exists,
* check each rule for w in ranked order, choose the first one that matches sentence; response <= apply transform
Else,
- response <= apply NONE transform (noncommital response) or
- response <= grab an action in memory queue
What are the differences between rule-based chatbots Eliza and Parry?
Parry uses the same pattern-response structure as Eliza, but it has a much richer control structure, language understanding capabilities, and a mental model (affective variables)
Where do IR (information retrieval) chatbots get their data from?
IR-based chatbots mine conversations of human chats or human-machine chats
They use corpora like microblogs (Twitter) or movie dialogs
What are the two IR-based chatbot architectures?
- Return the response to the most similar turn
* take user’s turn (what the user said) and find a (td-idf) similar turn t in the corpus C - Return the most similar turn
True or False. IR-based models can use other features like previous turns or information about the user.
True
or non-dialog texts like (information from Wikipedia)
Name some pros and cons of chatbots.
PRO:
fun
applications to counseling
good for narrow, scriptable applications
CON:
they don’t really understand
rule-based chatbots are expensive and don’t generalize well
IR-based chatbots can only mirror training data
What are frame-based dialog agents based on?
Frame-based dialog agents are based on “domain ontology” - a knowledge structure representing users intention
(flight booking for example)
What are frame-based dialog agents composed of?
Frame-based dialog agents are composed of one or more frames, that each has a collection of slots, and each slot has a value
each slot is associated with a type and a question
e.g.
SLOT-TYPE-QUESTION
departure-city-What city are you leaving from?
destination-city-Where do you want to go?
departure date-date-What day would you like to leave?
etc.
True or False. In frames, the types of slots can have a hierarchical structure.
True.
DATE: year, month, day etc.
In frame-based dialog agents, there is a finite control structure. What does that mean?
The system completely controls the conversation with the user.
It asks the user a series of questions.
Ignores (or misinterprets) anything the user says that is not a direct answer to the system’s questions
btw: systems that control conversations like this are called single initiative
Pros and cons of single initiatives in frame-based dialog agents
PRO: easy to build, OK for very simple tasks, both user and system know what the other one can say next
CON: too limited
One solution for the limited single initiatives of frame-based dialog agents is mixed initiatives. What do those entail?
Instead of taking one question and one answer at a time, the system can process multiple answers at a time
The system asks questions to the user, and fills any slots that the user specifies => when the frame is filled (all the slots are filled) do database query
e.g., single initiative: When? Tomorrow. Where? Home
mixed initiative: When? Tomorrow at home
fills both slots when and where
SIRI uses that btw
How can you use supervised machine learning to train a frame-based dialog agent?
Train on labeled data such as
“I want to fly to San Francisco on Monday afternoon”
DESTINATION: San Francisco
DEPARTURE DATE: Tuesday
Build a classifier to map from sentence to slots that the user want to fill in (logistic regression, neural networks)
What are the two metrics that we can use to evaluate the model performance for frame-based dialog agents?
- slot-error rate
2. end-to-end evaluation
