Test 01 Flashcards
What is Localization in the context of chatbots?
Localization refers to how well a chatbot ‘speaks’ a specific language or dialect and the relevance of the resources it provides to a user’s locale.
It includes spelling, grammar, and the use of culturally appropriate words.
Why are chatbots sometimes unresponsive to prompts?
Chatbots make errors and are still learning; user interactions and feedback help them improve over time.
Feedback is crucial for refining chatbot responses.
Do chatbots have access to the internet?
No, chatbots do not have access to the internet and are not up to date with current information.
Avoid asking questions that require recent data or including links.
What should you do if a chatbot claims to perform impossible tasks?
You should penalize the chatbot for Truthfulness if it lies about its capabilities.
This includes pretending to have opinions on controversial topics.
What does a well-localized chatbot response look like?
It uses correct spelling, appropriate words, and native-like grammar for the specific dialect.
For example, American English should not use British spellings.
When providing a content text, does it change the type of Q&A prompt?
No, providing a content text does not make an Open Q&A prompt Closed Q&A.
The distinction is based on whether the answer can be found in the content.
What is a system prompt?
A system prompt is an instruction given to a chatbot to guide its responses, ideally used around 10% of the time.
It should be provided before initiating conversation.
Do prompts need to be in the same language for a single conversation?
Yes, all prompts should be in the same language as the project.
Consistency in language helps maintain clarity.
What should you do if a chatbot responds in the wrong language?
Select Major Issues for Instruction Following and evaluate the responses accordingly.
Do not rate a wrong-language response higher than a correct one.
True or False: Chatbots can remember prompts from previous turns.
True.
Chatbots should maintain context across turns.
Fill in the blank: A poorly rendered table in a chatbot’s response should _______.
not be penalized.
This is a known issue with the website, not the chatbot.
What should you do if your prompt for the first turn doesn’t fit the category?
Rewrite it or skip to a different category.
Ensure prompts align with the assigned categories.
What happens if both responses from different chatbots are in the wrong language?
Select Major Issues for Instruction Following for both responses.
You can select either as the better response based on normal criteria.
Why is it important to rate the chatbot’s responses for Truthfulness?
Rating for Truthfulness ensures the chatbot provides accurate information and does not misrepresent its capabilities.
This is crucial for maintaining user trust.
What is the significance of providing resources relevant to the user’s locale?
Resources should be useful to speakers of the dialect from the user’s country to ensure effective localization.
Irrelevant resources can confuse users.
What is the recommended approach if a chatbot ignores a system prompt?
Select Minor Issues for Instruction Following if this is the only instance of disobedience.
Multiple issues would warrant a Major Issues rating.
How should you categorize a prompt if unsure of its type?
Use your best judgment and categorize it as Other if it doesn’t fit any provided categories.
Ensure it really does not fit before categorizing as Other.
