Bias 2 Flashcards
Which type of dataset bias occurs when the training data doesn’t reflect changes over time?
Historical bias is described as occurring when the training data do not reflect changes over time.
Example: A model trained on data from several years ago might not accurately predict recent customer queries.
What type of dataset bias is represented if the RAKT chatbot’s training data only included customer queries related to car insurance policies?
This would likely represent confirmation bias.
Confirmation bias occurs when the dataset is biased towards a particular viewpoint.
What type of bias is created if the labels assigned to customer queries in the training data are very broad?
This would create labeling bias.
Labeling bias occurs when the labels applied to the data are subjective, inaccurate, or incomplete.
What type of bias might be introduced if the RAKT chatbot was trained primarily on formal, standard English?
This would introduce linguistic bias.
Linguistic bias occurs when the dataset is biased towards certain linguistic features, such as dialect or vocabulary.
What type of bias might be created if RAKT only collected training data from customers in a specific geographic region?
This would create sampling bias.
Sampling bias occurs when the training dataset is not representative of the entire population.
How is selection bias defined in the context of the chatbot’s training data?
Selection bias is defined as occurring when the training data are not randomly selected but are instead chosen based on some criteria.
Example: A language model trained on data suggesting certain demographics are more likely to file claims.
Could reporting bias be a potential issue in the customer complaints analyzed?
Yes, reporting bias could be a potential issue.
Customers who are extremely dissatisfied might be more likely to complain, skewing the analysis.
Could recall bias be a factor in the accuracy of customer complaints used to identify the chatbot’s issues?
Yes, recall bias could be a factor.
Customers might misremember or exaggerate aspects of their interaction due to frustration.
Could the Hawthorne effect influence the results of user testing for a new version of the chatbot?
Yes, the Hawthorne effect could influence the results.
Customers might alter their behavior knowing they are participating in a test.
Could the identification of the ‘root cause’ of issues by the supervisor be subject to confirmation bias?
Yes, absolutely.
The supervisor may emphasize data that supports their preconceived ideas.
