10 - User Research and Evaluation Flashcards
Why is it important to test our product with its users?
- Product must appeal to its users
- Allows you to find problems before selling, minimizing the amount customers lost
In what circumstances can a product be evaluated in?
- Natural settings (In the wild)
- Controlled setting (In the lab)
- Remotely
When should an evaluation be conducted?
In general:
- Before starting
- After prototypes
- After every iteration of the design cycle
- After adding new features
Final decisions have to be based on the data collected
What information is relevant about a user?
- Their goals
- Their behavior
- Their reaction to technology
Who makes up the user population?
People with different goals, capabilities, needs and wants
Important: The designer is not the user
What is a sample?
A subgroup of your users
What are four sampling methods?
- Convenience sampling
- Snowball sampling
- Random sampling
- Stratified sampling
What is convenience sampling?
Selecting a sample based on the people you know/have access to
What is snowball sampling?
Selecting a sample based on people you know and the people they know…
What is random sampling?
Selecting a random subset of the population
What is stratified sampling?
Selecting a sample in proportion to the users (e.g. If 40% of the user population is female, 40% of the sample is female)
What are key elements of proper research conduct?
- Consent forms
- Anonymized data
- Proper data storage
- Research ethics
What are three common user research methods?
- Interviews
- Questionnaire
- Observations
Explain the evaluation flow.
Define the task > Define performance measures > Develop the experiment > Recruit participants > Do the experiment (with corresponding conditions) > Analyze the data > Summarize the data
What metrics do you want to observe when doing usability testing?
- Number of users completing a task successfully
- Time to complete a task
- Time to complete a task after a specified time away from the product
- Number and type of errors per task
- Number of errors per unit of time
- Number of navigations to online help or manuals
- Number of users making a particular error
