User Research Flashcards
What is qualitative research?
Research methods that produces messy data. It is:
- Not measurable
- Unstructured
- Subjective
- Smaller data samples
- Interpretive
- Soft science
- Produces deep insights
- Involves the interpreter’s opinion
What are examples of qualitative research methods?
- Usability tests
- Focus groups
- Field studies
- Interviews
- Diary/camera study
- Eye tracking
- Open text survey questions
What is quantitative research?
Research methods that produce statistical data. It is:
- Expressed by stats, numbers, graphs
- Measured
- Structured, numerical
- Broad insight
- Objective
- Hard science
- Larger samples sizes
- Produces high level insights
What are examples of quantitative research methods?
- Analytics
- A/B testing
- Card sorting
- Message board mining
- Customer feedback calls
- Multiple choice surveys
What is observational research?
Research methods that watch what users do, not necessarily even talking to them.
What is attitudinal research?
Research methods that listen to what people say.
What are examples of observation research?
- Eye tracking
- Usability tests
- Contextual inquiries
- Field studies
- Analytics (Google)
- A/B testing
What are examples of attitudinal research?
- Card sorting
- Message board mining
- Customer feedback calls
- Surveys
- Interviews
- Diary/camera study
- Focus groups
What makes up the 4 quadrants of the Landscape of Research?
Observational, Quantitative, Attitudinal, Qualitative
What are examples of shallow research?
- Yes/No questions
- Asking for opinions
- Asking loaded questions
- Asking assuming questions
- Asking what people will do or have done
What are 4 bias types?
- Self-referential design
- Knowing the answer
- Being defensive/precious
- Asking leading questions
Who is not part of the target audience?
- Me
- People working in the company (colleagues, clients)
We know too much about the business, the business model, the decisions that were made, and the politics behind the decisions, ect. We have too much bias to be the target audience.
What is usability testing?
Usability testing is the most powerful, most prevalent tool in UX. The premise is to observe how a user uses the product.
What do you need to conduct a usability test?
- Computer
- Moderator & user
- User testing software (record screen/environment)
What are benefits of user testing?
- Show a users experience video
- Presents product from the user’s perspective
- Unites stakeholders
- Challenges/validates assumptions
- Builds consensus instead of debate
- Provides variety of user data
- Cost effective/High ROI
What do you learn in user testing?
- What the user is trying to do (goals)
- What users do (behaviors)
- The context of use
- How the product facilities common behaviors
- Identifies pain points/road blocks to remove
- How product compares to competitors
- Is the product valuable / does it solve the problem?
How do you define your user test objectives/research goals?
- Define clear goals to narrow focus
- Be specific as possible, don’t try to cover everything
- Don’t treat it as a one-off event
- Don’t confuse it with functional testing (how it works)
How often should you schedule usability tests?
Every 4-6 weeks, or at least once a quarter
What does a moderator do in usability tests?
- Hold a conversation
- Help users relax
- Pay close attention
- Ask good questions
- Keep things on track
What is a test script?
- Aide memoir (memory bank)
- Specifies tasks to be completed
- Within the tasks, specifies questions to ask along the way
- Gives the test structure
What are the 3 parts of a test script?
- Introduction
- Interview
- Tasks
What is the goal of the introduction of a test script?
Make users relax by:
- Welcoming/thanking
- Explaining the purpose of the session
- Explaining the room/technical setup
What are the 4 things we want users to keep in mind, stated in the introduction of the test script?
- We are not testing them, we are testing the product. No right or wrong questions, we’re not trying to trick them.
- Speak their mind, be as candid and open as possible. Won’t hurt our feelings/cause offense.
- Think out load, verbalize thoughts: what they are trying to do, what they are looking for, what they expect to see after clicking on something, why they are clicking on something.
- Let them know they are free to ask questions whenever they want, but we may not always answer.
What are 4 parts of the interview section of a test script?
- Ask easy questions
- Describe the last time they….
- Understand goals
- Understand background context
What are parts of the task section of a test script?
- Start with a natural task, if appropriate
- Then pre-defined tasks
- Ask lots of “what” and “why” questions
- Clearly explain the scenario
- Clearly state the task
What goes into a screener when using a professional recruiter to find users for a usability test?
- Demographics
- Level of tech savvy-ness
- How many people, when and where
- Product history
Who should not be used as a user in a usability test?
- Colleagues/client
- Friends & family
How do you set up a usability test on desktop?
Have 3 people in the room:
User in front of the laptop, moderator next to the user, note taker viewing proceedings on a monitor
How do you set up a usability test on mobile?
Have 3 people in the room:
User in front of the phone and dimmed laptop, moderator next to the user, note taker viewing proceedings on a monitor
What are some tips for moderating a usability test?
- Ask right questions
- Know the software
- Never take the mouse
- Read verbatim from the script
- Rehearse with colleagues
- Never tell users what to do
- Practice
What questions should a moderator avoid in a usability test, and what should be used instead?
- Yes/no (instead try what/why)
- leading/loaded (instead try what/why)
- hypothetical/future (past specific)
- design (task questions)
What should you test on existing products?
- Current version
- Competitor software
- Prototype of new design
- Peers in other markets
What should you test on new products?
- Competitor software
- Prototype
- Peers in other markets
When should you conduct usability tests?
During the research, prototype and build phases to understand the problem and validate the solution.
What are some benefits of conducting online surveys?
- Easiest way to gather data
- Weight of numbers
- Cheap
- Quantitive and qualitative data
What are the 3 golden questions to ask on an online survey?
- Why did you visitor our __ today?
- Were you able to complete your task?
- What would you change?
What are the 2 objectives of a depth interview?
To understand the goals and context of use.
What shouldn’t you ask in a depth interview?
- Don’t ask about the product/idea.
- Don’t ask what they would do in the future.
- Don’t ask about page design.
Who should you talk to in a stakeholder interview?
- Anyone with requirements
- Anyone with stake
- Anyone with sign off
May include:
- Sponsoring executive
- Product team
- Technology team
- Customer support
- Marketing team
- Sales team
- Business analysts
What are the 4 objectives of a stakeholder interview?
- Understand the business
- Understand business goals
- Understand problems with the product
- Understand competitive landscape
What should you avoid talking about in a stakeholder interview?
- Talking about solutions
- Talking about features
- Deflect and focus on goals and problems
What are 4 things a stakeholder brief should include?
- 2-3 page summary of what you heard
- Focus on agreements everybody made
- Any contradictions for discussion
- Sign off after contradictions resolved
What are some benefits of a stakeholder brief?
- Gives project clarity
- Reference for future debates
- Shows you are listening
What is card sorting?
Technique to help organize large volumes of content or features of the product. The organization should correspond with the customer’s flow/mental model.
What is involved in an open card sort?
Post-it notes on a wall with no more than 20 users. The main task is to organize the cards in groups.
- Give a group name
- Must have more than 1 group
- Max of 10 groups
- No group names called “Misc, stuff, other”
What is the goal of an open card sort?
Understand the mental model and language/vocabulary of the customer.
What is involved in a closed card sort?
Start with defined headers and ask customers to populate the headers with the post-it notes.
What is the goal of the closed card sort?
To validate the open sort results.
What is A/B testing?
It’s a way to test multiple variables (multi-varient test)
What are 4 benefits of A/B testing?
- Facilitate improvement
- Encourage experiments
- Validate assumptions
- Facilitate data driven design
What is competitive benchmarking?
Looking at how other organizations are trying to solve the same problem.
What 4 questions are we trying to answer from competitive benchmarking.
- How do best in class websites/apps solve the problem?
- What are they doing well that we can emulate?
- What are they doing not so well that we can improve?
- What conventions have been established that we need to follow?
