Reporting On Your Test Flashcards
You’ve got all this data from your user test. What do you use the test report for?
You use a test report to negotiate design changes and to influence and guide your team towards more user-centered design, towards more usable designs and towards more designs with higher successful task - completion rate.
Adopting the usability attitude
From talking about “Our design” talk about “__________”.
The user’s goals
Adopting the usability attitude
From talking about “features and ideas”, talk about “__________”.
Validating tasks
Adopting the usability attitude
Instead of what the “team wants”, talk about “__________”.
User needs
Adopting the usability attitude
Instead of your “opinion on what you think the design is”, talk about “__________”.
The behavior you observed
What is adopting the usability attitude called?
User Advocacy
What are the two benefits of formal usability testing according to Jupiter Research?
- It’s to educate stakeholders, decision-makers, developers
- To get chance to influence yourself and your team by being present with a user, see what they’re doing, see them succeed, see them fail.
What are good moderating skills?
- Proper screening an recruiting
- Moderation with rapport
- Curtailing non-verbal responses
- Managing user interaction
- Observation without hidden agendas
- Usability Testing is not a clinical thing. When you show the user you are relaxed, they will also relax (and tell you more!)
Imagine you’re doing a think aloud test with a user who keeps forgetting to think aloud. What does Frank Spillers advice that you do in that situation?
Take ownership of the situation and gently remind the user to think aloud. You need the user to think aloud to be able to include him/her in your report, and you don’t want to waste both your and your users time.
What does Frank Spillers mention as the main advantage of having stakeholders observe your user tests?
If your stakeholders observe the test, it’s easier for them to develop empathy for the users.
Avoid User Emotion Traps
Trap:
Wants to know if they are doing a good job
Solution:
“Ok” or “You’re helping us identify what we are looking for”
Avoid User Emotion Traps
Trap:
Wants to get answers right
Solution:
“There are no right or wrong answers” or “You helped us identify what we were looking for”
Avoid User Emotion Traps
Trap:
Stubborn or Quiet
Solution:
Ask users to “Remember to Think Out Loud so I can follow along”
Avoid User Emotion Traps
Trap:
Waiting for permission to move on to next task
Solution:
Tell user to move at her own pace during test. Indicate before test instructions
Avoid User Emotion Traps
Trap:
Mumbles or speaks unclearly
Solution:
Ask user to speak louder
Avoid User Emotion Traps
Trap:
Panicking for help
Solution:
Let them struggle, within reason. Say nothing or just say “Ok”
What are Frank Spillers reporting tips?
- Prioritize findings
- Report actual and observe behavior
- Avoid micro-usability
- Stay focused on business objectives/technology constraints
- Suggest appropriate design recommendations
The more usability testing you do, the more confident you will be at interpreting findings and your analysis.
What are Frank Spillers report writing advice?
- Keep it short (10-15 pages)
- User screen shots to communicate findings
- Include positive findings in you reports.
- Include an executive summary
- Include user quotes - verbatim
- Avoid harsh words or tones
- Distinguish user opinion vs observer opinion
- Keep descriptions relevant
- Include analyst or competitive analysis where relevant
- Try to report out immediately after the test - as an email then followed by a report
Measuring satisfaction
- It helps me be more effective
- It is easy to use
- I easily remember how to use it
What are the Politics of Usability
- Build Alliances
- Avoid confrontations
- Pick you battles
- Position yourself as an ally
- Build trust by being completely open
Why is metric capture vital?
- Make usability recommendations concrete
- Ground teams in reality (probability-thinking)
- Help iterate and validate design concepts
- Provide objectivity to design debates
- Guide fact-based design decisions
What are Franks Spillers communication advice?
- Watch for the feelings of others
- Be aware that some people might take the criticism personally (even, culturally avoided in Asia)
- Be flexible with your recommendations. If there is another way to solve the problem- be open to that.
- If you are not sure, say so.
- Separate your observations from your thoughts
- Be aware of your own bias and agenda (check agendas at the usability lab door)
Tip: Be careful with qualitative date- it is less understood than quantitative. Be sure to provide a balance of statistics (quantitative) and observation (qualitative) since many more people understand numbers over insights.
When moderating, if a user is speaking softly, Frank Spiller advice that you:
Ask them to speak louder
Frank Spiller’s advice to reporting your test results is:
Use metrics, video and quotes from users.
In your test report you should:
Be truthful but be sensitive to stakeholder feelings
