Reporting On Your Test

Question 1

You’ve got all this data from your user test. What do you use the test report for?

Answer 1

You use a test report to negotiate design changes and to influence and guide your team towards more user-centered design, towards more usable designs and towards more designs with higher successful task - completion rate.

Question 2

Adopting the usability attitude

From talking about “Our design” talk about “__________”.

Answer 2

The user’s goals

Question 3

Adopting the usability attitude

From talking about “features and ideas”, talk about “__________”.

Answer 3

Validating tasks

Question 4

Adopting the usability attitude

Instead of what the “team wants”, talk about “__________”.

Answer 4

User needs

Question 5

Adopting the usability attitude

Instead of your “opinion on what you think the design is”, talk about “__________”.

Answer 5

The behavior you observed

Question 6

What is adopting the usability attitude called?

Answer 6

User Advocacy

Question 7

What are the two benefits of formal usability testing according to Jupiter Research?

Answer 7

1. It’s to educate stakeholders, decision-makers, developers
2. To get chance to influence yourself and your team by being present with a user, see what they’re doing, see them succeed, see them fail.

Question 8

What are good moderating skills?

Answer 8

• Proper screening an recruiting
• Moderation with rapport
• Curtailing non-verbal responses
• Managing user interaction
• Observation without hidden agendas
• Usability Testing is not a clinical thing. When you show the user you are relaxed, they will also relax (and tell you more!)

Question 9

Imagine you’re doing a think aloud test with a user who keeps forgetting to think aloud. What does Frank Spillers advice that you do in that situation?

Answer 9

Take ownership of the situation and gently remind the user to think aloud. You need the user to think aloud to be able to include him/her in your report, and you don’t want to waste both your and your users time.

Question 10

What does Frank Spillers mention as the main advantage of having stakeholders observe your user tests?

Answer 10

If your stakeholders observe the test, it’s easier for them to develop empathy for the users.

Question 11

Avoid User Emotion Traps

Trap:
Wants to know if they are doing a good job

Solution:

Answer 11

“Ok” or “You’re helping us identify what we are looking for”

Question 12

Avoid User Emotion Traps

Trap:
Wants to get answers right

Solution:

Answer 12

“There are no right or wrong answers” or “You helped us identify what we were looking for”

Question 13

Avoid User Emotion Traps

Trap:
Stubborn or Quiet

Solution:

Answer 13

Ask users to “Remember to Think Out Loud so I can follow along”

Question 14

Avoid User Emotion Traps

Trap:
Waiting for permission to move on to next task

Solution:

Answer 14

Tell user to move at her own pace during test. Indicate before test instructions

Question 15

Avoid User Emotion Traps

Trap:
Mumbles or speaks unclearly

Solution:

Answer 15

Ask user to speak louder

Question 16

Avoid User Emotion Traps

Trap:
Panicking for help

Solution:

Answer 16

Let them struggle, within reason. Say nothing or just say “Ok”

Question 17

What are Frank Spillers reporting tips?

Answer 17

• Prioritize findings
• Report actual and observe behavior
• Avoid micro-usability
• Stay focused on business objectives/technology constraints
• Suggest appropriate design recommendations

The more usability testing you do, the more confident you will be at interpreting findings and your analysis.

Question 18

What are Frank Spillers report writing advice?

Answer 18

• Keep it short (10-15 pages)
• User screen shots to communicate findings
• Include positive findings in you reports.
• Include an executive summary
• Include user quotes - verbatim
• Avoid harsh words or tones
• Distinguish user opinion vs observer opinion
• Keep descriptions relevant
• Include analyst or competitive analysis where relevant
• Try to report out immediately after the test - as an email then followed by a report

Question 19

Measuring satisfaction

Answer 19

1. It helps me be more effective
2. It is easy to use
3. I easily remember how to use it

Question 20

What are the Politics of Usability

Answer 20

1. Build Alliances
2. Avoid confrontations
3. Pick you battles
4. Position yourself as an ally
5. Build trust by being completely open

Question 21

Why is metric capture vital?

Answer 21

1. Make usability recommendations concrete
2. Ground teams in reality (probability-thinking)
3. Help iterate and validate design concepts
4. Provide objectivity to design debates
5. Guide fact-based design decisions

Question 22

What are Franks Spillers communication advice?

Answer 22

• Watch for the feelings of others
• Be aware that some people might take the criticism personally (even, culturally avoided in Asia)
• Be flexible with your recommendations. If there is another way to solve the problem- be open to that.
• If you are not sure, say so.
• Separate your observations from your thoughts
• Be aware of your own bias and agenda (check agendas at the usability lab door)

Tip: Be careful with qualitative date- it is less understood than quantitative. Be sure to provide a balance of statistics (quantitative) and observation (qualitative) since many more people understand numbers over insights.

Question 23

When moderating, if a user is speaking softly, Frank Spiller advice that you:

Answer 23

Ask them to speak louder

Question 24

Frank Spiller’s advice to reporting your test results is:

Answer 24

Use metrics, video and quotes from users.

Question 25

In your test report you should:

Answer 25

Be truthful but be sensitive to stakeholder feelings