Sept 26 - Usability Testing

Question 1

Why is it not resourceful to test more than 5 users?

Answer 1

The usability problems diminish after each user, and after the 5th the diminishing returns are too low.

Question 2

Why is it better to do 3 studies with 5 users rather than 1 study with 15 users that discovers all usability test?

Answer 2

It allows you to redesign and test usability fixes. It also allows you to probe deeper into the fundamental structure when the surface-level fluff has been fixed.

Question 3

Why not just test one user with 15 studies?

Answer 3

Because the user could be an outlier. Also starting new studies is costly, might as well run them on multiple users.

Question 4

How many users would you want in a study at minimum?

Answer 4

3

Question 5

What is usability engineering?

Answer 5

A process (or set of tools) for designing systems that match human capabilities.

Question 6

Do you apply usability engineering at the beginning or at the end of design process?

Answer 6

Neither, it’s an iterative process through the design phase.

Question 7

Why conduct Usability Evaluations?

Answer 7

There are many reasons, including:
1. system development is expensive; supporting a bad UI is even more expensive;
2. performers will reject or work around systems that do not meet their needs;
3. design teams need to discover performer problems with the UI to provide remedies;
4. the best designers are not representative of the performers; development teams need to
understand performer perspectives, experiences, and capabilities;
5. Usability Engineering is an inexpensive way to improve systems before they are
implemented, to improve existing systems, and to choose usable systems;
6. business users do not have a lot of time to spend learning how to use a system;
7. businesses cannot train fast enough to keep pace with attrition and changing business
rules;
8. performers prefer usable systems with fewer features to a system with gratuitous and
overwhelming features which they have to work hard at figuring out; and
9. performers often use systems in ways developers did not expect, and if designers find this
out, they can build these features into a new system.

Question 8

When And How Is Usability Evaluation Conducted?

Answer 8

Evaluation can be done at any point in the development or implementation of a system, and in a
variety of settings. There are the four primary rules:
1. keep usability goals in mind when designing the test;
2. make the test as similar to the performer’s work environment as possible (using simulated
or actual work tasks);
3. have performers do the testing (include representatives from all performer groups,
although it is not necessary to have many); and
4. accept results. If goals are not met, then the application is not usable and requires
redesign.

Question 9

Where Is Usability Engineering Performed?

Answer 9

Tests can be performed in a real work setting or in a usability lab

Question 10

What are extrinsic or external performance

support opportunities?

Answer 10

Areas of the UI where additional support may be needed

Question 11

How much increase in product usability is a round of testing estimated to improve?

Answer 11

30%

Question 12

Is it enough to meet functional requirements?

Answer 12

No, the system needs to be usable.

Question 13

What’s the purpose of the Dry Run

Answer 13

The Dry Run is a usability test of the usability test!

Question 14

What are the twelve steps for usability engineering?

Answer 14

The Twelve Steps:

1. Determine Test Site
2. Select Evaluators
3. Document Usability Goals
4. Develop Scenarios
5. Develop Briefing
6. Develop Debriefing
7. Schedule Evaluators
8. Set Up Lab
9. Select Observation Team
10. Conduct Dry Run
11. Conduct Evaluation
12. Document Usability Test Findings