4.2 Usability Test Methods

Question 1

_______ refers to the ease with which people can use your product or experience.

Answer 1

Usability

Question 2

______ _______ refers to a collection of methods that design researchers use to measure improve usability.

Answer 2

Usability testing

Question 3

Utility + Usability =

Answer 3

Useful

Question 4

______ = whether it provides the features you need.

Answer 4

Utility

Question 5

______ = how easy and pleasant these features are to use.

Answer 5

Usability

Question 6

Useful = _____ + _____.

Answer 6

Usability + Utility

Question 7

Interviews and open card sorts are an example of _______ research methods.

Answer 7

generative

Question 8

What people do:

a. attitudinal
b. behavioral
c. qualitative
d. quantitative

Answer 8

b. behavioral

Question 9

What people think/say

a. behavioral
b. qualitative
c. attitudinal
d. quantitative

Answer 9

c. attitudinal

Question 10

Descriptive data:

a. quantitative
b. behavioral
c. attitudinal
d. qualitative

Answer 10

d. qualitative

Question 11

Numerical data:

a. quantitative
b. behavioral
c. attitudinal
d. qualitative

Answer 11

a. quantitative

Question 12

What is Context of Use:

Answer 12

The third dimension of the graph, referring to the context in which we gather feedback from people.

Question 13

What is gaze tracking?

Answer 13

Eye-tracking: The process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head.

Question 14

A device for measuring eye positions and eye movement and can be configured to measure precisely where participants look as they perform tasks or interact naturally with websites, app, etc. is called?

Answer 14

Eye tracker

Question 15

What are the pros of eye tracking?

Answer 15

Eye tracking can be useful in determining the location of specific elements. The goal is to understand people’s eye gaze patterns so we can guide their eyes in the right direction or strategically place important elements in their natural line of sight.

Question 16

What are the cons of eye tracking?

Answer 16

Eye tracking equipment can be incredibly complex and difficult to calibrate. It’s possible to use remote versions to track mouse movement with the hypothesis that mouse movement closely correlates to eye movement, but it can be difficult to glean accurate insights from a large group.

Question 17

When should you use eye tracking?

Answer 17

Sophisticated organizations with adequate funding and expertise can employ eye-tracking as additional data to inform their design decisions. Heatmaps that aggregate eye movement or mouse movement can provide a useful snapshot of activity to help inform design.

Question 18

What is Ethnographic field studies?

Answer 18

Observing participants in their natural environment.

Question 19

What is the difference between Ethnographic field studies and Guerrilla testing?

Answer 19

Guerrilla testing tends to focus on the functionality of the system in question, while ethnographic field studies tend to focus on the test participant’s personal attitudes, behaviors, and environments.

Question 20

What are the pros of Ethnographic field studies?

Answer 20

These studies can yield incredibly rich information about not only behavior and attitudes with a product or service, but a participant’s general habits and environment, as well. Given the time and funding, it can yield valuable information about your customers that you’d miss out on by focusing simply on the product and its usability.

Question 21

What are the cons of Ethnographic field studies?

Answer 21

Tests can be expensive from both a time and a financial standpoint.

Question 22

When should you use Ethnographic field studies?

Answer 22

Given sufficient time and funding, these studies can be a great step towards designing more accurately for highly specific contexts and use cases.

Question 23

What is A/B Testing?

Answer 23

aka “live testing,” “bucket testing,” or “split testing,” is a method of scientifically testing different designs on a site by randomly assigning groups of users to interact with each of the different designs and measuring the effect of these assignments on user behavior.

Question 24

You could test, for example, multiple versions of a sign-up process for your site, multiple email layouts for your newsletter, or the success of text versus video in an online tutorial. When more than two choices are necessary, researchers might also use a similar process is called?

Answer 24

Multivariate testing

Question 25

What are the pros of A/B testing?

Answer 25

Quantitative results provide definitive direction so long as the test setup and results analysis are sound.

Question 26

What are the cons of A/B testing?

Answer 26

There must be significant traffic and usage of your product or service in order to garner useful data, and it can take a long time to gather enough data to reach “statistically powered” results. This means that you need a larger number of participants to be able to generate plausible test results. Expertise in data science is also required to make an accurate decision as to statistical significance and power.

Question 27

When should you use A/B testing?

Answer 27

For products or services with significant traffic and usage that also have teams with expertise in setting up A/B Tests and downloading results.

Question 28

What is a 5-second test?

Answer 28

aka memory test, 5-second tests are used to determine whether a particular webpage or app screen does a good job of getting its core message across.

For example, if you have a webpage of which the main purpose is to have visitors sign up for a newsletter, you could use a 5-second test to determine whether the users recognized that this page was asking them to sign up for a newsletter.

Question 29

What are the pros of a 5-second test?

Answer 29

A 5-second test is quick and easy to run. You just need a representation of the webpage or app screen you’d like to test—which can be anything from a paper prototype to a high-fidelity mockup—and a set of questions you’d like to ask users about what they remember from looking at the screen. Users are then shown the screen for five seconds and asked the prepared set of questions before the screen is covered or switched to a different one.

Question 30

What are the cons of a 5-second test?

Answer 30

5-second tests aren’t well suited for screens that have multiple functions as they will be difficult for users to assess in such a short space of time, and different users are likely to focus on different function’s features, leading to inconclusive results.

Question 31

When should you use a 5-second test?

Answer 31

5-second tests are best suited to testing screens that have a single primary purpose such as signing up for a service or downloading a piece of software.

Question 32

What is Usability testing?

Answer 32

A technique used by UX designers, design researchers, and interaction designers to evaluate the utility and usability of a prototype or live product.

Question 33

Why should you conduct a usability test?

Answer 33

An effective usability test reveals how a typical user would attempt to complete a task using your product. It can also grant insights into how to fix problems that may arise during the test.

Question 34

What are field studies?

Answer 34

Observing participants in their natural environment- where they’d be most likely to engage with your product or service.

Question 35

What are a few factors that can influence the cost of your research?

Answer 35

Usability test method, number of participants, time allocated.

Question 36

What do moderated in-person tests require?

Answer 36

A researcher or facilitator co-located with the participant (often in a lab, though locations can vary)

Question 37

When are moderated remote tests useful?

Answer 37

When the participant and facilitator are in different locations.

Question 38

What are unmoderated remote tests?

Answer 38

Conducted asynchronously online using a variety of specialized tools such as UserTesting, UserZoom, or UsabilityHub