Block 4 Flashcards

1
Q

What is the difference between a usability goal and a user experience goal?

A

Usability tends to focus upon those aspects of an interactive product traditionally related to work, such as productivity and effectiveness. It also tends to focus explicitly on the user interface. The six usability goals discussed in Block 1 Unit 2 are: effectiveness, efficiency, safety, utility, learnability and memorability.
User experience is a relatively new term, whose precise meaning is yet to be agreed. However, it relates to a wider set of concerns than those of usability, including issues such as whether an interactive product is fun to use, aesthetically pleasing, and so on. There are many user experience goals; the ten user experience goals concentrated on in Block 1 are: satisfying, enjoyable, fun, entertaining, helpful, motivating, aesthetically pleasing, supportive of creativity, rewarding and emotionally fulfilling.

[Unit 1, review question 1]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Any kind of evaluation is guided either explicitly or implicitly by an evaluation approach.
What is an evaluation approach? What are the three main evaluation approaches? When might each be used?

A

An evaluation approach is based on a distinct set of values and assumptions as to how evaluation should be conducted, either implicitly or explicitly.
The three main evaluation approaches are: usability testing, field studies and analytical evaluation.
Usability testing is an evaluation approach which uses a variety of methods to assess the usability and the experience of using an interactive product.
Field studies are undertaken to learn about what users do naturally in their own settings, and how products mediate their activity.
Analytical evaluation is used to predict problems that users may have with an interactive product. It is undertaken by experts, and users are generally not involved.

[Unit 1, review question 2]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

List the steps of the DECIDE framework, and for each step identify an activity associated with it.

A
  1. Determine the overall goals that the evaluation addresses. An activity here is to decide the goals for the evaluation, and then restate the goals as statements which can be assessed in the evaluation.
  2. Explore the specific questions to be answered. An activity here would be to turn the goals of evaluation into relevant questions for evaluation.
  3. Choose the approach and methods to answer the questions. An activity here would be to choose an approach and methods that will be of help in finding answers to the evaluation questions.
  4. Identify the practical issues that must be addressed. An activity here would be to select the participants for the evaluation.
  5. Decide how to deal with the ethical issues. An activity here would be to adopt and follow a set of guidelines to ensure the evaluation is done ethically and protects participants’ rights.
  6. Evaluate, analyse, interpret and present the data. An activity here would be to decide how best to present the findings of the evaluation to the development team.

[Unit 1, review question 3]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Define each of the below usability goals:

  • effective to use (effectiveness)
  • efficient to use (efficiency)
  • safe to use (safety)
  • having good utility (utility)
  • easy to learn (learnability)
  • easy to remember how to use (memorability).
A

Effectiveness - refers to how good a product is at doing what it is supposed to do.

Efficiency - refers to the way in which a product supports users in carrying out their tasks.

Safety - involves protecting the user from dangerous conditions and undesirable situations.

Utility - refers to the extent to which the product provides the right kind of functionality, so that users can do what they need or want to do.

Learnability - refers to how easy a system is to learn to use.

Memorability - refers to how easy a system is to remember how to use,
once learned.

[Unit 1, review question 4]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

For each of the usability goals, list the key questions you need to ask when establishing whether the particular goal has been satisfied.

A

Effectiveness - Is the product capable of allowing people to learn, carry out their work efficiently, access the information they need, or buy the goods they want?

Efficiency - Once users have learned how to use a product to carry out their tasks, can they sustain a high level of productivity?

Safety - What is the range of errors that are possible using the product and what measures are there to permit users to recover easily from them?

Utility - Does the product provide an appropriate set of functions that will enable users to carry out all their tasks in the way they want to do them?

Learnability - Is it possible for the user to work out how to use the product by exploring the interface and trying out certain actions? How hard will it be to learn the whole set of functions in this way?

Memorability - What kinds of interface support have been provided to help users remember how to carry out tasks, especially for products and operations they use infrequently?

[Unit 1, review question 5]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Describe the key characteristics of an evaluation approach.

A
  • the role of users
  • who controls the process and the relationship between evaluators and
    users during the evaluation
  • the location of the evaluation
  • when the evaluation is most useful
  • the type of data collected and how it is analysed
  • how the evaluation findings are fed back into the design process
  • the philosophy and theory that underlies the evaluation approach.

[Unit 1, review question 6] [Table 12.1 on page 594 of the Set Book]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Why would you want to observe users early in the interaction design lifecycle?

Why would you want to observe users later on in the interaction design lifecycle?

A

Early in design, observation helps designers to understand users’ needs. Later in design, observation is used to assess whether the prototype of the product meets users’ needs. Observing users is useful at any time during product development.

[Unit 2, review question 1]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Describe three types of observation.

Why and when might each be used for evaluating an interactive product?

A

Three types of observation are: direct observation in the field, direct observation in a controlled environment and indirect observation. Direct observation in the field would be most appropriate to evaluate a high- fidelity prototype or an existing product that could be used in the target context. Direct observation in a controlled environment would be most appropriate to study specific aspects of a product. Indirect observation would be most appropriate to evaluate a high-fidelity prototype or an existing product.

[Unit 2, review question 2]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What does it mean to use the think-aloud technique for gathering data?

A

The think-aloud technique involves the participant saying out loud everything they are thinking and trying to do while using a product.

[Unit 2, review question 3]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is a performance measure? What types of user performance measures are taken during a user test?

A

A performance measure is a quantitative measure related to the use of an interactive product. The following are example performance measures taken during a user test: the time taken to complete a typical task or tasks, the number of errors made while performing a task and the number of steps taken to complete a task.

[Unit 2, review question 4]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Which data-gathering techniques are most commonly used in usability testing?

A

Direct observation, questionnaires, interviews and indirect observation through keystroke logging are the data-gathering techniques most often used in usability testing.

[Unit 2, review question 5]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Why are scripts useful for usability testing?

A

Scripts prepared in advance of a usability evaluation – and used for each participant – ensure that all participants are given the same information and are treated in the same way for the evaluation. This also ensures that it is possible to compare the results from participants. This was discussed in Section 14.2.2 of the Set Book, in the context of the MedlinePlus
case study.

[Unit 2, review question 6]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Why is it important to select a representative sample of users whenever possible?

A

It is important to have a representative sample to ensure that the findings of the user test can be generalised to the rest of the user population. Selecting participants according to clear objectives helps evaluators to avoid unwanted bias. For example, if 90% of the participants testing a product for 9–12 year-olds were 12, it would not be representative of the full age range. The results of the test would be distorted by the large group of users at the top end of the age range.

[Unit 2, review question 7]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Suggest the kind of information that you would expect to include in a ‘welcome’ script for a user testing session.

A

Your welcome script would include: the goal of the study and how the results will be used; an outline of what the participant will be expected to do (but not in detail as this will come later); the data recording that will be used, including video cameras, keystroke logging, etc.; an introduction to the environment including the presence of recording equipment and one-way mirrors; and a pointer to the informed consent form.

[Unit 2, review question 8]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Describe the differences between experiments and usability testing [as discussed in the interview with Ben Shneiderman].

A

In the interview with Ben Shneiderman the differences between experiments and usability test studies are described as follows.

Experiments
- Experiments test a hypothesis that has practical implications and theoretical results.
- The results are of broad importance.
- At least two conditions are tested.
- Statistical tests are applied to the results to verify statistically significant differences.
- The results confirm or refute the hypothesis being tested.
- The experimental procedure is carefully described so that others can
replicate it.

Usability test studies

  • Usability test studies involve only small numbers of users who carry out required tasks.
  • Statistical analysis of the results is less important.
  • The goal is to refine a product as quickly as possible.
  • The outcome is a report to developers which identifies frequent problems and possibly suggests improvements.

[Unit 2, review question 9]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is the main purpose for conducting a field study evaluation?

A

Field studies are useful when you want to discover how products and prototypes will be used within their intended social and physical context of use.

[Unit 2, review question 10]

17
Q

What evaluation methods are used in a field study evaluation?

A

Field study evaluations involve observing users and asking users (see Table 12.2 on page 596 of the Set Book).

[Unit 2, review question 11]

18
Q

What are the three steps in data analysis for evaluation?

A

The three steps in data analysis for evaluation are:

  • collating the data
  • summarising the data
  • reviewing the data.

[Unit 2, review question 13]

19
Q

Briefly define quantitative data and qualitative data

A

Quantitative data is in the form of numbers, or can easily and meaningfully be translated into numbers. Qualitative data is difficult to measure, count or express in numerical terms.

[Unit 2, review question 14]

20
Q

In what way(s) can question design in a questionnaire affect data analysis?

A

Open-ended questions are likely to produce a wide range of answers that will need to be analysed qualitatively. Closed questions which offer respondents a finite set of possible answers can be analysed qualitatively or quantitatively.

[Unit 2, review question 15]

21
Q

Name and define the three different kinds of ‘average’?

A

The three different kinds of ‘average’ are mean, median and mode. Mean refers to the commonly known type of average where the figures are added and then divided by the number of figures you started with. Median is the middle value of the figures when they are ranked. Mode is the most commonly occurring figure in

[Unit 2, review question 16]

22
Q

What is a key advantage of analytical approaches to evaluation?

A

Analytical evaluations are relatively quick to perform and do not require recruiting of users to take part.

[Unit 3, review question 1]

23
Q

What is a key disadvantage of inspections?

A

A key disadvantage of inspections is that the expert who is role-playing a user can only make ‘guesses’ about how a real user will react to a product. Putting yourself in the shoes of another user is difficult.

[Unit 3, review question 2]

24
Q

Several researchers and evaluators have developed their own heuristics to complement Nielsen’s original set (listed on pages 686–7 of the Set Book). Why is this?

A

The original set of Nielsen’s heuristics was developed in the early 1990s. From your reading of Chapter 6 in the Set Book, you’ll have seen that many new and varied interface types have come onto the market since then. Nielsen’s original heuristics are too general for evaluating these new products.

[Unit 3, review question 3]

25
Q

Both user observation and heuristic evaluation are undertaken to find what problems users may have when using an interactive product. Using the following criteria, compare these two techniques.

  • Involvement of users.
  • Who is ‘observed’?
  • Who answers the questions?
  • What are the outcomes?
A

Here is a comparison of the two techniques:

Involvement of users.
In user observations, participants are asked to attempt tasks with the interface and to comment from their own point of view. In a heuristic evaluation, experts inspect the user interface while role-playing as typical users. They assess the user interface against each heuristic individually. The heuristics are meant to guide the expert in finding problems.

Who is ‘observed’?
In user observations, the evaluator observes what the participant does. In a heuristic inspection, inspectors ‘observe’ themselves – noticing where typical users might look, and trying to predict what a typical user might do.

Who answers the questions?
In user observations, participants may be asked to describe their thoughts in a think-aloud or retrospective protocol. In heuristic inspections, the inspectors asks themselves questions such as ‘Is this a simple and natural dialogue?’

What are the outcomes?
In user observations, the key measurement is often effectiveness: ‘Can the user do this task?’ In heuristic inspection, the key measurement is: ‘How many violations of heuristics are there in this user interface?’

[Unit 3, review question 4]

26
Q

What is a key benefit of pluralistic walkthroughs?

A

A key benefit is a strong focus on users’ tasks at a detailed level.

[Unit 3, review question 5]

27
Q

What is GOMS? When is GOMS used for evaluation?

A

GOMS (an acronym standing for goals, operators, methods and selection rules) is a model which describes how a user performs a computer-based task in terms of goals, and the selection of methods and operations from memory that are needed to achieve them.
GOMS is useful in helping make decisions about the effectiveness of new products. It is not often used for evaluation purposes.

[Unit 3, review question 6]

28
Q

What is the keystroke level model? When is it useful for evaluation?

A

The keystroke level model is a model which provides actual numerical predictions of user performance. The model encompasses a standard set of approximate times for the main kinds of operators used during a task, which are used to predict the time it takes to execute a given task.
The keystroke level model is useful for comparing the time it takes to perform a task using different strategies.

[Unit 3, review question 7]

29
Q

What is Fitts’ Law? When is it useful for evaluation?

A

Fitts’ Law predicts the time it takes to reach a target using a pointing device. In interaction design it is used to describe the time it takes to use a mouse and other input devices to click on objects on a screen.
Fitts’ Law is useful for evaluating systems where the time to physically locate an object is critical to the task at hand.

[Unit 3, review question 8]