Block 4 Flashcards
What is the difference between a usability goal and a user experience goal?
Usability tends to focus upon those aspects of an interactive product traditionally related to work, such as productivity and effectiveness. It also tends to focus explicitly on the user interface. The six usability goals discussed in Block 1 Unit 2 are: effectiveness, efficiency, safety, utility, learnability and memorability.
User experience is a relatively new term, whose precise meaning is yet to be agreed. However, it relates to a wider set of concerns than those of usability, including issues such as whether an interactive product is fun to use, aesthetically pleasing, and so on. There are many user experience goals; the ten user experience goals concentrated on in Block 1 are: satisfying, enjoyable, fun, entertaining, helpful, motivating, aesthetically pleasing, supportive of creativity, rewarding and emotionally fulfilling.
[Unit 1, review question 1]
Any kind of evaluation is guided either explicitly or implicitly by an evaluation approach.
What is an evaluation approach? What are the three main evaluation approaches? When might each be used?
An evaluation approach is based on a distinct set of values and assumptions as to how evaluation should be conducted, either implicitly or explicitly.
The three main evaluation approaches are: usability testing, field studies and analytical evaluation.
Usability testing is an evaluation approach which uses a variety of methods to assess the usability and the experience of using an interactive product.
Field studies are undertaken to learn about what users do naturally in their own settings, and how products mediate their activity.
Analytical evaluation is used to predict problems that users may have with an interactive product. It is undertaken by experts, and users are generally not involved.
[Unit 1, review question 2]
List the steps of the DECIDE framework, and for each step identify an activity associated with it.
- Determine the overall goals that the evaluation addresses. An activity here is to decide the goals for the evaluation, and then restate the goals as statements which can be assessed in the evaluation.
- Explore the specific questions to be answered. An activity here would be to turn the goals of evaluation into relevant questions for evaluation.
- Choose the approach and methods to answer the questions. An activity here would be to choose an approach and methods that will be of help in finding answers to the evaluation questions.
- Identify the practical issues that must be addressed. An activity here would be to select the participants for the evaluation.
- Decide how to deal with the ethical issues. An activity here would be to adopt and follow a set of guidelines to ensure the evaluation is done ethically and protects participants’ rights.
- Evaluate, analyse, interpret and present the data. An activity here would be to decide how best to present the findings of the evaluation to the development team.
[Unit 1, review question 3]
Define each of the below usability goals:
- effective to use (effectiveness)
- efficient to use (efficiency)
- safe to use (safety)
- having good utility (utility)
- easy to learn (learnability)
- easy to remember how to use (memorability).
Effectiveness - refers to how good a product is at doing what it is supposed to do.
Efficiency - refers to the way in which a product supports users in carrying out their tasks.
Safety - involves protecting the user from dangerous conditions and undesirable situations.
Utility - refers to the extent to which the product provides the right kind of functionality, so that users can do what they need or want to do.
Learnability - refers to how easy a system is to learn to use.
Memorability - refers to how easy a system is to remember how to use,
once learned.
[Unit 1, review question 4]
For each of the usability goals, list the key questions you need to ask when establishing whether the particular goal has been satisfied.
Effectiveness - Is the product capable of allowing people to learn, carry out their work efficiently, access the information they need, or buy the goods they want?
Efficiency - Once users have learned how to use a product to carry out their tasks, can they sustain a high level of productivity?
Safety - What is the range of errors that are possible using the product and what measures are there to permit users to recover easily from them?
Utility - Does the product provide an appropriate set of functions that will enable users to carry out all their tasks in the way they want to do them?
Learnability - Is it possible for the user to work out how to use the product by exploring the interface and trying out certain actions? How hard will it be to learn the whole set of functions in this way?
Memorability - What kinds of interface support have been provided to help users remember how to carry out tasks, especially for products and operations they use infrequently?
[Unit 1, review question 5]
Describe the key characteristics of an evaluation approach.
- the role of users
- who controls the process and the relationship between evaluators and
users during the evaluation - the location of the evaluation
- when the evaluation is most useful
- the type of data collected and how it is analysed
- how the evaluation findings are fed back into the design process
- the philosophy and theory that underlies the evaluation approach.
[Unit 1, review question 6] [Table 12.1 on page 594 of the Set Book]
Why would you want to observe users early in the interaction design lifecycle?
Why would you want to observe users later on in the interaction design lifecycle?
Early in design, observation helps designers to understand users’ needs. Later in design, observation is used to assess whether the prototype of the product meets users’ needs. Observing users is useful at any time during product development.
[Unit 2, review question 1]
Describe three types of observation.
Why and when might each be used for evaluating an interactive product?
Three types of observation are: direct observation in the field, direct observation in a controlled environment and indirect observation. Direct observation in the field would be most appropriate to evaluate a high- fidelity prototype or an existing product that could be used in the target context. Direct observation in a controlled environment would be most appropriate to study specific aspects of a product. Indirect observation would be most appropriate to evaluate a high-fidelity prototype or an existing product.
[Unit 2, review question 2]
What does it mean to use the think-aloud technique for gathering data?
The think-aloud technique involves the participant saying out loud everything they are thinking and trying to do while using a product.
[Unit 2, review question 3]
What is a performance measure? What types of user performance measures are taken during a user test?
A performance measure is a quantitative measure related to the use of an interactive product. The following are example performance measures taken during a user test: the time taken to complete a typical task or tasks, the number of errors made while performing a task and the number of steps taken to complete a task.
[Unit 2, review question 4]
Which data-gathering techniques are most commonly used in usability testing?
Direct observation, questionnaires, interviews and indirect observation through keystroke logging are the data-gathering techniques most often used in usability testing.
[Unit 2, review question 5]
Why are scripts useful for usability testing?
Scripts prepared in advance of a usability evaluation – and used for each participant – ensure that all participants are given the same information and are treated in the same way for the evaluation. This also ensures that it is possible to compare the results from participants. This was discussed in Section 14.2.2 of the Set Book, in the context of the MedlinePlus
case study.
[Unit 2, review question 6]
Why is it important to select a representative sample of users whenever possible?
It is important to have a representative sample to ensure that the findings of the user test can be generalised to the rest of the user population. Selecting participants according to clear objectives helps evaluators to avoid unwanted bias. For example, if 90% of the participants testing a product for 9–12 year-olds were 12, it would not be representative of the full age range. The results of the test would be distorted by the large group of users at the top end of the age range.
[Unit 2, review question 7]
Suggest the kind of information that you would expect to include in a ‘welcome’ script for a user testing session.
Your welcome script would include: the goal of the study and how the results will be used; an outline of what the participant will be expected to do (but not in detail as this will come later); the data recording that will be used, including video cameras, keystroke logging, etc.; an introduction to the environment including the presence of recording equipment and one-way mirrors; and a pointer to the informed consent form.
[Unit 2, review question 8]
Describe the differences between experiments and usability testing [as discussed in the interview with Ben Shneiderman].
In the interview with Ben Shneiderman the differences between experiments and usability test studies are described as follows.
Experiments
- Experiments test a hypothesis that has practical implications and theoretical results.
- The results are of broad importance.
- At least two conditions are tested.
- Statistical tests are applied to the results to verify statistically significant differences.
- The results confirm or refute the hypothesis being tested.
- The experimental procedure is carefully described so that others can
replicate it.
Usability test studies
- Usability test studies involve only small numbers of users who carry out required tasks.
- Statistical analysis of the results is less important.
- The goal is to refine a product as quickly as possible.
- The outcome is a report to developers which identifies frequent problems and possibly suggests improvements.
[Unit 2, review question 9]
