LU 5 EVALUATION

Question 1

What is evaluation?

Answer 1

Evaluation is about understanding if your newly designed interface is usable or not.

Question 2

Key step in evaluation (3)

Answer 2

First, understand why you evaluate? Evaluation can be for feedback or proof.

Second, think about what you want to observe. Decide between observing behavior or attitude.

Third, think about the type of data you want. Determine if you need to understand “why” something happens or “how much” it happens

Question 3

Why do we evaluate? (2)

Answer 3

Gather feedbacks with the goal of improving the product.

Prove that the product is usable or is not, usable

Question 4

What are you evaluating? (2)

Answer 4

Attitude

a person’s feelings or opinions, influenced by culture and experiences.

Behavior

a person’s actions in a situation, observable and measurable.

Question 5

What is the purpose of qualitative metrics?

Answer 5

aim to answer the “why” question about user behavior and preferences.

Question 6

What types of data are used in qualitative metrics?

Answer 6

Qualitative metrics use observations, symbols, and words to understand user experiences and perceptions.

Question 7

How do you approach qualitative metrics?

Answer 7

Qualitative metrics involve observing user interactions and interpreting their meaning within the context of the study.

Question 8

How do you analyze qualitative metrics?

Answer 8

Qualitative metrics are analyzed by grouping common data themes and patterns, focusing on non-statistical analysis methods.

Question 9

What is the purpose of quantitative metrics?

Answer 9

Quantitative metrics aim to answer “how many” or “how much” in user interactions and behaviors.

Question 10

What types of data are used in quantitative metrics?

Answer 10

Quantitative metrics use numerical data and statistical results to measure user activities and engagement.

Question 11

How do you approach quantitative metrics?

Answer 11

Quantitative metrics involve measuring and testing user behaviors objectively to gather numerical data.

Question 12

How do you analyze quantitative metrics?

Answer 12

Quantitative metrics are analyzed using statistical methods to identify trends, correlations, and significant findings in user data.

Question 13

What are inspection techniques in usability evaluation?

Answer 13

methodologies where evaluation is conducted by experts rather than involving participants or users.

Question 14

What are the pros of using inspection techniques?

Answer 14

Cheaper and faster compared to user studies.

Experts bring specialized knowledge and experience.

Question 15

What are the cons of using inspection techniques?

Answer 15

Experts may overlook issues that real users might identify.

There can be a bias towards more common errors, potentially missing less frequent but critical issues.

Different inspection techniques may have varying definitions of usability.

Question 16

What is GOMES in usability evaluation?

Answer 16

GOMES stands for Goals, Operations, Methods, and Selection rules. It’s an inspection methodology where experts break down tasks into physical actions, like pressing keys or moving a mouse, and calculate how long these actions would take an average person.

Question 17

How does GOMES categorize actions?

Answer 17

Actions in GOMES are categorized into general action classes based on the Keystroke-Level Model, which predicts the time it takes to perform various actions.

Question 18

Why is GOMES used in usability evaluation?

Answer 18

GOMES helps evaluate the efficiency of user interactions by analyzing the time required to perform specific tasks, providing insights into usability and potential improvements.

Question 19

What is Heuristic Evaluation?

Answer 19

Heuristic Evaluation involves experts evaluating an interface using a set of established criteria (heuristics), such as Nielsen’s 10 heuristics.

Question 20

What are the pros of Heuristic Evaluation?

Answer 20

Can be conducted by a single person.

-No ethical or recording issues.

-Identifies numerous potentially costly problems at minimal expense.

Question 21

What are the cons of Heuristic Evaluation?

Answer 21

Experts may overlook issues that end users would identify.

Question 22

What is Cognitive Walkthrough?

Answer 22

Cognitive Walkthrough is a method to evaluate if the sequence of cues and prompts in a system supports how people naturally process tasks and anticipate next steps.

Question 23

When to use Cognitive Walkthrough?

Answer 23

Initial evaluation of a system.

Low budget evaluations.

Systems that are walk-up-and-use or first-time use scenarios.

When access to HCI (Human-Computer Interaction) experts is available.

When not to use Cognitive Walkthrough?

Formal evaluation of your own system where you are the evaluator.

Question 24

What is the Cognitive Walkthrough process?

Answer 24

The Cognitive Walkthrough is a method used to evaluate whether the sequence of cues and prompts in a system supports users’ task processing and anticipation of next steps.

Question 25

When should you use Cognitive Walkthrough?

Answer 25

It is ideal for initial evaluations of systems, especially in low-budget scenarios, walk-up-and-use systems, or first-time user interactions, with access to HCI (Human-Computer Interaction) experts.

Question 26

When should you avoid using Cognitive Walkthrough?

Answer 26

Avoid using it for formal evaluations of your own system where you act as the evaluator.

Question 27

What are the key steps in the Cognitive Walkthrough process?

Answer 27

Briefing Session: Introduce experts to tasks and objectives, provide necessary materials like personas and task descriptions.

Evaluation Period (1-2 hours): Experts work individually, taking two passes to familiarize with the product and focus on specific features. They answer predefined questions for each correct step and document issues and good aspects.

Debrief Session: Experts collaborate to prioritize identified problems and discuss findings.

Reporting: Compile a detailed report for the client, outlining identified problems and their severity, along with positive observations.

Question 28

What are the four questions used in Cognitive Walkthrough?

Answer 28

Will users want to produce whatever effect the action has?

Will users see the control (button, menu, label, etc.) for the action?

Once users find the control, will they recognize that it will produce the effect they want?

After the action is taken, will users understand the feedback they receive and confidently proceed to the next action?

Question 29

What is a Usability Aspect Report (UAR)?

Answer 29

A Usability Aspect Report (UAR) is a structured document used to record findings from a usability evaluation, focusing on both positive and negative aspects of an interface.

Question 30

How does a Usability Aspect Report (UAR) differ from a bug report?

Answer 30

While a bug report typically focuses on technical issues and errors, a UAR specifically addresses usability issues identified through evaluations like Cognitive Walkthroughs. It documents observations related to how well the interface supports user tasks and interactions.

Question 31

What does a Usability Aspect Report (UAR) include?

Answer 31

It includes:

Answers to the 4-Question questionnaire used in Cognitive Walkthroughs.

Descriptions of usability issues, categorized by severity.

Positive observations highlighting effective interface elements.

Question 32

Why is a Usability Aspect Report (UAR) important?

Answer 32

It provides a clear record of usability strengths and weaknesses, guiding design improvements and ensuring that user-centered considerations are addressed in interface development.