Unit 5

Question 1

What are summative evaluations?

Answer 1

These are evaluations carried out for the final assessment of a user interface, which no longer has any influence on the design, this is referred to as a summative evaluation. This can be used to compare versions.

Question 2

What is the GOMS Model?

Answer 2

The GOMS model is used to calculate the amount of time it takes experienced users to perform certain tasks on a user interface.

Question 3

What is the basic structure of a usability test?

Answer 3

1. Identify the target audience.
2. Recruit test users.
3. Determine the task for the test.
4. Conduct evaluation.
5. Report results.

Question 4

What is the purpose of an executive summary?

Answer 4

The main part of the report briefly describes the subject of the tests and the methodological procedure.

Question 5

What are the methodological approaches used in usability testing?

Answer 5

Two methodological approaches in usability testing are the experiment and the cooperative evaluation.

Question 6

Why are experiments used?

Answer 6

Experiments are used to determine the influence of an isolated variable on the system using a controlled setup

Question 7

Why are experiments less suitable for design and development projects?

Answer 7

Experiments must meet high stringency and validity requirements to credibly support a generalizable hypothesis. This also means that more test participants are needed to reach a critical number of cases for statistical analysis of the results. Experiments are, therefore, less suitable for design and development projects

Question 8

What is the benefit of cooperative evaluation?

Answer 8

The method aims to generate constructive feedback for redesigning a system at an early stage of development. Thus, the method aims to quickly identify and solve concrete problems of a system

Question 9

What are usability metrics?

Answer 9

Quantitatively collected data in usability tests are also referred to as usability metrics. These can be measurements, e.g., data recording the operating speed and error rate, quantitative results of questionnaires, or body-related measurement data, such as skin conductance or eye tracking, from which emotional reactions and information about the visual perception of a system can be derived.

Question 10

What are the three central metrics for good usability?

Answer 10

Effectiveness, efficiency, and satisfaction are frequently named as the three central metrics for good usability

Question 11

How does DIN EN ISO 9241-11 define usability?

Answer 11

DIN Deutsches Institut für Normung e. V. (2018, p. 11) defines usablity as the following:

• effectiveness: accuracy and completeness with which users achieve specific goals
• efficiency: resources used in relation to the results achieved
• satisfaction: extent to which the userʼs physical, cognitive, and emotional responses resulting from the use of a system, product, or service match user requirements and user expectations

Question 12

What is eye tracking (in the UX context)?

Answer 12

Eye tracking is a technical procedure that records where the test users direct their gaze on a screen and how long they remain there.

Question 13

What is an expert evaluation?

Answer 13

An expert evaluation is a systematic inspection of a system based on a checklist or a recognized heuristic.

Question 14

How is an expert evaluaton conducted?

Answer 14

As the name suggests, this type of evaluation does not invite potential users to take part in tests; instead, two or three experts evaluate the system on the basis of the specified criteria.

Question 15

What procedures can be distinguished by expert evaluations?

Answer 15

Three procedures can be distinguished in expert evaluations: heuristic evaluations, checklist-based evaluations, and cognitive walkthroughs.

Question 16

What is a heuristic evaluation?

Answer 16

1. In a heuristic evaluation, the system is checked using a specific heuristic (set of guidelines).
2. A heuristic evaluation is a method for identifying design problems in a user interface

Question 17

What is a cognitive walkthrough?

Answer 17

In a cognitive walkthrough, a series of test tasks for a system is run like a usability test. In the process, problems and conspicuous features are noted that can lead to the selected user group having difficulties with the system.

Question 18

What is a pluralistic walkthrough?

Answer 18

A variation of the cognitive walkthrough is the pluralistic walkthrough, in which a number of different evaluators – potential users, developers, and usability experts – assess the system independently of one another and then discuss and combine their findings.

Question 19

List observation types and their descriptions

Answer 19

*Participant and non-participant observations: In a participant observation, the observer becomes part of the group being observed. In a non-participant observation, they are limited to the role of observer.
* Field and laboratory observations: An observation in a natural environment is called a field observation. An observation in a controlled environment is called a laboratory observation.
* Open and covert observations: In an open observation, the target person knows about the observation, whereas in a covert observation, they do not. For reasons of research ethics, covert observations are only used when the research question cannot be addressed in any other way.
* Self-observation and external observation: In a self-observation or introspection, there is no need for an external observer who records what is happening in an external observation.
* Direct and indirect observations: Direct observations are focused on an event as it happens. Indirect observations are directed at the results or traces of a particular behavior.
* Technically mediated or unmediated observations: Technically mediated observations use technical aids, such as video cameras or sound recorders, while unmediated observations do without.

Question 20

What is the Experience Sampling Method (ESM)?

Answer 20

The experience sampling method (ESM) refers to regular self-reports that participants in an ESM study provide on demand throughout the day and over longer periods of time. The method uses mobile communication devices to prompt study participants to provide self-reports at regular intervals and, in many cases, to store the self-reports themselves.

Question 21

What are the standarized types of questionnaires?

Answer 21

Unipolar and bipolar scales often use two opposite poles between which test takers select a gradation. If there is no suitable opposite term, a unipolar scale can also be used.

Numeric, verbal, and symbolic scales use numbers for categorization; verbal marks offer terms for categorization that may refer to frequency, intensity, probability, or rating. Symbolic marks are grasped more quickly and easily than verbal or numeric marks (e.g., stylized facial expressions).

Question 22

What are Likert Scales?

Answer 22

Likert scales determine agreement or disagreement with a particular statement. They represent a spectrum of statements ranging from “agree completely” to “disagree completely” and can have different numbers of gradations, but usually five to seven.

Question 23

Explain Semantic Differentials

Answer 23

Semantic differentials are a scaling instrument for measuring the connotative meaning or affective qualities of any object or term

Question 24

What is the AttraktDiff questionnaire?

Answer 24

The AttrakDiff questionnaire was developed to describe the increased interest in the qualities of a system that do not map a focus on efficiency, effectiveness, and satisfaction.

Question 25

What is the VisAWI questionnaire?

Answer 25

The VisAWI questionnaire is designed to quantitatively evaluate the visual aesthetics of graphical user interfaces, especially websites.

It is based on the assumption that high visual quality also positively influences the user-friendliness of a system, not only in terms of user satisfaction but also in terms of the performance of the system.

Question 26

Explain the concept of laddering in UX interview techniques

Answer 26

The laddering technique is an approach developed in market-oriented motive research to explore the motivations of consumers.

The aim of a laddering interview is to construct a relationship between the intervieweesʼ expressed goals and the concrete means to achieve them.

Question 27

Unit 5 Summary

Answer 27

The evaluation of user experience is a challenge in that, on the one hand, the term is used very broadly, and, on the other hand, subjective statements and perceptions play an important role in assessing user experience. Within the framework of usability tests, problems and misunderstandings that affect usability and user satisfaction can be discovered during the development of a system with the help of established and well-documented procedures. Expert evaluations are a cost-effective way to address obvious problems in advance while preparing empirical user tests. In a classic usability test, test takers are observed using a system and asked to think aloud.

In addition, there are a number of observation and survey methods that also focus more strongly on aspects of the quality of experience, such as the hedonic qualities, or which evaluate the effects of a system on emotional experience. In the field of user experience, methods of self-observation in the form of diary studies, in which test participants are asked to provide self-reports at regular intervals, are particularly applicable. Such studies are suitable for tracking changes in the use of a system. A comparable approach is ESM (Experience Sampling Method), in which study participants are asked via mobile communication devices to provide feedback on their use of a system at selected points in time.

In survey techniques, standardized questionnaires on various aspects of emotional experience play a major role. Well-known questionnaires in the field of UX evaluation are AttrakDiff and VisAWI, which survey the pragmatic and hedonic qualities of a system and its visual attractiveness. UX-oriented interview techniques reduce the influence of the interview situation. Laddering represents an interview technique that aims to correlate concrete product features and abstract wants and needs.

Question 28

What is Usability Testing in UX Design?

Answer 28

Usability Testing is a technique used in UX Design to evaluate a product or service by testing it with representative users. It involves observing users as they attempt to complete tasks and can be conducted at various stages of the design process. The primary goal is to identify any usability problems, gather qualitative and quantitative data, and determine the participant’s satisfaction with the product.

Question 29

What are the main types of Usability Testing?

Answer 29

The main types of Usability Testing are:

*Moderated Usability Testing: Conducted in person or remotely, where a moderator guides the test.

*Unmoderated Usability Testing: Conducted without a moderator, often remotely, using automated tools.

*Formative Usability Testing: Conducted during the early stages of product development to shape the design.

*Summative Usability Testing: Conducted at the end of the development process to assess the quality of the final product.

*Comparative Usability Testing: Compares two or more products or designs to see which performs better.

Question 30

What are key components to consider when designing a Usability Test?

Answer 30

Key components to consider include:

*Objective: Clearly defining what you want to test and learn.

*Participant Selection: Recruiting users who represent your target audience.

*Test Environment: Setting up an environment that simulates real-world use.

*Task Selection: Designing tasks that are representative of typical user activities.

*Data Collection Methods: Deciding on qualitative and quantitative methods like surveys, interviews, and analytics.

*Test Moderation: Choosing between moderated and unmoderated tests.

*Analysis Plan: Planning how to analyze and report the findings.

Question 31

What are common methods for recording and analyzing data in Usability Testing?

Answer 31

Common methods include:

*Screen Recording: Capturing the user’s screen as they perform tasks.

*Eye Tracking: Tracking where the user’s gaze falls on the screen.

*Note Taking: Documenting observations and feedback during the test.

*Surveys and Questionnaires: Collecting standardized responses post-test.

*Interviews: Gathering detailed feedback through direct conversation.

*Usability Metrics: Analyzing task success rate, error rate, time on task, and user satisfaction.

Question 32

Why is Usability Testing important in UX Design?

Answer 32

Usability Testing is important because:

• It provides direct input on how real users use and perceive the product.
• Helps identify usability issues before the product is launched.
• Enhances user satisfaction by ensuring the product meets user needs.
• Can prevent costly redesigns by identifying problems early.
• Supports creating a more intuitive and effective user interface.