Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Usability & User Experience Design Evaluation > Usability & UX evaluation slides information > Flashcards

Usability & UX evaluation slides information Flashcards

1
Q

Define Usability

A

“The user can do what he or she wants to do, that what he or she expects, to be able to do it, without hindrance, hesitation or questions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the Usbality measures

A

Efficiency (Time on task)
Effectiveness (Task completion)
Satisfaction (No discomfort, positive attitude)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Name aspects of Bailey’s human performance model

A
  • Somebody (human) -> Detach
  • Somewhere (context) => there are more places than you know
  • Something *activity) -> Does not exist in a vacuum

All these factors are in motion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Define User Experience

A

“A person’s perception / responses to the use, or anticipated use of a product, system or service”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Alternate defenition User Experience

A

The consequences of a user’s internal state (predispositions, expectations, motivation, mood) a systems characteristics (complexity, purpose,usability, functionality, etc.) And the context within which the interaction occurs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the difference between UX & Usability?

A
  • Usability is a part of UX

- User Experience is an extension of the satisfaction part of Usability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

The UX in time model

A

When:

  • Before Usage, Anticipated UX, Imagining an Experience
  • During Usage, Momentary UX, Experiencing
  • After Usage, Episodix UX, Reflecting on an experience
  • Over time, Cumulative UX, Recollection of periods of use
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Aspects you need to watch out for with Usefulness

A
  • Designing the right thing vs. Designing things right (optimizing Usability)
  • Danger of early usability testing
  • Apply the right method for the right phase (Concept drawings do not provide deep insights, Task-centered evaluations focus on the negative)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Why evaluate?

A
  • Informed design decisions
  • Identify design problems and fix them
  • Money (cost reduction, profit gain)
  • Create feeling of involvement
  • Generation of scientific research/ intermediate level knowledge
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Factors to inform your research design

A
  • Timing
  • Goal (Benchmark, high-level issue)
  • Desired output (Qualitative, Quantitative)
  • Subject (Prototype, product)
  • Available budget
  • Level of interaction with participants
  • Task-Driven or Natural form (free-form)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How to use evaluations to obtain input for user research

A
  • Evaluations of existing (competitor) products can be part of research
  • Other than unearthing existing issues, evaluations can give input for future directions
  • Shared methods
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Focus of your investigation is either on:

A
  • Behavior

- Attitude

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

The product feature your investigating is either

A
  • Interface

- Information architecture

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Choices for research

A

Lab vs field testing
Remote evaluations - (a)synchronous
Incidental - Long term

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Informal evaluations are..

A
  • Fast & Cheap
  • Less planning
  • Loose Recruitement (even colleagues)
  • Less structured
  • Less formal output
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Setting up a study holds the following considerations

A
  • Planning (which approach, methods at what time), triangulation)
  • Participants (Sampling, depends on question, sample size)
  • Ethics (Consent, Right to witdraw, Information is confidential)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Intermediate level knowledge

A

Represents interesting and important knowledge in design research

Examples:

  • Design methods and tools
  • Design guidelines
  • Usability heuristics
  • Patterns
  • Strong concepts
  • Experiential qualities
  • Criticism
  • Annotated portfolios
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are design research directions?

A
  • A holistic vision for UX evaluation (Hedonic & Pragmatic)
  • Inspection methods for Hedonic attributes (Psychophysiology)
  • Core skills needed for evaluation ( What do we all need as interdisciplinary field
  • Learn from evaluation in practice)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Triangulation

A

= An approach to data collection and analyses that uses multiple methods, measures or approaches to look for convergence on product requirements or problem areas

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Benefits of triangulation

A
  • More in-depth understanding
  • More richness, varied set of data
  • More convincing, persuasive recommendations
  • Reduce ‘innaproppriate certainty’ that not much is wrong with a design
  • Prioritizing requirements
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What can you triangulate?

A

Any source of data (usability tests, interview, think aloud)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Triangulation methods

A

Between methods:

  • Research methods (implicit & explicit)
  • Qualitative & Quantitative

Within methods:

  • Observer
  • User groups
  • Geographic
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Sequential triangulation

A

Sequential (between studies)

  • Dairy studies, followed by interview
  • User research, followed by evaluation of a prototype
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Concurrent triangulation

A

Concurrent ( In same study more methods)

- Eye tracking and a survey

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What are the performance metrics

A
  • Time on task
  • Task success
  • Errors
  • Click paths
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What are the advantages of using performance metrics?

A
  • Statistically valid
  • Clear to explain
  • Easy to compare
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

What are Challenges with performance metrics

A
  • Beware of number fetishism
  • Cause & effect implications
  • Covariates
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

When is a task not succesful?

A
  • Person gives up
  • Time limit reached
  • Researcher intervenes
  • Incorrect answer
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Levels of succes

A

Usability perspective:

  • Complete succes: with or without assistanc
  • Partial success: with or without assistance
  • Failure: User thought it was complete but it wasn’t, user gave up

UX perspective:
1 = no problem (user completed task without difficulty)
2= Minor problem (completed task, with slight detour or small mistakes)
3= Major problem (completed task, but with major problems)
4 = Failure / gave up

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

Different types of task succes:

A

a. Binary succes ( 1= succesful, 0= failure)
b. Levels of succes (
c. Task failure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

Types of task failure

A
  • Giving up
  • Moderator ‘calls’ it
  • Too long
  • Wrong
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

What can time on task mean

A

Short time on task: Not hindered by issues
Long time on task: engagement
Ask how the user perceived the passage of time

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

When to use ‘error’ metric?

A
  • Significant efficiency loss
  • Significant economic loss
  • Error would result in task failure
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

How to fix slips? (Typo, accidental misclick)

A
  • Reduce number of actions required

- Work on error handling and prevention

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

How to fix mistakes (clicking something unclickable, entering wrong data, performing wrong action)

A
  • Provide better support
  • Hints
  • Fix User Interface problems
  • Fix Usabillity study scenario / tasks
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

What do we measure with the metric ‘click path’?

A
  • Number of actions needed to complete a task
  • First click tests
  • ( may be relevant to identify ideal path)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

What are the challenges of user testing?

A
  • Walk up and use: natural first interaction, you may - need familliarity or expertise
  • Standardized tasks: Assign variance to what we want to test
  • Scale of the problem: User contexts differ greatly, need often be practical
  • Usability trap: Only creating what a usability test can measure (avoid complexity)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

What is log data?

A

Being able to reconstruct (parts of) the interaction of one particular system (while also being able to generalize and find patterns)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

What are advantages of log data?

A
  • Relatively non invasive
  • Natural product use
  • Suitable for measure over time
  • Customizable ‘events’
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

What are forms of logging?

A
  • Primitive form: Keyboard, mouse logging
  • Identifying tasks
  • Deriving performance measures
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

Analytics

A
  • A/B testing
  • A + B testing
  • Heatmaps and scrollmaps (Hotjar)
  • Video recording (Hotjar)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

Advantages of analytics compared to Usability

A
  • Generally collected remotely and ansynchronous
  • Larger sample size (more anonymous)
  • Over time (returning users, learnability, active users, adoption)
  • Allows you to discover patterns
  • Natural product use
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

Differences between logging and analytics

A
  • Very similar, but then with standardized measures

- Logging can be part of a usability study and an anlytics approach

44
Q

What are emotional & implicit measures?

A
  • Relatively objective measures (spontaneous, subconscious)

- Decreases the effect of social desirability

45
Q

Why do you need to handle implicit measures with care?

A
  • Noisy, sensitive to external influences

- Open to interpretation - What not why

46
Q

Types of implicit measures

A
  • Psychophysiology ( EEG, HR, fMRI)
  • Eye tracking
  • Facial behavior analysis
  • Pose, motion tracking
47
Q

What is eye tracking?

A

Focus on patterns of saliency of particular features,

doesn’t help to determine usability problems, but helps to determine what lead to that problem

48
Q

How does eye tracking work?

A

The light reflected from the cornea and the center of the pupil are used to inform the eye tracker about the movement and direction of the eye

49
Q

What is facial behavior analysis ?

A

The analysis of strong emotional reactions

Mostly primite emotions (hapiness, sad)

Done with facial reader

50
Q

When to use facial behavior analysis?

A
  • When strong emotions occur
  • When the temporal pattern of emotion is important
  • When strong emotional reactions matter in the usability of the product (if you want to avoid strong emotions for work for example, or want to trigger for advertisement)
51
Q

What are challenges of facial behavior analysis?

A
  • Limited to basic emotions and extreme cases

- Unclear what triggered a change in emotion

52
Q

What are challenges of Pose and Motion trackers?

A
  • Raw data, no basic interpretation

- Pose and motion are complex constructs

53
Q

The pyramyd of measures

A

What people: Say/think, do/use, know/feel/dream
Techniques: Interviews, observations, generative sessions
Knowledge: explicit, observable, tacit, latent

54
Q

Determinants of type of qualitative studies

A
  • Interaction with the user (observation, interview)
  • Location of data collection (lab, field)
  • Number of participants (individual, group)
  • Modality of interaction ( written, direct oral)
  • Freedom in interaction
55
Q

The goals of qualitative studies

A
  1. Get an overview of usability issues that are currently present in the product (based on do/use)
  2. Identify the user’s experiences with the product, and possible improvements
56
Q

Potential solutions to gather insights into usability

A
  • There is no foolproof way to identify all the usability flaws
  • Use mix of methods
  • Concentrate on the most important features
  • Think about the goals of the evaluation
  • Use multiple evaluatios (within-methods triangulation)
  • Reflect on your practices
57
Q

What are qualitative say&think methods

A
  • Individual interview
  • Contextual interview
  • Laddering
  • Focus group
  • Think aloud
58
Q

What are qualitative do & use methods?

A
  • Controlled observations
  • Naturalistic observations
  • Dairy studies
  • Expert review: Holostic approach
  • Expert review: Cognitive walkthrough
59
Q

What is an Individual interview?

A

A talk one on one with a user, f2f or through phone or video. This interview can discover beliefs, attitudes, desires and reactions towards a concept.

60
Q

What are advantages of an Individual interview?

A
  • Deep understanding of needs and wants

- Adaption

61
Q

What are dissadvantages of an Individual interview?

A
  • Costly

- Explicit knowledge only (just say/think, no do/use)

62
Q

What is an contextual interview?

A

Watch people’s behavior in their own environment
And ask questions while observing.
This type of interview is done when context is important.

63
Q

What are the advantages of a contextual interview

A
  • Natural behavior.
  • Real life task/environment
  • Explicit & observable knowledge
64
Q

What are the dissadvantages of a contextual interview?

A
  • Costly

- Poor control conditions

65
Q

What is laddering?

A

Laddering is a qualitative interview technique to discover meaningull information about a product or service. This is done by looking at Attributes-Consequences-Values. This technique is done by keep asking ‘why’.

66
Q

What are the advantages of Laddering?

A
  • Deep understanding and getting to high-level goals and underlying values.
67
Q

What are the dissadvantages of Laddering?

A
  • Can feel tedious/childish

- Not easy to conduct

68
Q

What is a focus group?

A

A group of people that haves moderated discussions (N=5-10), often informal and semi-structured. This type of research is done to discover attitudes, beliefs, desires and reactions.

69
Q

What are the advantages of a focus group?

A
  • Cost effective

- Discover what the user feels, needs, value

70
Q

What are dissadvantages of a focus group?

A
  • Explicit knowledge only
  • Requires an advanced moderator
  • Poor method for Usability testing
  • Group dynamics
71
Q

What is good to do during a Focus Group, and what is good not to do?

A
  • It is good to ask open-ended questions
  • Balance participation
  • Let participants use their own words
  • Not good to have a stiff and formal presentation
72
Q

What is think aloud?

A
  • Users verbalize the whole proces while performing tasks, this technique is used to uncover what they really think.
73
Q

What are advantages of think aloud?

A
  • Cost effective
  • Flexible (both for prototypes and finished products)
  • Real time measure of experience
74
Q

What are dissadvantages of think aloud?

A
  • Can feel unnatural
  • Chance of bias
  • Individual differences
  • Affects other measures (eye tacking)
75
Q

What are the strategies of thinking aloud?

A
  • Rigid: No interference/neutral reminder (keep talking)

- Speech/clues: Acknowleding contributions, ask for clarification, encouragement

76
Q

What is co-discovery?

A

It is the method of think-aloud with two people, more natural to share thoughts

77
Q

What is a controlled observation?

A
  • Observing users interact with a product or service in a lab environment
  • They are asked to reach a specific goal, or do a specific task
  • Context for quantitative measures (usability issues)
78
Q

What is a naturalistic observation

A
  • Observing users interact with a product ‘in the wild’

- This technique is used when the context is impotant

79
Q

What are the advantages of a controlled observation?

A
  • Control over the condition
  • Structured approach
  • Quick
80
Q

What are dissadvantages of a controlled observation?

A
  • Pre-defined tasks only

- Unnatural context/behavior

81
Q

What are advantages of a naturalistic observation?

A
  • Natural product use
  • Ecological validity
  • Observe how the product is used in daily life
82
Q

What are dissadvantages of a naturalistic observation?

A
  • Little control over conditions
  • Costly
  • Hawthorne effect (show what you think the observer wants to see)
83
Q

What can you observe?

A
  • Space
  • Actor
  • Activity
  • Object
  • Act
  • Event
  • Time
  • Goal
  • Feelings
  • Interaction (user-product, social)
  • Use in context (routine, appropriation)
84
Q

What is an Expert Review: Holistic evaluation?

A
  • Inspection method: Analysis by an expert
  • Comparing against heuristics
  • Holistic approach
85
Q

What is an Expert review: Cognitive walkthrough?

A
  • Inspection method: Soecific tasks, focus on ease of completion
  • Particular useful for ‘walk-up and use system
86
Q

Advantages of Expert review:

A
  • Flexible

- Quick

87
Q

Disadvantages of Expert review:

A
  • Better to use more than one expert (=costly)
  • No real user, using the product
  • Often focused on minor usability issues
88
Q

What questions can a expert aks him/herself during a cognitive walkthrough?

A
  • Will users want to produce whatever effect the action has?
  • Will users see the (UI) controls
  • Will users know that the control produces the effect they want
  • After the action is taken, will users understand the feedback
89
Q

Two types of Qualitative data coding:

A
  • Manual coding (computer-aided)

- Computerized analysis

90
Q

What are the different types of Manual coding?

A
  • Data driven coding (bottom up)
  • Theoretically driven coding (top down)
  • Combination of bottom up and top down coding
91
Q

What is a sentiment analysis?

A
  • Detect the polarity (positive/negative) of an opinion with a certain degree of confidence
  • Uncover themes of sentiment in large scale data (e.g. twitter posts, reviews)
92
Q

What are the steps of systematic coding?

A
  1. Transcription of verbal data
  2. Immerse yourself in the data, read the transcript several times
  3. Chunking into single meaningful units
  4. Close reading and detailed coding
  5. Analysis and sorting of similar codes into categories
  6. Recode text
  7. Finalizie coding system
93
Q

What are bio signals?

A
  • Brain activity
  • Heart rate
  • Skin conductance
  • Skin temperature
  • Muscular activity
94
Q

Which two constructs are determinants for emotion?

A
  • Arousal: Intensity of emotion

- Valence: Whether this arousal is positive or negative

95
Q

The history of psychophysiological measures, can you name different steps?

A
  • Behaviorism: Has a strong influence, Classical conditioning
  • Stimulus response model: Who says what to whom with what effect
96
Q

What are assumptions on information processing?

A
  • Humans have innate capacity (inherited tendencies)
  • Humans are active information seekers
  • Knowledge is stored in the brain (cognitive system requires representation)
  • Humans are dynamic systems
  • Systems are divisible
97
Q

What are the kinds of self-report measures?

A
  • Surveys & questionnaire
  • Dairy Studies
  • UX curve
  • Sentence completion
98
Q

Why would you write your own survey questions?

A
  • Helpful for demographics

- For behavorial and attitudinal metrics

99
Q

Why would you use a survey/questionnaire?

A
  • Profile your use group
  • Assess values, needs, beliefs
  • Evaluate design (Usability/UX)
  • Can use this anywhere in the process
100
Q

What is a dissadvantage of surveys/questionnaire?

A
  • Prone to social desirability

- After the fact

101
Q

What type of questions can be in a survey?

A
  • Open questions (free form, listings)
  • Likert scale
  • Semantic differential scale
  • Frequency scales
  • Paired comparison scale

Best to use standardized scales

102
Q

What is a dairy study?

A

Tracking everyday prolonged use, follow a specific type of (long-term) experience. Which can be done remote using paper or technologies.

103
Q

What is a UX curve?

A
  • Letting participants draw a curve to describe their experience about a product changed over time
  • It is a retrospective method (memory rather than in-situation experience)
  • Different scales for different aspects of UX (attractiveness, ease of use)
  • Reveal impactful experiences over time
104
Q

What types of data can you present?

A
  • Task performance (Often in graphs, task succes, time on task, errors)
  • Attitude (self-report, interviews)
  • Usability issues (Number of unique issues, frequency of issues per participant
105
Q

What are the two types of EDA?

A

CNS: Measures direct brain activity (EEG, fMRI)
PNS: Measures evoked response (heart rate, skin conductance)

106
Q

What are different types of PNS responses?

A
  • Autonomic (involuntary: Sympatetic, parasympatetic)
  • Somatic (voluntary)
  • Electrodermal activity ( Sympatetic activation in autonomic branch of the periphieral nervous system
107
Q

What are different types of Electrodermal activity:

A
  • Tonic activity (baseline of continuous activity)
  • Phasic activity (temporary evkoed response (to stimuli))
  • Nonspecific activity ( temportary evoked response (without stimuli))

Usability & User Experience Design Evaluation (2 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions