Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Usability & User Experience Design Evaluation > Usability & UX evaluation Article information > Flashcards

Usability & UX evaluation Article information Flashcards

1
Q

What are Annotated portfolio’s?

A

They are a way to communicate design research

- This is done to suggest that design can become productive as research

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How are Annotated portfolio’s made?

A
  • By selecting a collection of designs
  • Representing them in an appropriate medium
  • And combining the design re-presentations with brief textual annotations
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

The essence of design research

A

“The essence of research is to produce knowledge, and the essence of design is to produce artifacts.”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Purposes of research

A
  • Primary: Knowledge production

- Secondary: Changing society, curing illnes and furthering economic growth

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What happends with abstraction of artifacts?

A
  • Has to take place in high order for an artifact to reach a high level of knowledge that is applicable across a broader range of scenario’s.
  • Abstraction adds knowledge value to an artifact
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is intermediate level knowledge?

A

Represetens interesting and important knowledge n design

  • Heuristics
  • Methods and Tools
  • Guidelines
  • Criticism
  • Experiential Qualities
  • Patterns
  • Strong concepts
  • Annotated portfolio’s
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Why can Annotated portfolio’s be seen as intermediate level knowledge?

A
  • Annotated portfolio’s can be seen as abstractions in the sense that more abstract entities have wide scope of applicability, which makes it broader knowledge also applicable to other scenario’s
  • Even though an annontationn are non-abstractions from the original design, they may still be appropariate as abstractions by the receivers of the annotated portfolio.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the five phases of evaluation history?

A
  • The system reliability phase (1940’s-50’s) -> Concerns of evaluation were on system realibility (minimizing system fault time, quickly repairing errors)
  • The system performance phase(1940’s-60’s) -> Evaluation focuses on issues related to the system performance (THe processing speed)
  • The user performance phase (1960’s-70’s) -> Users and their performance brought a new type of evaluator *experimental psychologist) Tried to enhance productivity through performance based metrics
  • Usability phase (1970’s-2000s) -> Evaluation for the ease of use of computers systems developed.
  • User experience phase (2000s-Present) -> The user experience paradigm emerged for design and evaluation, We non consider ‘non-utilitarian’ aspects of computer use
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

In the “Usability phase” what were the dimensions for user performance?

A
  • Learnability
  • Throughput
  • Attitude
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

In the “Usability phase” what were the performance metrics?

A
  • Time on task
  • Error rate
  • Accuracy
  • Task completion (Task succes)
  • Satisfaction
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Fameworks developed in the “User Experience phase”

A
  • Hassenzahl model (Pragmatic & Hedonic)
  • Normans levels of emotional design
  • Folizzi and Battarbee ‘User interactions that influences UX’
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Explain the Hassenzahl model:

A
  • Pragmatic attributes -> Ability to help us achieve behavorial goals (usability metrics)
  • Hedonic attributes -> Ability to evoke pleasure, allow self-expression, and provoke memories (identification, stimulation, satisfaction)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Explain Normans levels of emotional design

A
  • Viceral
  • Behavioral
  • Reflective
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Explain Forlizzi and Battarbee ‘User Interaction that influence UX’ model:

A
  • Fluent
  • Cognitive
  • Expressive
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Difference between UX evaluation and Usability evaluation

A

UX evaluation focuses on Hedonic attributes, Usability evaluation on pragmatic attributes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

The present state of evaluations has common approaches, name these:

A
  • Metrics
  • Post-use usabillity questionnaires
  • Eye tracking
  • Mouse-tracking (click-paths)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

To test Hedonic attributes, which approaches are used?

A
  • Post-use questionnaires
  • Interviews
  • Focus groups
  • Self-report techniques (emocards, personal meaning maps)
  • Facial expressions
  • Psychophysiological reactions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are inspection methods?

A

They aim to replace user feedback with expert judgment and focus exlusively on pragmatic attributes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are Inspection methods?

A
  • Early methods include guidelines and checklists

- Common method is Expert reviews: Heuristic evaluation & Cognitive walkthrough

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are Field methods?

A
  • Represents techniques conducted in natural settings
  • Many of these methods can be adapted to adress either pragmatic or hedonic attributes
  • Common field methods
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What are common field methods?

A
  • Behavioral observations
  • Collages or artifacts
  • Log analysis
  • Dairy studies
  • The experience sampling method
  • Living laboratory (A/B testing)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What can be stated about the future of evaluations?

A
  • Evaluators have been foced to dapt to the chaning use context of technological innovations
  • Because of the current speed, and complexity of the changes, evaluators struggle to keep pace
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

On what do most of the discussions around the future of evaluation evolve now?

A
  • Criticism on current methods
  • Concerns raised about method reliability
  • Potential harmfulness in certain situations
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Five potential research directions for future design research:

A
  • Create a more Holostic vision for UX evaluation
  • Develop inspection methods for Hedonic attributes
  • Examine core skills needed for evaluation
  • Investigate informal approaches to evaluation
  • Learn from evaluation in practice
25
Q

User Experience defenition =

A

A person’s perceptions and responses that result from the use or anticipated use of a product, system or service.

26
Q

User experience design shift

A
  • UX has been extended to incorporate henoic qualities such as”Aesthetics, stimulation, and Identification
  • UX is also a dynamic concept influenced by contextual aspects.
  • UX offers a much more holistic and dynamic take on interaction with product that just usability
  • UX is a multidisciplinary field with: cognitive science, design, psychology, engineering
27
Q

Four triangulation opproaches (Denzin)

A
  • Data Triangulation: The use of a mix of data sources in a study
  • Investigator Triangulation: A number of researchers investigating the same phenomenon
  • Theory triangulation: The use of a number of theories used for interpreting results of a study
  • Methodological Triangulation: The use of more than one method to study a phenomenon
28
Q

Two overarching types of employing two or more methods:

A
  • Sequential: First either qualitative or quatitative is used, and the other type is used in a following study to explain, explore or validate results.
  • Concurrent : Two or more methods are emplyed within the same study to cross-validate findings
29
Q

Guidelines for performance metrics

A
  • An adequate sample size is required
  • Avoid overrelying on performance metrics when your goal is to simply uncover basic usability problems
  • Performance metrics tell the what, but not the why
30
Q

Five types of performance metrics:

A
  • Task succes: Binary succes, levels of succes, task failure
  • Time on task: measures how much time is required to complete a task
  • Errors: reflect the mistakes made during a task
  • Efficiency: The amound effort a user expends to complete a task
  • Learnability: improvement over time
31
Q

What is important to think about when you use ‘Task succes’ as a metric

A
  • Every task a user is asked to perform must have a clear end state or goal
  • To determine whether a task is succesful, succes criteria must be defined
32
Q

Types of task failure

A
  • Giving up
  • The moderator ‘calls’ it
  • Too long
  • Wrong
33
Q

Level of succes

A

Usability (Complete succces, partial succes, failure)

UX (1= complete succes, 2= Minor problems, 3= Major problems 4= Failure/give up)

34
Q

What are ways of giving assistance?

A
  • Moderator takes participant back to the home page/ reset task
  • Moderator asks the participant probing question or restates the task
  • Moderator answers a question or provides information that helps the participant complete the task
  • Participant seeks help from an outside source
35
Q

Common approaches to ending an unsuccesful task:

A
  1. Tell the user they should stop until they reach the point they would normally give up
  2. Apply a three strikes and you’re out rule
  3. Call the task after a predetermined amount of time has passed
36
Q

What doest Time on task measure well?

A

The efficiency of using a problem, this is only when a short time on task is desirable

37
Q

How to improve the validity of measuring Time on task right?

A
  • Two moderators can measure time at the same time

- The video can be recorded and include time stamps

38
Q

Errors, when to measure?

A
  • When an error will result in a significant loss in efficiency
  • When an error will result in significant costs to your organization or the end user (parkmobile)
  • When an error will result in task failure
39
Q

What type of errors are there?

A
  • Entering incorrect data into a form field
  • Making wrong choice in drop down list
  • Taking incorrect sequence of action
  • Failing to take key action
40
Q

What issues can occur when counting errors?

A
  • Make sure you do not count errors double

- Sometimes an error rate is not enough, you also need to look at the type of error

41
Q

Besides time on task, how can efficiency be measured?

A

By looking into the effor required to complete a task

42
Q

What types of effort are there?

A
  • Cognitive effort: Involves finding the right place to perform an action, deciding what action is necessary and interpreting the results of the action
  • Physical effort: involvves the physical activity required to take action (e.g. moving a mouse)
43
Q

what is the beheavior lostness?

A

Lostness consists of:

  • The number of different pages visited to perform the task
  • the total number of pages visited while performing task, including revisits to same page
  • The minimum number of pages that must be visited to accomplish task
44
Q

What is Learnability?

A

The extent to which something can be learned efficiently. It’s an essential metrics when you need to know how someone develops over time

45
Q

What are the approaches to measuring learnability?

A
  • Trials within the same session: tasks are performed one after another without breaks, this doest not take memory loss into account
  • Trials within the same session with breaks in between each task (break is a distractor)
  • Trials between session: The participant performs tasks over multiple sessions
46
Q

What are the key metrics according to ISO?

A
  • Effectiveness (accuracy and completeness to which users achieve specified goals)
  • Efficiency (resources expended in relation to the accuracy and completeness wiith which users achieve goals
  • Satisfaction (the comfort and acceptibility of use)
47
Q

What is the metric to measure Effectiveness?

A
  • Completion rate

- Number of errors

48
Q

What is the metric to measure efficiency?

A
  • Task time (in combination with task completion)
  • Overall relative efficiency (The relative efficiency uses the ratio of the time taken by the users who successfully completed the task in relation to the total time taken by all users. ) ‘
49
Q

What is the metric to measure satisfaction?

A
  • Task level satisfaction (after task completion user should fill in a task how difficult that task was)
  • Test level satisfaction (is measured by giving a formalized questionnaire to each test participant at the end of the session) This serves to measure their impression of the overall ease of use.
50
Q

What number of participants for different studies?

A
  • Qualitative user testing: 3-5 users

- Quantitative: 20 users for each design

51
Q

What are examples of heuristics?

A
  • Visbility of system status
  • Match between system and the real world
  • Aestetic and minimalist design
  • Flexibility and efficiency of use
  • Help and documentation
52
Q

What is laddering?

A

A technique that is particularly helpful in eliciting goals and underlying values and therefore, possibly helpfull during early stages of user experience research

53
Q

What is the Means End Chain theory?

A

Provides a framework for captuaring qualitative laddering research data in consumer space and a model for assessing cnsumer values and behavios:

  • Attributes
  • Consequences
  • Core values
54
Q

What is a hierarchical value map?

A

All the major means end and atribute-consequence-value connections and describes individuals behavior based on their core values.

55
Q

Types of eye movements in eye tracking

A
  • Saccades
    Observed while conducting search tasks (jerky eye movements, both eyes)
  • Smooth pursuit
    much slower eye movements, used to track slowly and regularly moving visual targets
  • Compensatory
    eye movements stabilize the image on the retina when the hear or trunk is moved.
  • Vergence
    rotational eye movements of the eyes in opposite direction, in response to focusing error from visual targets moving closer or further away.
  • Miniature
    refers to eye movement that are under 1 degree of visual angle and include small corrective notions
  • Nystagmus
    refers to a unique sawtooth-pattern of eye movement caused by a rapid back-and forth movement of the eye.
56
Q

Avantages of eye tracking

A

The ability to record the micro flow of visual attention in a non-intrusive way

57
Q

Dissadvantages of eye tracking

A
  • Sensitive for failure

- Strong individual differences

58
Q

Eye-mind hypotheses, what you need to know

A
  • Tasks should be designed to require endcoding and processing of visual information to achieve well-specified task goals

Usability & User Experience Design Evaluation (2 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions