Week 7 Expert Evaluation Flashcards
Expert Evaluation: Benefits
Cost-effective
Independent evaluation
Early detection and actionable
insights
Saving time and money
Expert Evaluation: When?
Early design phase
* to identify potential usability issues
Mid-design iteration
* to assess the effectiveness of the design changes
Before a redesign
* to identify areas for improvement and make recommendations
Pre-launch
* to conduct a final assessment before product release
Post-release
* for future product updates
Expert Evaluation: Types
Heuristics & Cognitive Walkthrough
Heuristic Evaluation
Used for:
* identifying problems with existing
prototypes (any kind of interface)
Not for:
* coming up with radically new solutions
Disadvantages:
* reinforces existing design - better solutions might exist
* not very repeatable - dependent on evaluators
Steps in Heuristic Evaluation
Find 3-5 evaluators
Briefing session on context
and process
Independent Evaluation
sessions (1-2 hours)
Consolidation
- Setting Context and Briefing
Set the scene
* Who are the target users?
* What can the users be expected to know?
* What context are they using this in?
* What tasks are you most interested in?
* Is there a particular focus for the evaluation?
Explain the process/expectations to all evaluators
* Provide necessary resources
- Independent Evaluation Sessions (1-2 hours)
Each evaluator:
steps through interface and
applies heuristics
uses heuristics list and
severity rating convention
produces a list of justified,
rated problems
goes through the interface at
least twice
Severity Ratings
Each violation/problem is
assigned a severity rating:
1. cosmetic problem (fix only if extra time available)
2. minor usability problem (low priority)
3. major usability problem - important to fix (high priority)
4. usability catastrophe - critical to fix
Rating considers:
* Frequency – common or rare? one time or repeating?
* Impact – easy or difficult to overcome?
Used to allocate resources
to fix problems
* Initially done independently by each evaluator
Heuristic Evaluation Cautions
Principles must be applied intuitively and carefully
* can’t be treated as a simple checklist
Scope issues
* Doesn’t necessarily predict users’ overall satisfaction
* Important problems may be missed
* Trivial problems may be identified
* Evaluators have biases
BUT: Sometimes it’s what
can be done within time
and resource constraints
* That will provide valuable feedback
Usability Heuristics: Which Heuristics to Use? (In general don’t list them)
Pick guidelines/principles/
heuristics most appropriate
for your context
Usually a set of 5-12
heuristics
Neilsen’s 10 Heuristics
- Visibility of system status
- Match between system and the real
world - User control and freedom
- Consistency and standards
- Error prevention
- Recognition rather than recall
- Flexibility and efficiency of use
- Aesthetic and minimalist design
- Help users recognize, diagnose, and
recover from errors - Help and documentation
https://media.nngroup.com/media/articles/attachments/Heuristic_Summary1-compressed.pdf
Cognitive Walkthrough
Evaluators walk through the
design prototype
* using a scenario, focusing on ease of learning
Simulates how users would
problem-solve each step in
the interaction
Cognitive Walkthrough (used for, Not for, Disadvantages)
Used for
* assessing ease of learning for a new user
* identifying detailed problems
* working out task sequence models
Not for
* assessing performance at highly skilled
or frequently performed tasks;
* finding radically new approaches
Disadvantages
* narrow focus
* more setup needed than heuristic evaluation
Walkthrough Context
Can be done at any phase after
detail specification of UI exists
May use software or low-fidelity
prototype
Evaluation group is usually
development group
Group members given roles
during evaluation
* Presenter
* Facilitator
* Scribe
* User expert
Necessary Components for a Walkthrough (PUTS)
A visual interface prototype
Descriptions of who the users are
Descriptions of representative
tasks most users will want to do
Correct sequence of actions for
each task with the given interface
Questions for Walkthrough
(Sorta about if the user knows things)
Will the correct action be
sufficiently evident to the user?
* Will user know conceptually
how to accomplish task?
Will users be able to notice that
the correct action is available?
* Can user see what they should
use for next action?
Will the user associate and
interpret the response from the
action correctly?
* Will user know from the feedback that
they have made correct/incorrect action?
Conducting the Walkthrough
Evaluators walk through scenario
* Make note of problems, guided by the 3 questions
Walk through the actions
* Focus on credible stories based on user knowledge and goals
* Understand the problem-solving process allowing users to guess * Explore UI and select actions that seem to accomplish task
* Observe UI reaction to see if desired effect is achieved
* Decide what to do next
Use a LARGE screen everyone can see
Avoid distractions and diversions
Video/audio record if possible
Walkthrough Analysis
Consider both success and failure cases
Success: interface worked as it should
Failure: interface clearly fails
Stay on track – when failure occurs
Suggest Improvements
If user not aware action is required
* Eliminate the step if possible
* Prompt for correct sequence
* Change earlier step so user understands need for
current action?
If user does not know action is available
* Make action more obvious (menu, prompt)
* Add easily distinguishable submenu or key sequence
If user does not know action will achieve desired effect
* Provide labels that make sense to user
* Add hint that action is correct
If user does not know things are okay
* Any feedback is better than none
* Explicit feedback is better than general feedback
* Relate feedback to action and task
* E.g., your form has been submitted!
