Week 7 Expert Evaluation Flashcards
Expert Evaluation: Benefits
Cost-effective
Independent evaluation
Early detection and actionable
insights
Saving time and money
Expert Evaluation: When?
Early design phase
* to identify potential usability issues
Mid-design iteration
* to assess the effectiveness of the design changes
Before a redesign
* to identify areas for improvement and make recommendations
Pre-launch
* to conduct a final assessment before product release
Post-release
* for future product updates
Expert Evaluation: Types
Heuristics & Cognitive Walkthrough
Heuristic Evaluation
Used for:
* identifying problems with existing
prototypes (any kind of interface)
Not for:
* coming up with radically new solutions
Disadvantages:
* reinforces existing design - better solutions might exist
* not very repeatable - dependent on evaluators
Steps in Heuristic Evaluation
Find 3-5 evaluators
Briefing session on context
and process
Independent Evaluation
sessions (1-2 hours)
Consolidation
- Setting Context and Briefing
Set the scene
* Who are the target users?
* What can the users be expected to know?
* What context are they using this in?
* What tasks are you most interested in?
* Is there a particular focus for the evaluation?
Explain the process/expectations to all evaluators
* Provide necessary resources
- Independent Evaluation Sessions (1-2 hours)
Each evaluator:
steps through interface and
applies heuristics
uses heuristics list and
severity rating convention
produces a list of justified,
rated problems
goes through the interface at
least twice
Severity Ratings
Each violation/problem is
assigned a severity rating:
1. cosmetic problem (fix only if extra time available)
2. minor usability problem (low priority)
3. major usability problem - important to fix (high priority)
4. usability catastrophe - critical to fix
Rating considers:
* Frequency – common or rare? one time or repeating?
* Impact – easy or difficult to overcome?
Used to allocate resources
to fix problems
* Initially done independently by each evaluator
Heuristic Evaluation Cautions
Principles must be applied intuitively and carefully
* can’t be treated as a simple checklist
Scope issues
* Doesn’t necessarily predict users’ overall satisfaction
* Important problems may be missed
* Trivial problems may be identified
* Evaluators have biases
BUT: Sometimes it’s what
can be done within time
and resource constraints
* That will provide valuable feedback
Usability Heuristics: Which Heuristics to Use? (In general don’t list them)
Pick guidelines/principles/
heuristics most appropriate
for your context
Usually a set of 5-12
heuristics
Neilsen’s 10 Heuristics
- Visibility of system status
- Match between system and the real
world - User control and freedom
- Consistency and standards
- Error prevention
- Recognition rather than recall
- Flexibility and efficiency of use
- Aesthetic and minimalist design
- Help users recognize, diagnose, and
recover from errors - Help and documentation
https://media.nngroup.com/media/articles/attachments/Heuristic_Summary1-compressed.pdf
Cognitive Walkthrough
Evaluators walk through the
design prototype
* using a scenario, focusing on ease of learning
Simulates how users would
problem-solve each step in
the interaction
Cognitive Walkthrough (used for, Not for, Disadvantages)
Used for
* assessing ease of learning for a new user
* identifying detailed problems
* working out task sequence models
Not for
* assessing performance at highly skilled
or frequently performed tasks;
* finding radically new approaches
Disadvantages
* narrow focus
* more setup needed than heuristic evaluation
Walkthrough Context
Can be done at any phase after
detail specification of UI exists
May use software or low-fidelity
prototype
Evaluation group is usually
development group
Group members given roles
during evaluation
* Presenter
* Facilitator
* Scribe
* User expert
Necessary Components for a Walkthrough (PUTS)
A visual interface prototype
Descriptions of who the users are
Descriptions of representative
tasks most users will want to do
Correct sequence of actions for
each task with the given interface
