Chapter 10A Flashcards
Conceptual Models?
Mental representation of how the object works and how interface controls effect it
Design model should equal customer’s model?
Mismatches lead to errors
Use customer’s likely conceptual model to design
Design guides?
Make things visible
Map interface controls to customer’s model
Provide feedback
Goals of evaluation?
Access extent of system functionality
Access extent of interface on user
Identify specific problems
Evaluation through Expert Analysis?
Cognitive Walkthrough
Heuristic Evaluation
Model-Based
Review-Based evaluation
Evaluation through User Participation?
Laboratory studies
Field studies
Cognitive Walkthrough?
Proposed by Polson
- Evaluates design on how well it support user in learning task
- Usually performed by expert in cognitive psychology
For each task walkthrough considers?
What impact will interaction have on user?
What cognitive processes are required?
What learning problems may occur?
Analysis focuses on?
Goals and knowledge: does the design lead the user to generate the correct goals.
The 3 Questions?
Will the correct action be sufficiently evident to the user?
Will the user notice that the correct action is available?
Will the user associate and interpret the response from the action correctly?
Advantages and Problems of Cognitive Walkthrough?
Few ethical & practical issues to consider because users are not involved.
Can be difficult & expensive to find experts.
Best experts have knowledge of application domain and users
Biggest problems of Cognitive Walkthrough?
Important problems may get missed
Many trivial problems are often identified
Experts have biases
Heuristic Evaluation?
Proposed by Nielsen and Molich
Usability criteria (heuristics) are identified
Design examined by experts to see if these are violated
Example heuristics?
System behaviour is predictable
System behaviour is consistent
Feedback is provided
Heuristic evaluation _____ design?
debugs
Why do User Testing?
Can’t tell how good UI is until?
- People use it
Expert review methods are based on evaluators who:?
- May know too much
- May not know enough
Hard to predict what real users will do
User Testing Requires an?
Artefact
- Simulation, prototype, full implementation
Laboratory Studies Advantages?
Specialist equipment available
Uninterrupted environment
Laboratory Studies Disadvantages?
Lack of context
Difficult to observe several users cooperating
Field Studies Advantages?
Natural Environment
Context Retained
Longitudinal Studies Possible
Field Studies Disadvantages?
Distractions
Noise
Experimental Factors?
Subjects
Variables
Hypothesis
Experimental Design
Ethical Considerations?
Usability tests can be distressing
Alleviating issues regarding Ethical Considerations?
Make voluntary with informed consent
Avoid pressure to participate
Let them know they can stop at any time
Stress that you are testing the system, not them
Make collected data as anonymous as possible
Preparing for a Test?
Select your customers Prepare scenarios that are - Typical of the product during actual use - Make prototype support these Practice to avoid "bugs"
Four Roles of Conducting a Test?
Greeter - puts users at ease and gets data
Facilitator - only team member who speaks
Computer - knows application logic
Observers - take notes and recommendations
Wizard of Oz technique?
Faking the interaction
User Test Proposal?
A report that contains
- Objective
- Description of system being tested
- Task environment and materials
- Participants
- Methodology
- Tasks
- Test measures
The tasks must?
Encourage participation
Order of Tasks?
Choose one simpler order
Training?
Depends how real system will be used
What if someone doesn’t finish?
Assign very large time and large # of errors or remove and note
Pilot study?
Helps you fix the problems of the study
Do two, first with colleagues, then with real users
Keeping variability down?
Recruit test users with similar background
Brief users to bring them to common level
Debriefing test users?
Often don’t remember, so demonstrate or show video segments
Ask for comments on specific features
Two types of data to collect?
Process Data
- Observations of what users are doing and thinking
- Qualitative
Bottom-Line data
- Summary of what happened
- Dependent variables
- Quantitative
Why focus on process data first?
Gives good overview of where problems are?
Evaluating Results?
Sort and prioritize observations - What was important - Lots of problems in the same area Create a written report on findings - Gives agenda for meeting on design changes Make changes and iterate
Evaluation Methods?
Think Aloud Cooperative evaluation Protocol Analysis Automated Analysis Post-Task Walkthrough Query Techniques
Think Aloud?
Users observed performing the task, describe what he/she is doing and why
Advantages of Think Aloud?
Simplicity -requires a little expertise
Can provide useful insight
Can show how system is actually use
Disadvantages of Think Aloud?
Subjective
Selective
Act of describing may alter task performance
Steps to Think Aloud?
Prompt/Encourage the user to keep talking
Only help on things you have pre-decided
Recording
Instructions to Participants?
Describe the purpose of evaluation
Tell them they can quit anytime
Demonstrate the equipment
Cooperative Evaluation?
Variation on think aloud where the interviewer cooperates with the user
Cooperative Evaluation Advantages?
Less constrained and easier to use
User is encouraged to criticize system
Clarification possible
Protocol Analysis?
Tools:
- Paper and pencil
- Audio
- Video
- Computer logging
- User notebooks
Automated Analysis Advantages?
Analyst has time to focus on relevant incidents
Avoid excessive interruption of task
Automated Analysis Disadvantages?
Lack of freshness
May be post-hoc interpretation of events
Interviews?
Analyst questions user on one-to-one basis usually based on prepared questions
Interview Advantages?
Can be varied to suit context
Issues can be explored more fully
Can elicit user views and identify unanticipated problems
Interview Disadvantages?
Very subjective
Time consuming
Questionnaires?
Set of fixed questions given to users
Advantages of Questionnaires?
Quick and reaches large user group
Can be analyzed more rigorously
Disadvantages of Questionnaires?
Less flexible
Less probing
Questionnaires styles of questions?
General Open-ended Scalar Multi-choice Ranked
Emotional Response Linked to?
Physical changes
These may help determine a user’s reaction to an interface
Using the Test Results?
Summarize the data
What does data tell you?
Update tasks & rethink design
Measuring Bottom-Line Usability?
Situations in which numbers are useful
Ease of measurement
Two Alternatives (Groups)
Between Groups Experiment
Within Groups Experiment
When in process?
Design vs Implementation
Style of evaluation?
Laboratory vs field
How objective?
Subjective vs objective
Type of measures?
Qualitative vs quantitative
Level of information?
High level vs low level
Resources available?
time, subjects, equipment, expertise
