Lecture 3 Flashcards
Evaluation with users
Qualitative vs Quantative
Literative Review
Use existing literature to analyze usability (only works if very similar context)
Cognitive Walkthrough
Expert evaluator walks through system and analyzes questions
Heuristic Evaluation
Choose usability heuristic and step through tasks and check if heuristics are followed
Nielsen Usability Principle
- Keep the interface simple!
- Speak the user’s language!
- Minimize the user’s memory load!
- Be consistent and predictable!
- Provide feedback!
- Design clear exits and closed dialogs!
- Offer shortcuts for experts!
- Help to recover from errors, offer Undo!
- Prevent errors!
- Include help and documentation!
Working with Users
Setting goals, Relationship with participants, Triangulation, If questions reveal that goal was not sufficiently refined
Working with participants
Hard, uncomfortable so treat with respect.
Interviews
+ Allow to go deep and explore
+ Allows to capture hard to capture data
- Harder to conduct, take time, hard to analyze,
Types of Interviews
Unstructured: Some questions are planned but interviewer has freedoms
Structured: Rigid script is followed
Semi-structured: Mix of the above
Types of Questions:
Closed: Yes/No, multiple choice (easy to analyze)
Open: Any answer possible (good to explore)
Tasks: Finish this sentence
Mapping: Draw a diagram
Conduct an interview
Make participant feel comfortable, start with question they can relate to, then insert some harder questions, most difficult questions near the end, end with easy questions, debrief.
Conceptual Model Extraction
Show user prototype or screenshots and ask user to explain
+ helps understanding of native conceptual model
- does not capture learnability
Silent observation
Watch user in lab while working on task
+ no overly helpful assistent, can discover bigger problems
- no understanding of decisions, may end in no result at all
Think aloud
As in Silent observation but ask user to say thoughts out loud.
+ allows for more results
- can change behaviour, feels weird, is not easy for participant
Constructive Interaction
Two people work on task together
+ more natural then think aloud , and allows people with different roles
Retroperspative Testing
User looks at recordings and explains actions
+ good starting point for interview, results in constructive advice
Grounded theory
Produce theories that are grounded in data, inductive approach that begins with data but may have deductive component
Glaser
No literature review until theory is developed, everything is data(interviews, open ended questions).
Coding: Substansive (Open), Theoretical (all substansives codes are related to core category)
Inductive Coding
Kathy Charmaz
Concepts are constructed not discovered, Literature review must be done at start
Coding: Initial open coding, select categories from most frequent codes, specify relationships between categories and form theory
Strauss
Literature review can be done at the beginning, develops and tests theory
Coding: Open, Axial (connections), theoretical
Deductive Coding
Inductive coding
Two coders:
First creates categories from data of preliminary findings
Second gets evaluation objectives and raw text from which categories have been found, creates second set of categories
Are then merged into one set of categories
Inter coder agreement
Compare two sets of categories and report inter coder agreement.
Cohen Kappa > 0.75 good level of coding agreement.
