Lecture 3

Question 1

Evaluation with users

Answer 1

Qualitative vs Quantative

Question 2

Literative Review

Answer 2

Use existing literature to analyze usability (only works if very similar context)

Question 3

Cognitive Walkthrough

Answer 3

Expert evaluator walks through system and analyzes questions

Question 4

Heuristic Evaluation

Answer 4

Choose usability heuristic and step through tasks and check if heuristics are followed

Question 5

Nielsen Usability Principle

Answer 5

1. Keep the interface simple!
2. Speak the user’s language!
3. Minimize the user’s memory load!
4. Be consistent and predictable!
5. Provide feedback!
6. Design clear exits and closed dialogs!
7. Offer shortcuts for experts!
8. Help to recover from errors, offer Undo!
9. Prevent errors!
10. Include help and documentation!

Question 6

Working with Users

Answer 6

Setting goals, Relationship with participants, Triangulation, If questions reveal that goal was not sufficiently refined

Question 7

Working with participants

Answer 7

Hard, uncomfortable so treat with respect.

Question 8

Interviews

Answer 8

+ Allow to go deep and explore
+ Allows to capture hard to capture data
- Harder to conduct, take time, hard to analyze,

Question 9

Types of Interviews

Answer 9

Unstructured: Some questions are planned but interviewer has freedoms
Structured: Rigid script is followed
Semi-structured: Mix of the above

Question 10

Types of Questions:

Answer 10

Closed: Yes/No, multiple choice (easy to analyze)
Open: Any answer possible (good to explore)
Tasks: Finish this sentence
Mapping: Draw a diagram

Question 11

Conduct an interview

Answer 11

Make participant feel comfortable, start with question they can relate to, then insert some harder questions, most difficult questions near the end, end with easy questions, debrief.

Question 12

Conceptual Model Extraction

Answer 12

Show user prototype or screenshots and ask user to explain
+ helps understanding of native conceptual model
- does not capture learnability

Question 13

Silent observation

Answer 13

Watch user in lab while working on task
+ no overly helpful assistent, can discover bigger problems
- no understanding of decisions, may end in no result at all

Question 14

Think aloud

Answer 14

As in Silent observation but ask user to say thoughts out loud.
+ allows for more results
- can change behaviour, feels weird, is not easy for participant

Question 15

Constructive Interaction

Answer 15

Two people work on task together
+ more natural then think aloud , and allows people with different roles

Question 16

Retroperspative Testing

Answer 16

User looks at recordings and explains actions
+ good starting point for interview, results in constructive advice

Question 17

Grounded theory

Answer 17

Produce theories that are grounded in data, inductive approach that begins with data but may have deductive component

Question 18

Glaser

Answer 18

No literature review until theory is developed, everything is data(interviews, open ended questions).
Coding: Substansive (Open), Theoretical (all substansives codes are related to core category)
Inductive Coding

Question 19

Kathy Charmaz

Answer 19

Concepts are constructed not discovered, Literature review must be done at start
Coding: Initial open coding, select categories from most frequent codes, specify relationships between categories and form theory

Question 20

Strauss

Answer 20

Literature review can be done at the beginning, develops and tests theory
Coding: Open, Axial (connections), theoretical
Deductive Coding

Question 21

Inductive coding

Answer 21

Two coders:
First creates categories from data of preliminary findings
Second gets evaluation objectives and raw text from which categories have been found, creates second set of categories
Are then merged into one set of categories

Question 22

Inter coder agreement

Answer 22

Compare two sets of categories and report inter coder agreement.
Cohen Kappa > 0.75 good level of coding agreement.