User Testing Flashcards
From UI to UX
Evolution of computer systems:
- OF ‘Can this goal be achieved with a comp’
- UI ‘How easily can I achieve this goal with a comp’
- UX ‘How __ can I achieve this goal with a comp’
Controlled Experiment: Formative Evaluation and Controlled Experiment
- Putting an interface in front of real users
- Formative evaluation:
- Find problems for next iteration of design
- Qualitative observations
- Controlled Experiment:
- Test hypothesis e.g. X is faster than Y
- Quantitative observations (time, error rate)
Controlled Experiment: Measuring Usability of Alternative UIDs
- The experiment will measure time taken to make a choice from the menu on 2 designs:
Design X - menu with invisible triangular zone
Design Y - Menu without invisible triangular zone
Controlled Experiment: Ethics of User Testing
- Users are human beings:
- Human subjects have been seriously abused in the past e.g. Yale electric shock study
- What’s dangerous about a computer interface
- Hopefully nothing, minimal physical and psychological risk
- Psychological pressure on a user:
- performance anxiety
- comparing self to other subjects
- feeling stupid in front of observers
Controlled Experiment: Q1 What should you do when preparing a user test and what should be said to users when briefing them for the test, for ethical reasons?
- Treat the user with respect:
- Time: Dont waste time (pilot the test with a few dress rehearsal and practive experimenter role)
- Comfortable: Make the user comfortable (‘we’re testing the system not testing you’)
- Information: Inform the user as fully as possible (Tell them how they’re going to be recorded and answer any questions before hand)
- Privacy: Preserve their privacy (‘your results are completely confidential’)
- Control: Let them stop at any time (‘You can stop at anytime’)
Experiment the Design:
- start with a testable hypothesis
- dependant and independent variables
Controlled Experiment: Q2 What’s the hypothesis and what are the independent and dependent variables?
- Hypothesis: DesignX is faster than DesignY
- Independent variables: DesignX and DesignY
- Dependent variables: Time taken to make choice
Controlled Experiment: Q3 Can observed results be generalised to the world outside the lab? (External Validity)
- Overcoming External Validity:
- Try to recruit truly representative participants
- Self reported expertise in some domain e.g. novice, expert
- Frequency of use e.g. no. of web visits per day
- Amount of experience with something relevant
- Demographics e.g. age
Activities e.g. use of particular features
Controlled Experiment: Q4 Are observed results actually caused by the independent variables? (Internal validity)
- Overcoming Internal Validity:
- Hold variables constant
- Conduct all user tests:
- in the same room
- with the same lighting
- with the same computer
- the same mouse and keyboard
- same training
Controlled Experiment: Q5 Will consistent results be obtained by repeating the experiment? (Reliability)
Which is better:
- Between subject design:
- Results compared between diff groups
- Suffers from reliability - differences may be due to differences between users
- Within subject design:
- Results compared for each user
- Suffers from reliability problem due to ordering effects
Controlled Experiment: Q5 Will consistent results be obtained by repeating the experiment? (Reliability) 2
Ordering effects:
- the order in which different levels independent variables are applied: people will learn and people get tired or bored
- overcome ordering effects by randomizing the order of the task and counterbalancing the ordering
