Experimental design

Question 1

What are the 6 issues when creating new interaction techniques?

Answer 1

1. Social acceptability
2. Walk-up use (at the Start)
3. Learning curve (over time)
4. Intrusiveness
5. Context of use
6. Robustness and Error handling

Question 2

What is social acceptability when referring to the issues when creating new interaction techniques?

Answer 2

Can a user use a particular interaction technique in society?

Question 3

What is walk-up use when referring to the issues when creating new interaction techniques?

Answer 3

Can the interaction technique be used straight away with minimal setup

Question 4

What is Learning curve when referring to the issues when creating new interaction techniques?

Answer 4

How long does it take for users to reach the crossover point?

Question 5

What’s the crossover point?

Answer 5

Point in time when new interaction technique performs better than what users are familiar with

Question 6

What is intrusiveness when referring to the issues when creating new interaction techniques?

Answer 6

Does the user have to wear a cumbersome piece of apparatus or markers to use the system?

Question 7

What is context of use when referring to the issues when creating new interaction techniques?

Answer 7

How does the use context affect user’s interaction with the technique?

Question 8

What is robustness and error handling when referring to the issues when creating new interaction techniques?

Answer 8

Is the interaction technique robust with respect to noise? How are errors handled? Can the user easily recover from errors?

Question 9

How are new interaction techniques evaluated?

Answer 9

In controlled experiment in the lab.

Question 10

Why are controlled experiments used?

Answer 10

Have baseline to compare against

Question 11

What is the trade-off between internal and external validity?

Answer 11

Reliable experimental outcomes VS being able to generalise the results to actual use contexts.

Question 12

What is the difficulty when evaluating with an experiment?

Answer 12

Asking the right questions.

Question 13

What’s the purpose of a controlled experiment in HCI?

Answer 13

To experimentally determine if we can find a measurable difference in dependent variables between levels of independent variables.

Question 14

What are threats to the validity of conclusions of an experiment?

Answer 14

• Reproducibility
• Study heterogeneity
  Can it be compared?
• Internal validity
  Are there no confounding factors you do not control that influence the result.
• External validity
  Do results generalise to the world outside of the experiment?

Question 15

What’s included in the Method section of an experimental design

Answer 15

Setup
Participants
Apparatus
Material
Procedure

Key: Must be written with enough detail that someone else can replicate the experiment.

Question 16

What is between subjects design?

Answer 16

Each participant exposed to one condition

Question 17

What are the pros of between subject design?

Answer 17

No risk of skill-transfer

Question 18

What are the cons of between subject design?

Answer 18

Variance isn’t controlled within the participant (A better at keyboard use than B)

Each participant source of error.

Question 19

When can within-subject design never be used and why?

Answer 19

When there’s a risk of asymmetrical skills transfer. This causes more skill transfer one way than the other, and therefore means it can’t be counterbalanced.

Question 20

What is within-subjects design

Answer 20

Each participant is exposed to all conditions

Question 21

What are the advantages of within-subjects design?

Answer 21

Variance is controlled within the participant

Question 22

What are the disadvantages of within-subjects design?

Answer 22

Must counterbalance to avoid skill transfer effects.

Risk of asymmetrical skill transfer.