Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Human Computer Interaction > 10: Experimental Design > Flashcards

10: Experimental Design Flashcards

1
Q

Stages in usability experiment

A
  1. identify objectives
  2. formulate hypothesis
  3. choose variables
  4. choose experimental design
  5. choose the tasks
  6. recruit participants
  7. run the experiments
  8. perform statistical test on data
  9. analyze and interpret results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q
  1. identify objectives
A

determine objective, goal of the experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q
  1. formulate hypothesis
A
  1. make a prediction about the outcome of the experiment
  2. must be testable
  3. should be presented in terms of independent, dependant variable
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

what makes a bad hypothesis

A
  1. vague
    - hypothesis does not predict outcome
  2. complex
    - hypothesis able to explain any results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q
  1. choosing variables
A
  1. independent
    - characteristics that is changed to produce different condition
    - manipulated by researcher
  2. dependent
    - used to test hypothesis
    - variable that is being measured
    - outcome is driven by independent variable
  3. control variables
    - variable to be held constant to ensure validity
    - depends on experimental design
  4. confounding variable
    - variable that can alter outcome of experiment
    - not dependent
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q
  1. choose experiment design
A
  1. specify what independent variables needs to be manipulated
  2. which dependent variables to measure
  3. ensure results are due to manipulation of independent variable only
  4. ensure that some variables are controlled to ensure validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

main types of experimental design

A

Between subjects (independent samples)

  • participants are randomly grouped
  • each group take part in only one condition
  • compare one group with another

within subjects (repeated measures)

  • same group of subjects take part in more than one condition
  • compare each participant against himself
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

between subjects strengths

A
  1. no learning effect
    - lower chance of participant having carry over effect
    - lower possibility of gaining practice and experience
    - lower chance of skewed results
  2. less fatigue
    - only subjected to one experiment
  3. multiple variables can be tested simultaneously
    - divide into groups
    - each group test different condition
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

between subject limitations

A
  1. need many participants
  2. individual variability
    - difference in abilities and expertise
    - what if bad group of participants
  3. assignment bias
    - control and experimental group are from different population
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

within subject strengths

A
  1. need fewer participants
  2. less chance of variation between participants
  3. compares everyone with themselves(lower confounding variables)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

within subject limitations

A
  1. carry over effect
    - participation in one condition may affect performance in another
    - confounding variables that vary with independent variables
  2. fatigue effect
    - tired
    - negatively affect result
    - give break
  3. practice effect
    - get used to experiment
    - positively affect result
  4. order effect
    - outcome is due t order of experiment
    - randomise order of experiment(counter balancing)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Matched experimental design

A
  1. participants matched in pairs
  2. pairs can be formed on gender, expertise, personal relationship
  3. each pair allocated one condition
  4. matching criteria may affect result
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

ladder of experimental validity

A
  1. content validity
    - does result reflect variable of interest
    - is the result interesting
  2. construct validity
    - does the result align with the theoretical concepts
  3. internal validity
    - does the hypothesis hold/ valid?
  4. external validity
    - does outcome hold for other data sets or sample users
    - can it be generalised
  5. ecological validity
    - is the test meant for type of user? eg. developed for bling users tested on normal users
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q
  1. choose the tasks
A

experimental tasks should be:

  1. constrained to test just the thing of interest
  2. need to collect data to analyse
    - more data = less influence of outliers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

writing experiment software

A
  1. finalise design before developing
  2. decide on best platform
    - desktop
    - mobile
  3. test that it works on yourself
  4. pilot test on others
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q
  1. run the experiment
A
  1. pilot the experiment
    - test prototype
    - fix problem if it occurs
    - every participant run through should be the same
  2. noise level
    - information given
    - task and training
  3. explain what experiment will involve
    - do not explain hypothesis
  4. give standardise information
  5. get informed consent
    - explain what experiment does
    - participants sign to give consent
  6. provide necessary training
    - practice tasks to understand how it works
17
Q

statistical test

A

compare data sets gathered after experiment and find statistical differences

Null hypothesis
H0: changes in behaviour due to chance

Alternate hypothesis
H1: hypothesis you are trying to demonstrate

if test results are significant, reject null hypothesis

18
Q

choosing statistical test depends on:

A
  1. type of comparison
    - difference
    - correlation
  2. type of data
    - nominal/ categorical
    - ordinal/ scale
    - interval
    - ratio
    - parametric
  3. number of conditions
    - 1 independent variable can have 2 or 3 conditions
19
Q

use parametric tests when

A
  1. data is interval/ ratio
  2. data is normally distributed
  3. data can be categorised by measure of central tendency

between subjects
- independent T test

within subjects
- paired T test

procedure

  • test statistic t calculated
  • significance value based on t calculated
  • descriptive statistic calculated
20
Q

use non parametric test when:

A
  1. data is ordinal/ nominal

between subjects
- Mann-whitney test

within subjects
- wilcoxon signed rank

procedure

  • difference between ranked position of scores in the 2 groups
  • test statistical U calculated
  • U mapped to critical values to obtain significance value P
21
Q

statistical significance

A
  1. produce value P
  2. P value indicates the likelihood that differences between conditions is due to chance
  3. lower P means less likely due to chance

minimum value = 5%
moderate value = 1%
high significance = 0.1%

if P<0.05, null hypothesis is rejected
if P>0.05, null hypothesis is not rejected
P>0.05 does not prove null hypothesis, not enough evidence or data may be noisy

22
Q

types of errors

A

type 1
incorrect rejection of true null hypothesis (false positive)

type 2
fail to reject a false null hypothesis (false negative)

23
Q

statistical power depends on:

A

significance value
sample size
effect size

24
Q
  1. analyse and interpret result
A

conclusion
hence we reject the __ hypothesis
explain reason of outcome

Human Computer Interaction (13 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions