Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Experimental Design & Analysis > Mid Tri Exam > Flashcards

Mid Tri Exam Flashcards

1
Q

Reliability

A

the consistency of repeatability of measures.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Validity

A

are we measuring what we are trying to measure.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Inter-rater or inter-observer reliability

A

Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon - agreement between the scores of two or more independent observers or judges.
Especially important when measures are subjective

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Test-retest reliability

A

Used to assess the consistency of a measure from one time to another.
The correlation between scores across two admin of the measure (take one test then take it again at a later time)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Parallel-forms reliability

A

Used to assess the consistency of the results of two tests constructed in the same way from the same content domain
- Split Half reliability (splitting a test into 2 halves. If its reliably the answers should be similar)
- Item-total correlation (comparing the mid tri to the overall mark might not be the best because people might suck at exams but good at lab reports)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Internal consistency reliability

A

Used to assess the consistency of results across items within a test
Cronbach’s alpha - the average correlation among all possible pairs of items (.80 or above)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Reliability puts a ceiling on validity

A

If reliability is .70 validity can only reach .70
It can be reliable without being valid
It can’t be valid without being reliable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Construct validity

A

measurement validity - A construct refers to a behaviour or process that we are interested in studying

Both measures and manipulations must be valid!
Manipulations can be
instructional
environmental
stooges

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

instructional manipulations

A

experimental conditions defined by what you tell participants

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

environmental manipulations

A

stage and event, present a stimulus, induce a state

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

stooges manipulations

A

use fake participants to alter experiment condition

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Convergent validity

A

Do scores on the measure correlate with scores on other similar measures related to the construct
Relates to the degree to which the measure converges on (is similar to) other constructs that it theoretically should be similar to

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Discriminant (divergent) validity

A

Do scores on the measure have low correlations with scores on other different measures that are unrelated to the construct
Relates to the degree to which the measure diverges from (is dissimilar to) other constructs that is should be not similar to

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Face validity

A

On its face value, thes the measure seem to be a good translation of the construct
Does it make sense?
Ask experts in the field

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Content validity

A

Does the measure assess the entire range of characteristics that are representative of the construct it is intending to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Criterion validity

A

concurrent - Do scores on the measure distinguish participants on other variables that we would expect to be related to it (depressives from non-depressives, criminals from non-criminals)
predictive - Are scores on the measure able to predict future outcomes (attitudes, behaviours, performance)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How to Correct manipulations

A

Reduce random error (replicate procedure)
Reduce experimenter bias
Reduce participant bias
Ensure manipulation has construct validity
Do a manipulation check - ask participants about various aspects

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

External validity

A

extent to which the results can be generalised to other relevant populations, settings or times
Studies have good external validity when results can be replicated
Ecological validity
Population generalisation
Environmental generalisation
Temporal generalisation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Ecological validity

A

The extent to which the results can be generalised to real-life settings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Population generalisation

A

Applying the results from an experiment to a group of participants that is different and more encompassing than those used in the original experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Environmental generalisation

A

Applying the results from an experiment to a situation or environment that differs from that of the original experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Temporal generalisation

A

Applying the results from an experiment to a time that is different from the time when the original experiment was conducted

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Internal Validity

A

ability to draw conclusions about causal relationships from the results of a study
The extent to which we can say that any effects on the DV were caused by the IV
The elimination of alternative explanations for the observed relationships
Inferences of cause and effect require 3 elements for strong internal validity
Co-variation
Temporal precedence
Elimination of alternative explanations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Threats to internal validity

A

Selection bias
Maturation
Statistical Regression
Mortality
History
Testing
Practice Effect
Instrumentation
observer reactivity
Social desirability

Controlling these threats
Randomly allocate participants
Treat all conditions equally except for intended IV manipulations
Use appropriate control conditions
Use double blind studies where possible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Experimenter bias

A

errors in a research study due to the notions or beliefs of the experimenter

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Selection bias

A

A threat to internal validity that can occur if participants are chosen in such a way that groups are not equal before the the experiment
Differences after the experiment may reflect differences that existed before the experiment began plus a treatment effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Maturation

A

changes in participants during and experiment or between the DV due to time (age, cognitive development)
Permanent - (age, biological growth, cognitive development)
Temporary - (fatigue boredom, hunger)
Most common naturally occurring developmental processes (children)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Statistical Regression

A

regression towards the mean
Participants with extreme scores on the first measurement of the dv tend to have scores closer to the mean on the second measurement
Subsequent scores are still likely to be extreme in the same direction but not as extreme
When you have extreme scores, it is difficult to maintain that degree of extremity over repeated measures
If participants are selected on the basis of extremes scores- regression to the mean is always going to be a possible explanation for higher or lower scores on a repeated test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Mortality

A

attrition (premature dropouts and differential dropout across experimental conditions)
Relates to premature dropouts
Differential dropout rates occur
If the intervention is unpleasant

30
Q

History

A

outside events that may influence participants in the experiment in a repeated measures design
History can include major events like terrorist attacks or smaller personal changes like joining a gym, changing jobs
If they are relevant to the study in some way these events can influence the DV score

31
Q

Prior Testing

A

Prior measurements of the DV may influence the results
Measuring the DV can cause a change in the DV
Participant becomes aware of the study aims

32
Q

Practice Effect

A

When a beneficial effect on a DV measurement is caused by previous

33
Q

Instrumentation

A

changes due to the measuring device

34
Q

observer reactivity

A

changes to behaviour when being watched

35
Q

Social desirability

A

changes from people to present in the best possible light

36
Q

homogeneity of variance been met and what does this mean for the analyses?

A

Homogeneity of variance has been met so analyses should continue.

37
Q

one-way ANOVA

A

comparing several means in situations where we want to compare more than two conditions

38
Q

Assumptions of ANOVA

A

Levels of measurement
Random sampling
Independence of observations
Normal distribution
Homogeneity of variance

39
Q

Levels of measurement

A

Dependent variable must be measured at the interval or ratio level

40
Q

Random sampling

A

Participants must be obtained using a random sample from the population of interest

41
Q

Independence of observations

A

The observations that make up the data must be independent of one another (one person can’t do multiple everyone has to do the same)
Violation of this assumption is very serious as it dramatically increases the type 1 error rate

42
Q

Normal distribution

A

The populations from which the sample are taken is assumed to be normally distributed
Need to check this for each group separately in one-way ANOVA

43
Q

Homogeneity of variance

A

Samples are obtained from populations of equal variances
ANOVA is fairly robust to this violations - provided the size of groups are reasonably similar

44
Q

F ratio is reported with how many decimal points

A

2

45
Q

Family wise error

A

the probability of at least 1 false positive when multiple comparisons are being tested.

46
Q

Types of multiple comparisons

A

Planned comparisons (contrasts) prior to study
Post-hoc analyses post study

47
Q

Planned comparisons (contrasts)

A

prior to study
A priori
Break down variance into component parts
Test specific hypothesis

rules
Once a group has been singled out - it cannot be used in another contrast
Each contrast must only compare 2 chunks of variation
There should always be 1 less comparison than the number of groups (number of contrasts = k-1)

(If the rules are met we are doing)
Orthogonal contrasts - compare unique chunks of variance

if not
Non Orthogonal contrasts
Overlap or use the same chunks of variance in multiple comparisons
Require careful interpretation
Lead to increased type 1 error rate

48
Q

Post-hoc analyses

A

post study
Compare all groups using stricter alpha values (this reduces type 1 error rate)

49
Q

Polynomial contrasts

A

Only used when the IV is ordinal
Linear
Quadratic
Cubic
Quartic

50
Q

Standard contrasts

A

Orthogonal: Helmert and difference
non-orthogonal L deviation, simple, repeated

51
Q

Helmert

A

Compare each category to the mean of subsequent categories (based on the odeer they are coded in SPSS) which might be alphabetical
With 3 groups
1 vs 2 & 3
2 vs 3
With 4 groups
1 vs 2 & 3 & 4
2 vs 3 & 4
3 vs 4

52
Q

Difference planned contrast

A

Compare each category to the mean of the previous categories
With 3 groups
3 vs 2 & 1
2 vs 1
With 4 groups
4 vs 3 & 2 & 1
3 vs 2 & 1
2 vs 1

53
Q

Post-hoc tests

A

Involve comparing all possible difference between pairs of means
Good approach with exploratory research or where there are no predefined specific hypothesis

Simplest post-hoc test is bonferroni

54
Q

Bonferroni a(alpha) =

A

A(alpha) / number of tests

55
Q

Tukey’s HSD

A

Honestly significant difference
The cumulative probability of a type 1 error never exceeds the specified level of significance (p<.05)
Supplies a single critical value (HSD) for evaluating the significance of each pair of means

The critical value (HSD) increases (with each additional group mean)
It becomes more difficult to reject the null hypothesis as a greater number of group means are compared
If the absolute (obtained) difference between two means exceeds the critical value for HSD, the null hypothesis for that pair of means can be rejected

56
Q

Statistical power is difficult to calculate but important to know

A

Hypothesis testing
Estimating statistical power
Factors that influence power

57
Q

Effect Size formulas

A

Eta squared
Omega Squared
r
Cohen’s d

58
Q

Statistical power means

A

what is the probability that study will detect an effect when there is an effect there to be detected

59
Q

calculating Statistical power

A

Power = 1 - Beta

60
Q

Factors affecting statistical power

A

Alpha level
Error variance
Sample size
Effect size

61
Q

Alpha level

A

If beta is smaller the statistical power will be larger

62
Q

Error variance

A

Same means lower error variance
Decreasing the amount of variability increases the chance we will find an effect if there is one

63
Q

Sample size

A

Works similar to error variance
Increasing the amount of samples will decrease the error variance
Testing more people we are able to better describe a distribution

30 people we hit central limit theorem where going over it might decrease. But 30 is normally a good line to understand a population

64
Q

Effect size

A

The magnitude of difference between our samples
Larger effect size -means further apart
Power will increase as we increase effect size

65
Q

Effect size measurements

A

Main effect (ANOVA)
Eta Squared
Omega Squared

Multiple comparisons (planned contrast or post-hoc)
r
Cohen’s d

66
Q

Eta squared

A

Used for main effect

n2 = ssbetween / sstotal

67
Q

Omega Squared

A

Used for main effect
Most accurate measure of effect size for main effect

w2 = ssB-(dfb*MSw) / sst+MSw

68
Q

Effect sizes for planned contrasts

A

r
Used for follow-up tests
Particularly useful for planned contrasts

r= squareroot : t2 / t2+df

69
Q

Effect sizes for Post-hoc tests

A

Cohen’s d
Used for follow up tests
Can be used for tukey’s post-hoc tests

Spooled = squareroot: (n1-1)s12+(n2-1)s22 / n1+n2

d = X(mean)1 - X(mean)2 / Spooled

70
Q

Calculating required sample size

A

Using power and effect size to calculate sample size
Before running an experiment we want to ensure that if there is and effect present, that we will observe it (power)
Part of that will rely on testing sufficient numbers of participants (see effect of sample size on power)
We can use effect size and desired power to estimate how many people we will need to test

Experimental Design & Analysis (2 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions