Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Research Skills > Week 4 - Reliability > Flashcards

Week 4 - Reliability Flashcards

1
Q

What does quantitative research necessitates?

A

Measurement, whether that be physical or abstract things

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the assumption with quantitative research?

A

Everything we want to measure has a true value, or a true score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the goal or quantitative research?

A

Measurement will give us a pure score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the reality of quantitative research?

A

Measurement always includes some amount of “random noise”.

Measured score = true score + noise

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What does reliable mean in a quantitative study?

A

Tools used for measurement must be consistent and accurate (precise and unbiased)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What does valid mean in quantitative research?

A

Descriptions, relations and explanations (causation) must be truthful

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does reliability equal?

A

Consistency

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the three broad types of consistency in research?

A

Consistency:

Over time (test-retest reliability)

Across items (internal consistency)

Consistency across researchers (inter- rater reliability)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is consistency measured by?

A

Variability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is an inverse relationship?

A

Research should indicate an inverse relationship if the two variables are related. Therefore:

High variability = low reliability
Low variability = high reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

A reliable measure is also a what?

A

Valid measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Can any measure that is not consistent provide trustworthy evidence?

A

No

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Is a measure that is reliable always valid?

A

Not necessarily no.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are the two reasons for which variability occurs?

A
  1. Causal relationship between an IV and DV variable (effect)
    • when a IV changes, a DV also changes
  2. Variability due to random factors (noise)
    • unknown (possibly unknowable) factors affect DV
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Careful research designs are needed to separate effect from noise

A

Noise can be reduced, but not eliminated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is variability due to noise called?

A

Error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What does error mean in research design and statistics error?

A

Means variability whose cause is unknown, which does not mean MISTAKE.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What might short term change in results indicate?

A

Problems with reliability (and therefore validity).

Caution: some measures are inherently variable (e.g reaction time)

Might need multiple measurements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What might long term change (with short term stability) indicate in research?

A

More likely to reflect a real effect. More likely to reflect real changes in variable, not merely error.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are two things body temperature can measure with an illness.

A

To indicate development of illness following exposure

Track progression of illness during sickness.

But the measurement device must be reliable.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is test retest reliability?

A

Relating a measure on two occasions then calculating the correlation between two occasions. High test retest reliability results in a strong positive correlation.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is internal consistency?

A

Consistency of responses across test items

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What do assessments of psychological, behavioural and health related constructs often use?

A

Questionnaires.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Why do researchers use different phrasing’s (positive and negative phrasing’s) to assess items against the same construct?

A

To capture the construct fully. Therefore if participants are consistent in their responding to the questions, the different items should be correlated.

25
Q

What is reverse scored?

A

Negatively phrased items in a questionnaire need to be negatively scored to measure against positive results.

26
Q

What is an example of a questionnaire which needs reverse scoring?

A

Rosenbergs self esteem scale (1989) uses a split half reliability (every second question is to be reverse scored)

27
Q

What is Cronbach’s alpha?

A

Related to split half reliability. Calculates all possible split half correlations for a set. Rosenberg self esteem scale has 252 unique split half correlations.

28
Q

Does Cronbach a require every individual correlation to be separately calculated?

A

No, but the mean of the set of correlations is equivalent to it.

29
Q

What is parallel forms reliability?

A

Two versions of test for the same test. Scores for the two versions should be highly and positively correlated.

30
Q

What is interrater reliability?

A

Many assessments rely on judgements by different raters/researchers. Important that these scores are consistent.

31
Q

How is interrater reliability assessed?

A

Using a statistics called Cohen’s kappa (k) which is analogous to Cronbach’s a.

32
Q

What is validity?

A

The extent to which measured represent what they are intended to represent (loosely the truthfulness of a measure)

33
Q

What are the three major forms of validity?

A

Construct validity

Internal validity

External validity

34
Q

What is construct validity?

A

Extent to which manipulated variables (IVs) and measured variables (DV) represent what they are interned to.

35
Q

What are some examples of construct validity?

A

Are IQ tests a measure of intelligence?
Can intelligence be assessed by measuring head circumference?
Can reaction time be a measure of decision making speed?

36
Q

What is face validity?

A

Do IVs and DVs appears to reflect or measure what they’re intended to measure?

37
Q

Is face validity an important form or validity for valid research?

A

No. But research is often unfairly challenged when it lacks face validity. Reliance on face validity might explain some unusual medicines.

38
Q

What is internal validity?

A

Extent to which causal statements about the relations among variables can be made.

39
Q

What is internal validity dependant on?

A

Study design.

Designs that effectively controls extraneous variables have high internal validity.
Designs that poor control extraneous variables have low internal validity.

40
Q

Campbell (1957) identified what seven threats to internal validity?

A 
History
Maturation 
Instrumentation decay 
Statistical regression
Selection
Testing 
Mortality
41
Q

How does history affect internal validity?

A

Measurements that occur over time are susceptible to effects from factors other than the IV.
Risk increases as time between measures do. (Longitudinal studies particularly at risk)

42
Q

How does maturation affect internal validity?

A

Rather than change in the DV being caused by the IVs, change may be a result of natural maturation processes age-related biological change is one example of maturation (needs to be considered for developmental psychology)

43
Q

What is regression to the mean and how does this affect internal validity?

A

When groups are formed by sale find extreme scores at pretest, those scores tend to become less extreme at post test.

44
Q

Give an example of regression to the mean affecting internal validity.

A

People with shortest and longest reaction time are given fake medication. After it “works”, longest groups time gets quicker because they’re told it’s a stimulant. Shortest groups time gets longer because they’re told it’s a relaxant.

45
Q

What is instrumentation and how does it affect internal validity?

A

Changing the measurement instrument during a study can affect measurement or the DV.
Instrument is a broad term and mean any of the following:
- mechanical devises
-questionnaires
-researchers

46
Q

What is selection and how does it affect internal validity?

A

When groups are not selected through random assignment, there is potential bias in group formation. Bias could be introduced unintentionally.

47
Q

How does mortality affect internal validity and what is it?

A

In studies with more then one group, any non random factor that results in greater drop out (attrition) rage in one group than another results in differential attrition.

48
Q

What is tearing and how does it affect internal validity?

A

Some studies require a pretest (baseline measure or for task familiarisation)
After that participants are tested again using same test. Pre test itself might influence the post test outcomes.

49
Q

What is reactivity in research?

A

Experiments are intended to measure behaviour, but may also produce or alter behaviours.

Is your behaviour at a part the same when you know you’re being filmed?

50
Q

What are the different type of subject roles when it comes to participants in research?

A

Good subject roles
Faithful subject roles
Negativistic subject roles
Apprehensive subject roles

51
Q

What is external validity?

A

The extent to which the findings of a study can be generalised beyond the study itself. Externally invalid research occurs when conclusions are limited to original setting.

52
Q

External validity can only be confirmed by what?

A

Replication.

53
Q

Why might external validity not necessarily be too important?

A

Because students are humans. So studying only students will translate to other settings also. Sample size does not necessarily affect external validity.

54
Q

What is convergent validity?

A

Tests whether constructs or measures that should be related are related.

55
Q

What is discriminant validity?

A

Tests whether constructs/measures that should be unrelated are unrelated.

56
Q

What is predictive validity?

A

Tests whether current performance on some measure predicts future performance on a different

57
Q

What is concurrent validity?

A

Tests whether performance on a validated tests correlated with performance on a new test will

58
Q

Construct validity can be tested with what two outlets?

A

Convergent and discriminant (sometimes called divergent) validity

Example: affection towards ones partner. Find factors that do and do not relate to affection.
Convergent validity would be demonstrated by strong, positive correlations with things within affection factors.
Discriminant validity would be demonstrated by weak correlations among assumed unrelated measures (physical touches and pairs of shoes)

Research Skills (13 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions