Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PSYC1111 > Reliability and Validity > Flashcards

Reliability and Validity Flashcards

1
Q

What is reliability

A

Reliability refers to the consistency/repeatability of results of a measurement. “How” reliable something is relative and depends on the situation.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Types of reliability

A
  • Observers: Inter-Observer reliability
  • Observations: Internal (Split-half) reliability
  • Occasions: Test-retest reliability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is inter-observer reliability?

A

Inter-observer reliability is the degree to which observers agree upon an observation or judgement
• Can be frequency or categorical judgement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How is inter-observer reliability tested?

A

Measure by looking at shared relationships between observers (i.e. correlation). Cohen’s kappa, Pearson’s correlation coefficient (r), etc. depending on whether continuous or discrete

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is internal reliability?

A

Internal reliability is the degree to which specific items/observations in a multiple item measure behave the same way. I.e. are they measuring the same thing?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

How is internal reliability tested?

A

Tested by dividing test into 2 halves, then looking at correlation between them. If it has high internal reliability, an individual’s performance on the first half should correlated with the second half.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What is test-retest reliability?

A

Test-retest reliability is the extent to which scores on a test/measure remain stable over time.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is validity?

A

Validity refers to how well a measure or an

operationalised variable corresponds to what it is supposed to measure/represent.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Types of validity

A
  • Internal validity
  • External validity
    • Population validity
    • Ecological validity
  • Construct validity
  • Content validity
  • Criterion validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Internal validity

A

• How convincing is the evidence for causality in a study/series of experiments?
• i.e. how strong is the inference that the independent
variable and the dependent variable are causally
related?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

J.S. Mill: 3 requirements to establish causality

A
  • Covariation
  • Temporal Sequence
  • Eliminating confounds (rival explanations/hypotheses)
    • Third-variable problem
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Equifinality

A

Another threat to internal validity. Most things have multiple causes .

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

External validity

A 
How well does a causal relationship
hold across different people, settings,
treatment variables, measurements
and time. 
Two types: Ecological and population validity.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Population validity

A
  • Making cross-cultural inferences from Western, Educated, Industrialized, Rich, Democratic samples?
  • Differences in tasks ranging from motivation, reasoning and even visual perception
    • Muller-Lyer Illusion: Americans vs. the San people of the Kalahari
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Ecological Validity

A

How well do results of laboratory experiments generalise to real-life settings?
• E.g. aggression studies in the lab vs. in real life
• Bandura (1961, 1963): Bobo doll experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Construct validity

A

How well do your operationalized variables
(independent and/or dependent) represent the
hypothetical or abstract variables of interest

17
Q

Content validity

A

Degree to which the items or tasks adequately sample the
target domain
• i.e. how well does a measure/task represent all the facets of a construct

18
Q

Criterion validity

A

To what extent can a procedure be used to infer or predict some criterion (outcome)
Two types: concurrent and predictive

19
Q

Concurrent validity

A

A type of criterion validity - to what extent can a procedure be used to infer some criterion

20
Q

Predictive validity

A

A type of criterion validity - to what extent can a procedure be used to predict some criterion

21
Q

When do person confounds occur?

A

Person confounds occur when a variable seems to cause something because people who are high or low on this variable also happen to be high or low on some individual difference variable (e.g. demographics characteristic) that is associated with the outcome variable of interest.

22
Q

When do operational confounds occur?

A

Operational confounds occur when a measure designed to assess a specific construct such as depression, memory, or foot size inadvertently measures something else as well.

23
Q

What is a confound?

A

A threat to internal validity, that undermines a causal explanation

24
Q

What is an artifact?

A

A threat to external validity, a by-product of testing procedure or sample that biases all results.
Unlike confounds, artifacts stay constant and are present in all groups

25
Q

What are some artifacts?

A

Hawthorne effect, history effect, and selection bias/non-response bias

26
Q

What is the Hawthorne effect?

A

That mere act of measurement changes the nature of something

- a form of participant reaction bias

27
Q

Advantages to Surveys?

A

Quick and efficient
Large samples
Obtain public opinion almost immediately
Easy to use

28
Q

Limitations of surveys?

A 
Non-random sampling
Haphazard samples (availability and volunteer bias)
Question wording effects
29
Q

Advantages of naturalistic observation?

A

Allows the study of issues not amenable to experimentation

Useful in the initial stages of investigation

30
Q

Limitations of naturalistic observation?

A

Low internal validity
Time consuming
Hawthorne effect - threat to external validity

31
Q

Advantages of participant observation?

A

It can be used in situations that otherwise might

be closed to scientific investigation

32
Q

Limitations of participant observation?

A

The dual role of the researcher maximizes the chances for the observer to lose objectivity and allow personal biases to enter into the description

33
Q

Advantages of longitudinal studies?

A 
• Genuine changes and stability of some
characteristics observed
• Major points of change observed
• Temporal sequence
• Minimise age-cohort effects
34
Q

Disadvantages of longitudinal studies?

A
  • Time consuming and expensive

* Participant attrition – threat to validity

35
Q

Advantages of cross-sectional studies?

A

• Relatively inexpensive and less time consuming • Low attrition rate

36
Q

Disadvantages of cross-sectional studies?

A
  • Cannot observe changes in individuals
  • Insensitive to abrupt changes
  • Age-Cohort effects
37
Q

What are age-cohort effects in cross-sectional studies?

A

Cohort differences are confounded with age differences.

For example, measuring computer skills at different age points - Different exposure to computers

38
Q

What is experimental mortality also known as? What’s an example of a situation in which it is a problem?

A

Experimental mortality is also known as heterogenous attrition.

For example, in study on antipsychotics and schizophrenia, experimental group receive antipsychotics which produce side effects and also reduce potentially pleasurable symptoms such as schizophrenia. Hence, more drop out of the experimental group.
This becomes a threat to internal validity.

39
Q

Individual difference confounds include…

A

Experimental mortality

Participant Reaction Bias (participant expectancies, participant reactance, evaluation apprehension)

PSYC1111 (6 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions