Chapter 5 Flashcards

1
Q

Facts

A

It’s is not possible to calculate reliability, researchers can only estimate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Validity Definition

A

Wether the scale measures what it was intended to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Reliability to Validity Relationship

A
  • low reliability to low validity - high reliability to low validity - high reliability to high validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Face Validity

A

Examines how the test appears - the logical sense of the survey

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Criterion- Related Validity

A

Measure one topic in two different ways

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Construct Validity

A

measures a concept that is not actually observable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Content Validity

A

How well a test measures the specific content intended to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the four types of Validity?

A
  • Face Validity - Criterion - Related Validity - Construct Validity - Content Validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the nine threats to Internal Validity?

A
  1. History 2. Maturation 3. Testing 4. Instrumentation 5. Regression 6. Ceiling and floor effects 7. Attrition 8. Selection 9. Hawthorn effect
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Internal Validity: History

A

When an event happens during research that influences the behavior of participating individuals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Internal Validity: Maturation

A

The natural change that occurs over time with individuals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Internal Validity: Testing

A

Differences noted from pre-test to post test that can be attributed to students becoming familiar with the test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Internal Validity: Instrumentation

A

Measures changes in respondent performance which cannot be credited to the treatment or intervention

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Internal Validity: Regression

A

Some respondents performing well on pre-test and poorly on post-test Orr vice versa merely by chance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Reliability definition

A

Related to consistency or ability to repeat results

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Internal Validity: Instrumentation

A

Measures changes in respondent performance which cannot be credited to the treatment or intervention

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Internal Validity: Regression

A

Some respondents performing well on pretests and poorly on posttests or vice versa merely by chance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Internal Validity: Ceiling and floor effects

A
  1. Ceiling effect is when all participating individuals perform extremely well on a pretest and posttest 1. Floor effect occurs when individual performance starts out low and remains low
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Internal Validity: Attrition

A

Individuals lost from the study

20
Q

Internal Validity: Selection

A

When participating individuals are different at the onset of the study

21
Q

Internal Validity: Hawthorne effect

A

Workers at the Western Electric Company in Hawthorne, Illinois improved their performance when they know that they are being watched

22
Q

Generalizability

A

Generalizability is linked to independent variables

23
Q

Independent variables

A

variables that researchers manipulate and control

24
Q

Dependent variables

A

variables are fixed and not manipulated

25
Threats to External Validity
- Refers to the generalizability of research results - Repeating research in different populations is the best way to access generalizability
26
Seven Factors that influence generalizability
1. Population 2. Environment 3.Temporal / Sequential 4. Participants 5.Testing and treatment interaction 6.Reactive arrangements 7. Multiple treatment conditions
27
External Validity: Population
When population selection is so specific, treatment is matched to a specific sample and doesn’t apply to a wider population
28
External Validity: Environment
The change from a controlled environment to a less controlled environment vice versa
29
External Validity: Temporal / Sequential
Change of temperature could affect study
30
External Validity: Participants
- Animal to human links - Human to human links - Gender bias - Racial bias - Cultural and ethnocentric bias
31
External Validity: Testing and treatment interaction
If participants learn from the pretest, then they may be less likely to learn as much from treatment
32
External Validity: Reactive arrangements
If individuals change their behavior when observed (threat to internal validity), results are not generalizable to real world conditions (threat to external validity)
33
External Validity: Multiple treatment conditions
Multiple treatments may create an artificial setting that does not exist in the real world, so results are not generalizable
34
Relationship between Internal and External Validity
Internal validity is more critical than external Without internal validity, research is not testing what it reports to measure As the study inclusion criteria becomes more selective, the results become less generalizable
35
Random errors
- Occur by chance and are inconsistent across the respondents - Increase or decrease results in an unpredictable manner - Researchers have no control over the occurrence of random errors - Reduced through statistical methods by averaging scores over a larger sample size - Influences reliability
36
Systematic errors
- Consistent in the same direction (all results have the same error) - Introduce inaccuracy and bias into the measurement - Problematic to detect and eliminate - Not possible to reduce the effect of systematic errors through statistical methods - Influences validity - Occurs in three areas: Environment, Observation, Drift
37
Randomized Controlled Trial (RCT)
- The gold standard of research design - Participants are randomly assigned to either a treatment group or a non-treatment group - Participants in each group have similar characteristics - Allows researchers to draw conclusions with confidence
38
If Sample Size is too small?
When sample is too small, results are inconclusive and significant differences among groups are statistically harder to determine
39
If Sample Size is too large?
When sample is too large, cost, feasibility and time become problematic
40
Precision and Accuracy of the study Increase as...
as the sample size increases
41
Selection bias:
Specific individuals or groups are purposely omitted from research
42
Measurement bias
Occurs because of systematic errors in measurement
43
Intervention bias:
How intervention groups are treated differently than control groups if the researchers involved know which group is which
44
Pilot Testing
Involves conducting a preliminary test of data collection tools and procedures to identify and eliminate problems
45
Conduct a pilot study by categories:
- Sample of Respondents - Data Collection for Pilot Tests - Data Analysis - Outcome
46
Cronbach's alpha
Cronbach's alpha is a measure of internal consistency, that is, how closely related a set of items are as a group