Chapter 4 Flashcards Pt.2 - Threats to Internal & External Validity & Measurement and Sampling in Quantitative Research

Question 1

Internal vs External Validity?

Answer 1

Internal being whether the researcher has accounted for everything in the research such that the only thing influencing the outcome is the independent variables

External being that the researcher has made the study so that it can be manipulated or is generalizable; it can be used in other studies and other populations

Internal validity allows for the researcher to believe that there is a cause and effect relationship between the independent and dependent variable and no extraneous variables manipulated the outcome

Question 2

Logical validity is?

Answer 2

The test should appear correct in that the measurements used, the data collected and analyzed should end with the correct outcome

Question 3

Content Validity is?

Answer 3

Do all the contents of the measure of the study cover everything that is necessary to cover; rationale for each test item

Question 4

What is construct validity?

Answer 4

Does the test measure the idea/concept accurately

Question 5

What is predictive validity?

Answer 5

Do test scores predict a future result
i.e. Do MCAT scores predict whether or not you will be a good physician

Question 6

What is concurrent validity?

Answer 6

Compare “established” test to a new one

Question 7

What are some threats to internal validity?

Answer 7

1. Local History
2. Pre-testing
3. Maturation
4. Instrumentation
5. Selection Bias
6. Statistical Regression

First three are out of the researchers control

Question 8

How is local history an internal validity threat?

Answer 8

Anything that can happen outside the study; studies done inside the lab, the things people do outside the lab are not studied

Car accident for example can have a mental and physical impact on the person participating in the study

Question 9

How is Pre-Testing an internal validity threat?

Answer 9

Learning occurs everytime a participant is tested; the more they are tested the better they will do

Question 10

How is maturation an internal validity threat?

Answer 10

The longer you follow people up, they will change their views on certain things and that can have an impact on the result of the study

Question 11

How is instrumentation an internal validity threat?

Answer 11

Should use the same instruments for a research design (i.e. collecting blood with the same tools and in the same way) as using different instruments can get different results and we don’t need variability

Question 12

What is selection bias?

Answer 12

When selecting people without randomized assignment, likelihood of people being biased on some characteristics

Question 13

How is statistical regression a threat to internal validity?

Answer 13

People who have extreme scores (either high or low) will tend to regress slowly to the mean average and this is unwanted in smaller sample sized studies

Question 14

What is the John Henry Effect and the Halo Effect?

Answer 14

John Henry is the control group trying harder to influence the outcome of the study but that’s only if they find out they are the control group

Halo Effect is the initial rating may influence the future ratings

Question 15

What is Population Validity (External validity) & Ecological Validity (External Validity)

Answer 15

Population takes the data from one study and generalizes it to another similar study

Ecological is taking the experimental conditions and using it for another study to find the same effect that first study showed

Question 16

What are overgeneralization and expectancy as threats to external validity?

Answer 16

Overgeneralization is looking at sample from the study and relating it back to a broader sample to see if it is too generalized (i.e. thinking that one blood test means that a young person has a good blood pressure)

Expectancy is (placebo effect) - participants think the treatment is supposed to change them, so they respond with a change

Question 17

What is a post hoc error?

Answer 17

Assuming an effect occured because of the cause (false assumption of a cause and effect relationship)

I.e. Having negative events occur after taking the covid vaccine does not mean the vaccine had somethign to do with it

This is an external validity threat because it is a generalization

Question 18

What is measurement in quantitative data? What types of data are used for measurement in quantitative data?

Answer 18

The type of measures used, results in data that is numerical and quantifiable

1. Discrete Data - Looking at groups that can be counted; there are gaps in the values where people can fall under
2. Continuous Data - Looking at people who report within a range of values

Continous being the weight of a mouse (range of values) & Discrete being number of different exercises in a program (certain # the person will correspond to)

Question 19

What are Nominal and Ordinal Data

Answer 19

Fall under discrete data

• Nominal - Classifying people into groups; assignment of numbers for classification purposes like eye colour, or blood type (Can only be in one group)
• Ordinal - Classification according to order or magnitude; finishing in a race as first, second or third or even classifying people in weight categories

Question 20

What are Interval and Ratio Data

Answer 20

Falls under Continous Data
* Interval - Classification using ordered continuum with equal intervals/equal distance from one another (i.e. Difference between 10 degrees celsius and 11 is 1 and that applies to 84 to 85 degrees celsius)
* Ratio - Classification using an ordered continuum with equal intervals and a true zero

Question 21

Data method collection in quantitative studies include?

Answer 21

1. Observation (aka watch) - Coding for different things that happen as you watch them do stuff
2. Measurement (aka assess)
3. Questioning (aka “ask”)

Coding is categorizing non-numerical collected data into groups and assign numerical codes to these groups

i.e. Watching children’s behaviour and making a number for how many times a kid is flailing their arms around. You code that data by putting it into a group

Question 22

What are direct, indirect, and participant observation techniques for data collection methods?

Answer 22

• Direct - Participants are aware of the researcher’s presence and they are collecting data for the quantitative study which causes the person to act different at times
• Indirect - Film or videotaping; recording a parent and kid getting a shot and then watching the tape after to code for how the kid responded to getting pricked or how the parent was feeling
• Participant - Researcher is part of the research activity; active participant and often happens in qualitative study

Question 23

What are physical, cognitive and affective measures when it comes to “Measuring techniques” for quantitative study designs?

Answer 23

• Physical measures are simply taking someone’s height and weight or their obesity level based on BMI
• Cognitive measures would be neuropsychological exams (i.e. dementia exams)
• Affective measures are asking people their emotions, motivations, and attitudes to something

Question 24

Questioning techniques of quantitative study designs include?

Answer 24

• Structured/unstructed Questionnaires
• Checklist
• Structured/Unstructed Interviews
• Delphi Techniques = consensus view of subject experts

Question 25

What are probability sampling and non-probability sampling?

Answer 25

• Probability sampling: Likelihood of different units in the population having equal probabilities of being chosen
  (Includes: random selection, stratified random selection)
• Non-Probability Sampling: Any method not using random selection (I.e. Convenience or purposive sampling methods)

Question 26

What is a simple random sample vs a stratified random sampling?

Answer 26

Both are Probability sampling

• Simple random sampling is likelier for everyone to have a chance of being selected as a sample
• Stratified is more or dividing and grouping on a characteristic before random selection takes place; so picking people of certain characteristics that need to be represented in the sample

Stratified: Selecting people that fall under a group (i.e. Selecting people who play at least 2 intramural sports or selecting people who are only the age of 19)

Question 27

What are Systematic vs. Cluster Sampling?

Answer 27

• Systematic is a process where lists or inventories are used to select every Nth person
• Cluster sampling is pulling in within groups of people who share physical or social characteristics or environments

Cluster is comparing a cluster of people from two different settings to one another

Question 28

What is convenience sampling?

Answer 28

Drawing sample from familiar groups of people (i.e. Psychology programs offering course credits for participation in research studies)

Question 29

What is purposeful sampling?

Answer 29

Identifying people who represent a characteristic of interest

• Snowball sampling - identify one person with the characteristic of interest and have that person connect with others in their network
• Quota sampling - Identifying certain number or representation needed for the study then sampling up to that number
• Expert sampling - Getting people who are known experts or who have expertise in that area of interest