Exam 2: Research Control - Internal and External Validitiy Flashcards
What is rigor?
rigorous adherence to rules, procedures, and techniques during research.
how do researchers ensure adequate rigor?
- understand rules and procedures
- understand approaches
- adhere to methods
- inform others about problems and limitations.
What are 4 issues of rigor in quantitative designs?
- quality of design
- quality of instrument
- rigor of procedure
- adequacy of sample
What are 3 issues of rigor in qualitative designs?
- depth and comprehension of data obtained.
- procedures used to engage with participants
- reflexivity and analysis of themes.
What does measurement mean in quantitative research?
translating what has been observed into numerical values.
what does validity mean in quantitative research?
- establishing the strength of a relationship between a measurement indicator and the underlying concept
- are we measuring what we say we’re measuring?
what does reliability mean in quantitative research?
- the consistency of the measures.
What are some criteria for choosing a good instrument? (3)
- is it widely used?
- does procedure fit the study?
- is there info. on reliability and validity of scores?
What are the 4 levels of measurement?
- Nominal
- ordinal
- interval
- ratio
What is nominal measurement?
- Categorical
- describes traits or characteristics
- no ranking of data.
- gender, ethnicity, religion
What is an ordinal measurement?
- Categorical
- data ranked from smallest to largest
- interval between data may not be equal
- rating scale: strong agree, agree, etc.
What is an Interval measurement?
- continuous
- equal intervals bt data categories
- date, temperature
What is a Ratio measurement
- continuous
- interval scale with an absolute zero
- weight, income
T/F: Categorical data can’t compute in addition, subtraction, multiplication, division.
True
Name an example of an interval instrument
- equal spacing bt measures
- purdue pegboard test
- focus on speed of performance
- does not start with zero point
name an example of a ratio instrument
- scale beings with absolute zero
- grip test: start at zero to grip strength
Reliability
-consistency of an instrument
- degree to which random error exists in instrument
O (observed score) = T (true score) + E (error score)
R = .80 or above is acceptable
Threats to reliability
- random error
- i.e. misreading questionnaire, observer interprets incorrectly, fatigue
Types of reliability (3)
- stability
- internal consistency
- equivalence
Stability
test-retest; consistency of repeated measures
internal consistency
test homogenity; uses split half consistency ( split test in half; evens/odds)
equivalence
agreement bt two measurement events; inter-rater, alternate forms, etc.
Validity
What is being measured true to the concept
What are two threats to validity?
- systematic bias
- non-random error
less systematic error = more validity
Internal validity
are we measuring what we want to measure/manipulate?
external validity
generalizability; can we generalize our study to larger population
What are the 3 types of validity?
- construct
- content
- criterion
Construct validity
how the instrument is designed; independence, happiness, etc
Content validity
how the assessment covers the relevant information
criterion validity
predictive validity; how it can predict the outcome.
Ways of strengthening internal validity designs (4)
- randomization
- homogenity
- matching characteristics
- stratification; separate by gender
Ways of strengthening external validity
- replication
What are the 4 criteria for merits of trustworthiness in qualitative inquiry
- credibility
- transferability
- dependability
- confirmability
Name the Truth value criterion in qualitative and quantitative approach
- credibility
- internal validity
name the applicability criterion in qualitative and quantitative approach
- transferability
- external validity
name the consistency criterion in qualitative and quantitative approach
- dependability
- reliability
name the neutrality criterion in qualitative and quantitative approach
- confirmability
- objectivity
Credibility
- true value
- are informants believable
transferability
- applicability
- can this be repeated in a different situation
dependability
- consistency
- can the findings be repeated if replicated
confirmability
- neutrality of data
Methods to enhance trustworthiness
- prolonged (breadth) and persistent engagement (depth)
- time sampling
- reflexivity
- triangulation
time sampling
sample all possible situations, times, and groups
reflexivity
diary, notes, peer interview, group participation
triangulation
convergence of multiple perspectives to ensure all aspects have been investigated
member checking
ask participants to review and react to study data
structural coherence
analysis to explain contradictions
audit trails
review process notes, raw data
sources of bias in qualitative research
- researcher presence
- over reliance on accessible participants
- influence of participants and site on researcher
procedural challenges
- entering study site
- establishing rapport and respect
- balancing professionalism and familiarity with participants
