W3 Measurement, Reliability, Validity Flashcards
Measurement
if a thing exist it exits in some amount, and if exists in some amount, it can be measured
A research question
Exposure Measurement
Available dose:
Administrated dose:
Absorbed dose (uptake):
Active dose (biologically effective):
Available dose: cumulative vs current
Administrated dose: the amount that comes in contact
Absorbed dose (uptake): the amount that enters the body
Active dose (biologically effective): that actually affects the specific target organ
-can be substituted for each other if the mechanism is known
Count
-the number of occurrence of an event
ex,
- in a major Canadian city, average annual number of homicides (30 homicides/ year in the 1970’s, 50/year in the 1990’s, 90/year in 2020-21)
-45 individuals older than 75 admitted to a hospital in London for hip fracture
Ratio
-relationship between 2 numbers
-Numerator NOT in the denominator
ex, (binary) sex ratio
Proportion
-relationships between 2 numbers
-numerator necessarily INCLUDED in the denominator
-proportion always range between 0 and 1
Odd
the probability of an event occurring relative to to it not occuring
Rate
speed of occurrence of an event over TIME
Numerator
-# of EVENTS observed for a given time
Denominator
-population in which the events occur (POP AT RISK)
-incorporates a set period of TIME
Measure of Health Events
Measures of Prevalence
What is Prevalence Rate?
What studies is this obtainable from?
What are the two types of prevalence
Prevalence rate: the proportion of the population (or the population sample or the sample subset that has a given disease or other attribute at a specified period of time
Obtained from cross-sectional studies
Two types:
Point prevalence rate and period prevalence rate
Point PR:
PPR= # with disease at SPECIFIC TIME
___________________________
Population at SAME TIME
Example 2: Prevalence of Arthritis in Postmenopausal Women
Period PR= go over
Period PR= # with disease at specific TIME PERIOD
________________________________
Total defined population at same period
Measures of Incidence
Incidence Rate:
Incidence Rate: the proportion of the Population At Risk that DEVELOPS a given disease or other attribute during a specified time period.
-Obtainable from cohort (longitudinal) studies (session 6, oct 14)
What is the pop at risk?
What is the correct numbe rof peole going to the denominator?
Whom should be excluded?
Relationship between prevalence and incidence
Incidence (develop disease)
-measures frequency of disease ONSET
-WHAT IS NEW
Prevalence(has disease)
-Measures population disease STATUS
-WHAT EXISTS
the next step is death or recovery
-incidence and prevalence may be expressed in any power of 10 (Per 100, 1000, 10,000, 100,000)
-more when we lear designs
Incident rate
Incident rate= # new events during specified time period
__________________________________
Population ‘at risk’
Example #2: IR of depression during 1st year of university
Measures of Association
Four Hallmarks of Health Studies
1) a research question/plausible theory
2) A well thought design to address the research question
3) Measurement of exposure and outcome
4) Analysis to compare groups (i.e, rates of disease among exposed vs in unexposed
4 main focus (refer to image)
Relative risk
-tells us HOW MANY TIMES AS LIKELY it is that someone who is ‘exposed’ to something will experience a particular health outcome compared to someone who is not exposed
-tells us about the STRENGTH of an association
-Can be calculated using any measure of disease occurrence:
—-prevalence
—-Incidence rate
does NOT tell us anything about how much MORE disease is occurring
Calculation of Relative Risk
Calculation of Relative Risk
Ex 1: risk of heart disease in smokers
note: for second column
- (3614 smokers wo hd-552smokers w hd=3062)
-(16386nonsmokers wo hd-1421non smokers w hd)=14955
Random error
error due to ‘chance’
Systematic Error
error due to recognizable source
-In epidemiological methodology: systematic error=bias
Is it possible to have both random and systematic error?
YES it is possible to have both random and systematic error
Precision vs Accuracy in Measurement
A measurement tool/scale with high
PRECISION IS RELIABLE (think of PR-consistent with bong price and reliable thats why anto bought bong)
A measurement tool/scale with high
ACCURACY is VALID
Insufficient Precision
Can be because
1.📏🦍
2.🕵🏻♀️⚖️✓🙍🏼♂️ 🕵🏿♀️⚖️𐄂🙍🏼♂️
3.🕵🏻♀️⚖️💁🏿♂️
- The measurement tool is not precise enough (a ruler in cm is not precise when meaningful differences are in millimeter)
- Two (independent) interviewers rate the same person differently using the same scale (inadequate training?)
- The same interviewer rates the same person differently
Statistical procedures are developed to quantify these reliability issues
A Valid Measure
We need to conceptualize the variable we want to measure clearly
Then, make sure that the scale
1)measures what it is supposed to measure, relevant information, underlying construct
2)does not include unnecessary questions
3)is able to do what it is designed to do: predict, discriminate, etc.
More relevant when a set of questions are used to measure a disease such as disability or depression, a behavioural traits such as gender role, etc.
A valid measure example
Sources of Measurement Error
Interviewer or observer
-record abstracting (random) error
-biased overestimation or underestimation
Participants
-recall
-random or systematic
Identify the most likely sources of Measurement Error
1) asking mothers with preterm birth if they were exposed to pesticide during pregnancy
2) Using 1st year psychology students to administer a questionnaire about depression without any training
3) Using only ‘STUDENTS WITH A HISTORY OF DEPRESSION’ for 2
4) Asking a group of older adults about the number of their friends in high school
Note: in real research always think of THE OTHER exposure or outcome group..so 3 is most likely source of measurement error since it is only using people with history of depression….what about the control group… the ones that have 0 history of depression.
Two by Two Table with No Measurement Error
Two by Two Table with Measurement Error
Information(measurement) (IM)ERROR can lead to MISclassification
Classification…
Non-differential🙎🏻♂️🙎🏻♂️🙎🏻♂️📝 vs Differential🙎🏻♂️💁🏿♂️📝
CLASSIFICATION of participants into the wrong exposure or outcome groups
Non-differential (the same in all study groups )
-usually weakens associations- i.e brings effect estimates (RR, OR, AR) closer to the null value but not always..
Differential (different in different study groups)
-effect estimates may change in any direction, depending on the particular error
The ‘Truth’ results of a hypothetical case-control study with no measurement error
OR= (💀Exposed cases x Unexposed controls)
_______________________________
(Unexposed cases x exposed controld)
OR= (💀ECa x UECo)
___________
(UECa x ECo)
-think of cases on left side, controls on right side, exposed and unexposed are diagonal to each other
Non-differential Misclassification of 10%
OFTEN due to RANDOM MEASUREMENT ERROR
USUALLY biases towards null (dilutes the estimate)
Differential Misclassification of 20%
How can we reduce measurement error?
- little (or nothing) can be done to fix information bias once it has occured
-information biases such as misclassification arising from measurement error MUST be avoided through careful study design and conduct
—–clear measurement protocols, pilot tests, validation
-appropriate choice of instrument
——-is it accurate(valid)? Precise(reliable)
information bias cannot be ‘controlled’ in the analysis
Tutorial
look at tut and practice each question… right it down and understand the difference between each of them and how to approach the problem
Measures of event
1.
In a study of 500 patients with coronary heart disease, 100 people already had diabetes when the study started on 1 January 2021. Over the next year, 50 more people developed diabetes. What is the prevalence of diabetes at the end of 2021? (Assume the diabetes is permanent, nobody was treated and there are no
losses or entries to the group of patients with heart disease.)
Measure of event
2.
A 10-year longitudinal study recruited 12,197 middle-aged adults (40-49 years old) at high risk of type 2 diabetes. Of these individuals, 1,002 had prevalent type 2 diabetes at the baseline exam. By the end of the 10-year follow-up, a total of 2,451 incident cases of type 2 diabetes had developed in the at-risk population within this cohort of adults. The 10-year incidence rate of type 2 diabetes in this cohort of adults is ____ per 1,000 persons.
Measures of association
1.
35,271 women without a history of breast cancer were identified in Sept 2012. Of these, 7054 were smokers, 5001 were former smokers, and 23216 were never smokers. Over a 10-year follow-up, 189 of the smokers developed breast cancer, 102 of the former smokers developed breast cancer, and 199 of the never smokers (unfortunately) also developed breast cancer.
The relative risk of breast cancer in smokers relative to never-smokers was ____
The relative risk of breast cancer in previous smokers relative to never-smokers was ____
The relative risk of breast cancer in smokers relative to never-smokers was ____3.13.
The relative risk of breast cancer in previous smokers relative to never-smokers was ____2.38.
Measure of association
2. A cross-sectional study of 4,000 older adults showed that 25% of the participants are obese, defined as BMI [weight/(height)2 ] larger than 25. 300 of the overweight participants have fatty liver disease and 325 of the ‘normal weight’ participants have fatty liver disease. The odds ratio and relative risk for fatty liver disease in overweight participants relative to ‘normal weight’ participants were ____ and ____.
Misclassification 1
The measurement in the study above was by direct assessment of weight and height, a time-consuming process. A researcher thinks that self-report of height and weight provides the same information with much less effort. They repeated the measurement in the same participants and found that 5% of overweight participants (in both outcome groups) are classified as ‘normal weight’ and 5% of ‘normal weight’ individuals are classified as overweight, also in both outcome groups. How the odds ratio calculated from the new data will be different from the original data? Is it a differential or non-differential misclassification?
Misclassification 2.
What will the odds ratio be if the above misclassification happens only among controls (not in cases)? Is it a differential or non-differential misclassification?
