Quiz 5 Flashcards
Reliability
Issues related to the soundness of the data collection procedures
Validity
Is the authenticity of the results
- External Validity
- Internal Validity
External Validity
The extent to which the results of the study will be true for different groups of people or similar people in different settings. A researchers decisions about research design and sampling method will impact the degree of external validity in each study
Internal Validity
The extent to which the results of the study are true. When a researcher conducts a study, they want to make sure the results they get are because the intervention worked instead of as a result of some confounding variable.
Confounding Variable
A variable the researcher is unaware of and that has an impact on the outcome of the study
Threats to Internal Validity
- History
- Maturation
- Testing
- Instrumentation
- Statistical Regression
- Placebo effect
- Hawthorne Effect
- Selection Bias
- Attrition (loss to follow up)
History
-Threat to Internal Validity
-An outside event that occurred during the research study that can impact the results of the study.
Controlling for this threat: Control Group
Maturation
-Threat to Internal Validity
-Time passed and the participants grew “older, wiser, stronger, more experienced”
Controlling for this threat: Control Group
Testing
-Threat to Internal Validity
-Subjects become better at a test because they become more familiar with it
Controlling for Testing: Select an instrument that has high validity & reliability
Instrumentation
-Threat to Internal Validity
-Changes in accuracy of measurement from start to conclusion
Controlling for Instrumentation: select an instrument that has high validity and reliability
Statistical Regression
- Threat to Internal Validity
- Participants with high scores on first test tend to perform lower on second test (vice versa)
- Controlling for Statistical Regression: select a data collection tool that has very high validity and reliability
Placebo Effect
- Threat to Internal Validity
- The participants’ or researchers’ expectations that something will work can impact the results of the study.Controlling for the Placebo effect:
- Double-Blind: Both parties don’t know which variable is being tested in which group
- Placebo-controlled: The experimental group gets the real treatment and the control group gets a fake treatment.
Hawthorne Effect
- Threat to Internal Validity
- Research participants will change their behavior simply because they know they are being observed
- Controlling for the Hawthorne Effect: Select a research design that includes a control group & placebo. In obervational studies, conduct sustained observations and employ unobtrusive observation methods.
Selection Bias
-Threat to Internal Validity
-How the researcher selects people to participate impacts the study
Controlling for Selection Bias: Use probability sampling method
Attrition (loss to follow-up)
- Threat to Internal Validity
- People leaving the study
- Controlling for Attrition: N/A
Validity and reliability are exclusive to quantitative research
True
Qualitative practices established Trustworthiness
- Transferability
- Credibility
- Dependability
- Confirmability
Transferability
The reader of the research study determines the extent to which findings can be transferred to their settings or group. In other words, the person who reads the research determines if the findings of the study are a good fit to their situation.
One way that helps the reader make this determination is to have a lot of detailed info about the participants, the research setting, and findings in the journal article. This is known as THICK Description.
Credibility
The confidence in the truth of the findings. Two strategies:
1. Triangulation:
collecting different types of data (verbal, textual, images, etc), collecting data at different times, and/or having 2 researchers collect and analyze data. Looking at many different types of data helps the researcher have confidence that they have uncovered all the data required to gain a complete understanding of the research question.
- Memebr Checking:
sharing the data analysis with the research participants and/pr experts in the field working with participants. When the researcher begins to see meaning develop from the data, the researcher asks the participants if the meaning revealed from the data is true. Assists the researcher in uncovering any hidden bias.
Dependability
Relies on whether the results of the study make sense to another researcher. Here the practice is not for a researcher to replicate the study and achieve identical results; rather it is to ask the question “are the results consistent with the data collected”
One way to accomplish this is through an Audit Trail
Audit Trail: A detailed reporting of how the researcher conducted the study, especially the collection and analyses of the data.
Sample
A group selected from a population in the hopes that the smaller group [the sample] will be representative of the entire population
Population
A group that shares a common characteristic as defined by the researcher
Inclusion Criteria
QUAN- Determines who is suitable to be a a participant based on certain characteristics that are defined by the researcher based on the purpose of the study; high internal validity (dependability), low external validity (generalizability) if the criteria is too strict
Exclusion Criteria
QUAN- people who have met inclusion criteria but should not be included in the study (ex. If the effects of a medicine can harm a pregnant person then it would be considered exclusion criteria)
Probability Sampling Methods
only quan, focus on cause and effect, high internal and external validity
Allows the researcher to obtain a random selection of individuals from the population (random sample)
Simple Random
-probability methods (ONLY QUAN).
Simple random: equal chance of random selection in population
Stratified Random
- probability methods (ONLY QUAN).
The researcher identifies a subgroup or subgroups in the population and wants to ensure that the sample represents the subgroup(s) found in the population
Proportional Stratified
- probability methods (ONLY QUAN).
The researcher identifies a subgroup or subgroups in the population that are very unequal in size and wants to ensure that the sample will represent the population
Systematic
- probability methods (ONLY QUAN).
The researcher selects participants based on a randomly chosen number