5.0 Research design Flashcards
Outline the steps to formulate a research problem (4)
Idea emerges
Brainstorm
Review literature
Identify variables
Research problem formulated
What is a research design?
The physical ‘doing’ part of the research process (e.g. sampling, data collection, data analysis).
Independent variable
The variable that is assumed to cause or influence the dependent variable(s) or outcome. The independent variable is manipulated in experimental research to observe its effect on the dependent variable(s). It is sometimes referred to as the treatment variable.
i.e. factor influencing the outcome being studied,
Dependent variable
The variable presumed within the research hypothesis to depend on (be caused by) another variable (the independent variable); it is sometimes referred to as the outcome variable.
i.e. result or outcome being studied.
Internal validity
The extent to which the outcome of a study can be attributed to the cause (Independent variable); also refers to the accuracy of the findings.
(Refers to the extent to which changes in the dependent variable (the observed effects) can be attributed to the independent variable rather than to extraneous variables.)
External validity
The extent to which the findings can be generalised beyond the sample; can the results be replicated with other samples.
(Refers to the degree to which the results of a study are generalisable beyond the immediate study sample and setting to other samples and settings.)
True or false. Internal validity is a prerequisite for external validity
True
What are the observational research designs?
Cross sectional studies
Case control studies
Cohort studies
(Researcher observes what occurs)
What are the experimental research designs?
RCTs
Quasi experimental designs
(Researcher intervenes to change the situation and watches what happens)
Cross-sectional study is aka
Prevalence study
A cross-sectional study provides …
Provide a snapshot of frequency and characteristics of a health condition/state among a population at a point in time.
Most common type of obsevational study. Most likely when assessing prevalence of a disease. Provides snapshot - one point in time. No follow up. e.g census survey.
True or false. In cross-sectional studies, ,measurement of exposure and outcome occur simultaneously among population of interest.
True
A cohort is…
a group that has something in common who are followed or studied over time
A cohort study
Commonly, a researcher studies the development of a particular health outcome or disease state.
Participants do not have the disease when recruited.
Then participants are classified according to whether they have one or more explanatory variables hypothetically related to the outcome. E.g. 2 groups - exposed smokers and unexposed non-smokers.
Then studied overtime to determine who develops the outcome of interest (e.g. lung cancer)
Can be cross-sectional (one point in time, or sequences of time points) or longitudinal (over long time periods).
Can be used to examine relationships retrospectively (more bias) and prospectively (more costly).
Can determine incidence e.g. new cases of COVID-19.
Case-control
Retrospective study design
Compares the personal characteristics and exposures of individuals with and without the disease/outcome of interest
Selects participants according to presence or absence of outcome of interest (DV, usually the disease); the exposure (IV) is determined retrospectively by careful examination of the participant’s history
Is a case-control study retrospective or prospective?
Retrospective - look back to identify risk factors for the outcome
In a case-control study who are the cases and who are the controls?
Cases = those who exhibit the outcome
Controls = those with similar characteristics but who do not exhibit the outcome
Note often have many more controls than cases.
Cases = outcome present Controls = outcome absent
Exposure present, exposure absent
Steps in a case control study (4)
- Identify cases
- Select controls which may be matched to the cases
- Measure exposure or risk factors of interst
- Compare the presence or absence of exposure in cases and controls
Steps in a prospective cohort study (3)
- Identify exposed and unexposed cohort groups
- During follow up period, identify diseased subjects (incident cases)
- Analyse differences (i.e. incidence or relative risk) among cohort exposed and cohort unexposed.
Steps in a retrospective cohort study (3)
- Identify exposed and unexposed cohort groups
- Identify diseased subjects by interview or written records
- Analyse differences (i.e. incidence or relative risk) among cohort exposed and cohort unexposed.
Pyramid of evidence - top to bottom
RCTs
Prospective cohort
Retrospective cohort
Case-control
Cross-sectional
Case series
Fewer threats to internal validity at the top.
Less bias –> more bias
Randomised controlled trials
Gold standard for testing cause and effect relationships in clinical research. Use when you want to determine effectiveness.
- Participants are randomly assigned to the experimental and control groups so that any pre-intervention differences are meausured/controlled.
- Intervention introduced to treatment group and outcome variable is measured again to see whether it has changed.
Control group gets no experimental treatment but is also measured for comparison purposes.
- Difference between the 2 groups at post-test reflects whether a causal link actually exists between explanatory and outcome variables.
True or False. Experimental study designs can establish cause and effect between the IV and DV.
True
What are the characteristics of experimental studies?
MANIPULATION of the IV which should be reflected in variation in the DV .
RANDOM ASSIGNMENT: participants have an equal and independent chance of being allocated to one of the study conditions; i.e. creation of equal study groups
CONTROL: the researchers instigate measures to ensure the study has high internal validity; i.e. reduce the likelihood of confounders impacting the cause-effect relationship being studied
What are the 4 types of qualitative research designs?
Descriptive (an emerging trend)
Phenomenology (Traditional)
Ethnography (Traditional)
Grounded theory (Traditional)
Descriptive studies
Qualitative
Rely on pure description rather than theory
The goal is to provide comprehensive summary of events, situation or experience
It is a method of choice when a straight description is required
The use of this approach means that the researcher is more likely to move away from methodological, theoretical and discipline specific language in order to remain close to the data.
Data is usually collected from a small sample and analysed thematically
The researcher adopts general qualitative principles and are not governed by complex methodological, theoretical or philosophical positions that require more detailing and exploration.
Phenomenology
[Looking at experience]
- Uncovers thoughts, perspectives, feelings and behaviours from the perspective of the person.
- Does not offer causal explanations or theories.
- It provides an account of the experience of ‘being in the world’ of everyday life, of living in and through the world.
- No single unified philosophical standpoint.
- Reject scientific realism as the only way of describing the social world
- Consider people tied to their worlds and are only understandable in their contexts - some background of beliefs and practices (culture, language, gender, education…) which is never completely capable of articulation
Ethnography
[Culture]
Origins in anthropology
Provides a means for exploring cultural groups
Culture: The beliefs, behaviours, norms, attitudes, social arrangements and forms of expression that form describable patterns in the lives of members of a community or institutions
Facilitated by explaining one culture in terms of another- Recognition of subcultures in societies
The insider view - the emic perspective is what the researcher attempts to describe
Difficult for insiders to describe
Always conducted in the natural setting - in the field
Usually over an extended period of time
Direct study of the lives of the cultural groups
Seeks key informants – have the ability to reflect on and describe the culture
Grounded Theory
[Used to generate theories]
A qualitative research method that uses a systematic set of procedures to develop an inductively derived grounded theory about a phenomenon.
Origins in sociology
Philosophical underpinnings in pragmatism
The aim is to construct a theory to explain these processes that is grounded in the data
Data sources include in depth interviews and field notes
Types of phenomenology
Transcendental phenomenology (descriptive, knowledge based, Husserlian, lebenswelt/lived exp)
Existential phenomenology (lived experience, Satre)
Hermeneutic phenomenology (interpretive, concerned with being, Heieggerian, dasein/existence)
Assumptions of phenomenology
Human behaviour occurs in the context of relationships to things, people, events and situations
Reality is comprehended through the embodied experience or the lived experience
Experiences provide evidence of the world – not as it is thought to be but as how it is lived
Experiences include perception (hearing, seeing), believing, remembering deciding, feeling, judging, evaluating
Imagination, dreams and other spiritual experiences are valid experiences
Language is the central medium for transmitting meaning
However, naming meaning requires interpretation
Experience precedes language
Assumptions of ethnography
Cultural beliefs and behaviours are:
- Embedded within cultural groups
- Learned and transmitted in the group
Culture is dynamic, changing and adapting to new circumstances
Best explored by researchers who are not part of the cultural group
Forms of ethnography (3)
TRADITIONAL
- in a culture unfamiliar to the researcher
- focus on a single setting
- over an extended period of time
FOCUSED (aka micro ethnography)
The topic is specific and may be identified prior to the research
Often conducted with sub cultural groups
Institutions
Groups with like characteristics
AUTO ETHNOGRAPHY
Originally referred to as the cultural study of one’s own people
Combines ethnography and autobiography
Characteristics of grounded theory (4)
Requires high quality data from the actual settings
Continual interplay between data analysis and data collection – constant comparison
Purposeful/purposive sampling
Theoretical sampling : researchers seek additional data based on concepts developed from initial data analysis
Theoretical perspective is symbolic interactionism, which has three main assumptions:
1.Humans act towards things on the basis of meanings individuals have for them.
2.Meaning is created through interaction between people.
3.Meanings are modified through an interpretive process
What is meant by Validity?
Validity determines whether the research/test truly measures that which it was intended/claims to measure
A researcher who asks, “Does my study measure what it says it measures?” is looking to make her study valid. Validity has to do with the strength of conclusions made about research.
Measurable and static = Positivism or Interpretivism
Positivism
Multi-dimensional and dynamic reality = = Positivism or Interpretivism
Interpretivism
internal and external validity are the two primary types of validity. What are the others?
Construct validity
Statistical Conclusion validity
Face Validity
Content validity
What is construct validity?
Construct validity: Does the test measure the concept that it’s intended to measure?
A construct represents a collection of behaviours that are associated in a meaningful way to create an image or an idea invented for a research purpose.
A test has construct validity if it demonstrates an association between the test scores and the prediction of a theoretical trait. Intelligence tests are one example of measurement instruments that should have construct validity.
i.e. Depression is a construct that represents a personality trait which manifests itself in behaviours such as over sleeping, loss of appetite, difficulty concentrating, etc. The existence of a construct is manifest by observing the collection of related indicators.
What is Statistical Conclusion validity
A determination of whether a relationship or co-variation exists between cause and effect variables. Requires ensuring adequate sampling procedures, appropriate statistical tests, and reliable measurement procedures. This is the degree to which a conclusion is credible or believable.
Face Validity
This is the least scientific method of validity as it is not quantified using statistical methods. It is concerned with “whether it seems like we measure what we claim”. In research its never sufficient to rely on face judgments alone and more quantifiable methods of validity are necessary in order to draw acceptable conclusions.
Content Validity
This is also a subjective measure but unlike face validity we ask whether the content of a measure covers the full domain of the content. Where it distinguishes itself is through its use of experts in the field or individuals belonging to a target population. This study can be made more objective through the use of rigorous statistical tests.
What is meant by errors?
All threats to validity are types of systematic error, not random error.
What are some threats to internal validity?
History → Specific events that happen during the course of a study can affect the variable being measured.
Maturation → Changes that happen within a person due to time passing can affect the variable being measured.
Regression toward the Mean/Statistical regression → The tendency for people who receive high or low scores on a particular measure to score closer to the mean on subsequent testing
Testing Effect → People tend to do better on a test the second time they take it.
Experimental Mortality → When different amounts of people or different types of people drop out of the two conditions of your experiment
Participant Reaction Biases → Participants try to behave in ways that are consistent with – or the opposite of – the researcher’s hypothesis.
Experimenter Bias/influence → Experimenters’ expectancies can affect either what they observe or how they act with participants.
Confounds → Some additional variable (that you don’t care about) varies systematically along with the thing you manipulated; alternative explanation.
Homogenous attrition → When a particular type of person drops out of your study, regardless of which condition they are in.
What is meant by Reliability?
Reliability tells us how accurate and trustworthy a test/score is. Reliability is the consistent, reproducible estimates of what is assumed to be an underlying true score.
It tells us the extent to which results are consistent over time and an accurate representation of the total population under study. If the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable i.e. If someone else did the same experiment/study, they would get the same result over and over again. Reliability is not measured, it is estimated.
How is reliability usually measured?
- Test-re-test method i.e. spirometry
- Internal consistency method
Explain test re-test method to measure reliability.
The correlation between the scores from two different administrations of the same instrument to the same students i.e. you should get the same score on test 1a as you do on test 1b
There are 3 main components to this method:
a. implement your measurement instrument at two separate times for each subject
b. compute the correlation between the two separate measurements
c. Assume there is no change in the underlying condition (or trait you are trying to measure) between test 1 and test 2.
Explain Internal consistency method
Internal consistency estimates reliability by grouping questions in a questionnaire that measure the same concept
The primary difference between test-re-test and internal consistency is that test-re-test involves two administrations of the measurement instrument, whereas the internal consistency method involves only one administration of that instrument.
True or False. Using data triangulation can also increase the reliability of results
True
Triangulation
Triangulation — the use of a variety of data sources or methods to examine a specific phenomenon either simultaneously or sequentially in order to produce a more comprehensive and accurate account of the phenomenon under investigation.
Alternate (or parallel) forms method
To eliminate practice effects and other problems with the test-retest method (i.e., reactivity), test developers often give 2 highly similar forms of the test to the same people at different times. i.e. 2 correlated tests measuring the same thing
Split half reliability method
Split-half methods propose that many of the temporary factors influencing the test-re-test method and alternate forms method can be eliminated if test is administered only once. Two scores can be obtained simply by splitting the test into two halves. Scores on each halve are correlated with each other. A high level of correlation is taken as evidence that the items are consistently measuring the same underlying construct
Two sources of error for reliability:
Experimental variability→ real differences highlighted by the test/study
Error variability →
a) Random fluctuation: random or chance elements: fatigue, noise, mood etc: factors that are temporary and shifting
b) Systematic or constant error: confounding variable
Sometimes some measurement instruments require the person making the measurement to render a judgement, such as concerning an individual’s functional ability in some area. When any aspect of the measurement process or scoring of an instrument involves human judgement and, hence a degree of subjectivity, a third source of measurement error must be considered
c) inter-rater or inter-observer variability: the extent to which two or more individuals (coders or raters) agree
Factors influencing Reliability
Length of test
Restrictions in group performance
Objectivity of assessment
Method of estimating reliability
Lebenswelt =
life world, the world of lived experience
Associated with Husserlian / transcendental phenomenology
dasein
existence
“consciousness” and “mind,”
Associated with Hermeneutic / Heideggerian phenomenology
Etic =
Emic =
Outsider
Insider
E.g. Etic perspective
Grounded theory originates from and has been further developed by:
Glaser and Strauss.
