Quantitative Research Flashcards
Epistemology
the philosophy of knowledge
methodology
an approach to knowing
Deductive thinking
- theory
- hypothesis
- observation
- confirmation
inductive thinking
- observation
- pattern
- tentative hypothesis
- theory
Hypothetico-deductive model
circle of
hypothesis –> deduction –> predictions –>observation –> test of predictions –> induction
Paradigm
a set of implied assumptions
ontology
- deals with questions concerning what entities exist or can be said to exist
- Specifies the nature of reality that is to be studied, and what can be known about it.
epistemology
the study of knowledge
methodology
the branch of knowledge that deals with the methods of a particular discipline, the study of the directions and implications of empirical research
positivist view of research
- science is a way to learn the truth
- science is deterministic (x causes y)
- science is mechanistic (either prove or disprove)
- science uses methods
- science only deals with what we can see or hear (empiricism)
- best way to learn the truth is to experiment
- science is objective
Post-positivist view of research
- Similarity between common sense and science
- natural selection model of knowing (random variation –> selective retention
- multiple perspectives (all observation is errorful)
- observation is theory-laden (judgement is unavoidable in science)
- nonreductionism
critical realism
- Believe there’s a reality that we should try to ‘get right’
- Critical of our ability to ever get it perfectly right
Quantitatively oriented social scientists
- primarily work in a postpositivist paradigm
- are principally interested in numerical data
- use statistical analyses
- use mostly probability sampling
Qualitatively oriented social scientists
- primarily work in a constructivist (or interpretive) paradigm
- are principally interested in narrative data
- use thematic strategies to analyse data
- use mostly purposive sampling
Belmont Report- three principles of ethics for research with humans
- respect for persons
- beneficence
- justice
Institutional Review Boards
- Mechanism for reviewing proposed research
- Mechanism for protecting the institution and researcher
- Submitting an IRB proposal
Ethical Issues
- Voluntary participation
- Informed consent
- Risk of harm
- Anonymity
- Confidentiality
- Right to services
Obtaining informed consent
- Must provide participants with adequate information, AND secure explicit agreement to participate.
- Use simple, straightforward language (no jargon)
- Declare why you are conducting the study
- Declare nature, duration of tasks participants will be asked to complete
- Declare anticipated risks, and benefits of participation
- Inform participants of their right to withdraw from the study
- -But also advise consequences of withdrawing
- Indicate how confidentiality or anonymity of data will be achieved
- Allow and encourage participants to ask questions about participation
- Provide contact information of PI (principal investigator)
Problems with informed consent
- In general, the act of obtaining informed consent may detrimentally affect the validity of a study
- -Induce contrived behaviour
- -Induce demand characteristics
Debriefing
Once the study is complete:
- If participants were deceived, explain why this was necessary
- Make sure that there were no ill effects, if this is a potential issue
- provide additional resources if necessary
Threats to external validity
- selection, setting and history
- maybe it is just these people, these places, these times
how can we improve external validity
- random sampling
- repetition
- use theory
Probability sampling
Each member of the population has a certain probability of being selected
Nonprobability sampling
Members selected not by mathematical rules, but by other means (e.g. convenience of access)
Cannot be used for most statistical analyses
Well suited for qualitative research, where distribution is not important
Element
- The unit about which information is collected
- Typically the elements are people
Sampling frame:
a master list of the population (total or partial) from which sample is drawn
Sampling process
- defining the population
- developing a sampling frame
- determining sample size
- specifying sample method
- selecting the sample
Importance of sampling properly
- A sample exists to represent its parent population
- We must know what the actual parent population is, otherwise we draw false conclusions
4 different probability sampling
- systematic sampling
- simple random sampling
- stratified random sampling
- cluster sampling
systematic sampling
-Put your population in a list
-Select every nth participant (e.g. every 12th)
n is determined by desired sample size
-e.g. With a population of 300, if we want a sample of 10, choose every 30th case
-Only useful if you have a complete list of population
systematic sampling advantages
- easy
- but assumes that the population is randomly ordered
- suitable for homogeneous populations
Simple random sampling
- Random: without a rule or method
- Typically use random number generators e.g. on Excel to assist us in this
- Each element in the population has an equal and independent probability of being chosen
stratified sampling
-Expands on random sampling
-Build sub categories, then sample randomly inside each one
Eg: decide you will have 10 men and 10 women
-Random sampling cannot ensure equal group size; stratification can
Stratified Sampling - Purposes:
- To insure representation of each strata.
- Increase precision (lower variance) if strata are homogeneous within (like blocking).
Cluster sampling
- Is a type of sampling in which clusters or groups of elements are sampled at the same time.
- Such a procedure is economical, and it retains the characteristics of probability sampling.
- Administratively useful, especially when you have a wide geographic area to cover.
Types of Nonprobability Samples
- Accidental, haphazard, convenience
- Purposive
- Snowball
Accidental, Haphazard or Convenience Sampling
- “Man on the street”
- Available or accessible respondents
- Volunteer samples
- Most convenient
- Common in exploratory research
- Problem: No evidence for representativeness; exercise caution
Purposive Sampling
- Might sample several pre-defined groups (e.g., the shopping mall survey that attempts to identify relevant market segments)
- Deliberately sampling an extreme group
- Problem: Proportionality
- Problem: Need theory to correctly sample an extreme group
Snowball Sampling
- One person recommends another, who recommends another, who recommends another, etc.
- Good way to identify hard-to-reach populations (for example, homeless persons), or covert groups
- Typically used in qualitative research
- Primarily used for exploratory purposes
descriptive research
what is the opinion of a group of people?
relational research
how is their opinion related to other characteristics
casual research
what factors affect their opinion
Exploratory studies
-To develop an initial, rough understanding of a phenomenon
-Methods:
literature reviews
interviews
case studies
key informants
Descriptive studies
-Precise measurement and reporting of the characteristics of the population or phenomenon
e.g. what is the case? How many people have depression?
What is the nature of the relationship? Are married people less depressed?
-Methods: census, surveys, qualitative studies
Relational studies
-the relationship between two or more variables. eg. gender and voting patterns
Attribute:
A specific value on a variable
Internal validity
The internal logic of the experiment - can the design sustain the conclusions?
External validity
- The generalizability of the experimental results - can the conclusions be generalized to the population?
- Says nothing about the truth of the result that we are generalizing
- External validity only has meaning once the internal validity of a study has been established
- Internal validity is the basic minimum without which an experiment is uninterpretable
Internal validity – how to judge
- Do the conclusions or findings follow from the data, and procedures used?
- Threats (examples)
- Look for alternative explanations of the results (i.e. not just confirmation)
- Testing for alternative explanations is a powerful evaluation: if we identify a plausible rival explanation, it undermines the very foundation of the design
External validity – how to judge
Ask about three aspects of the design
Participant selection
Operationalization of IV
Operationalization of DV
Participant selection (threats to external validity)
If unrepresentative participants are used, it will be difficult to generalize results beyond the experiment/the study.
This threat always applies: all experiments (designs) will suffer from problems of representativeness; the degree is important.
Operationalization of IV (threats to external validity)
Always important - there are usually many ways of operationalizing the IV. Will the results generalize to other operationalizations?
Present case: Time parents spend with child; type of parent; type of material read; nature of reading contact.
Operationalization of DV (threats to external validity)
There are also many ways of operationalizing the DV. Will the results generalize to other operationalizations?
Present case: use standardized test, which helps; but we could have used the end of semester mark that each child scores. The point is that we want the benefit of the intervention to extend beyond mere performance on the standardized reading test.
Internal validity - Two key interpretive problems
- The 3rd variable problem
2. The problem of direction
Causal Hypotheses
- Statement of relationship between an independent and dependent variable
- Describes a cause and effect
- Usually stated in two forms
- –The null hypothesis
- –The alternative hypothesis
Experimental research
- The overarching aim behind experimentation is to identify a cause-effect sequence
- -the aim is to show that changing the IV results in a change in the DV, and to leave no doubt that the observed change is NOT due to some other change
Elements of a Design
- Observations or measures
- Treatments or programs
- Groups
- Assignment to group
- Time
Characteristics of ‘true’ experiments
- The independent variable(s) is an active variable i.e. one that can be manipulated
- The conditions under which the independent variable is implemented are equivalent in all conditions except for planned differences on the independent variable
Threats to internal validity
- Covarying events (History)
- Independent natural change (Maturation)
- Reactive measurement (Testing)
- Measurement unreliability (instrument decay)
- Statistical regression of extreme participants
Attrition
-Sometimes called experimental (or subject) mortality
-If subjects drop out, it creates a bias to those who did not
e.g. comparing the effectiveness of family therapy with discussion groups for treatment of drug addiction
addicts with the worst prognosis more likely to drop out of the discussion group
will make it look like family therapy does less well than discussion groups, because the “worst cases” were still in the family therapy group
