Reading and Evaluating Scientific Research Flashcards
Objective Measurements
Are the measure of an entity or behaviour that, within an allowed margin of error, is consistent across instruments and observers
Variable
- Refers to the object, concept, or event being measured
- Behavioural measures
- Magnetic resonance imaging (MRI)
- Blood or saliva
- Self-reporting
Operational Definitions
Statements that describe the procedures (or operations) and/or specific measures that are used to record observations
Reliability
- Is when a measure provides consistent and stable answers across multiple observations and points in time
- Test-retest reliability
- Alternate-forms reliability
- Inter-rater reliability
Validity
Is the degree to which an instrument or produce actually measures what is claims to measure
Generalizability of Results
- Refers to the degree to which one set of results can be applied to other situations, individuals, or events
- Study large groups
- Sample to population
- Critical evaluation of findings
- Beware of cases of over-generalization
- “Mozart effect”
- Study large groups
What are the 2 types of samples?
- Random Sample
- Convenience Sample
Random Sample
Every individual of a population has an equal chance of being included
Convenience Sample
Samples of individuals who are the most readily available
Location of study
Laboratory vs naturalistic research
Ecological validity
The degree to which the results of a laboratory study can be applied to or repeated in the natural environment
Sources of Bias in Research
Researcher Bias vs Subject Bias
Hawthorne Effect
A term used to describe situations in which behaviour changes as a result of being observed
Demand Characteristics
- Inadvertent cues given off by the experimenter or the experimental context that provide information about how participants are expected to behave
- Clever Hans effect
Clever Hans Investigation
- Isolated Hans and questioner from any spectators
- Using people other than his owner to ask the questions
- Tested Hans with blinders
- Varied whether the questioner knew the answer to the question in advance
Social Desirability Responding
- Research participants respond in ways that increase the chances they will be viewed favourably
- Can minimize through assurances of anonymous/confidential questioning
Observer Expectancy Effect
- Researcher’s expectations can influence subject’s behaviour
- Teacher ‘favouritism’
- ‘Bright’ rats vs. ‘dull’ rats
Placebo effect
- A measurable and experienced improvement in health or behaviour that cannot be attributable to a medication or treatment
- “All in their head” or actual physiological response?
- Some evidence of physiological pain relief and changes in brain activation
- Conditioning effects
- Nocebo effect
- “All in their head” or actual physiological response?
What are the techniques that reduce bias?
- Anonymity
- Confidentiality
- Inform Participants
- Single-blind Study
- Double-blind study
Anonymity
Each individual’s responses are recorded without any name or other personal information that could link a particular individual to specific results
Confidentiality
Means the results will only be seen by the researcher
Inform Participants
Reduces participant’s anxiety and social desirability bias
Single-blind Study
The participants do not know the true purpose of the study, or else do not know which type of treatment they are receiving (e.g., placebo or treatment drug)
Double-blind study
A study in which neither the participant nor the experimenter knows the exact treatment for any individual
Sharing the Results
(Academic Journals)
- Peer Review
- Replication
Peer Review
Is a process in which papers submitted for publication in scholarly journals are read and critiqued by experts in the specific field of study
Replication
The process of repeating a study and finding a similar outcome each time
The Replication Crisis
- Chance results can and do happen
- Publication bias results in studies with affirmative evidence being more likely to be accepted into academic journals
- The original report or the failed replication
- Studies can fail to be replicated for any number of reasons
- We must base our beliefs on the entirety of the evidence
Weak forms of evidence
- Anecdotal evidence
- Appeal to authority
- Appeal to common sense
- Tradition or novelty
- Selective use of data
- Statistics are often inappropriately used to bolster weak arguments
Anecdotal evidence
An individual’s story or testimony about an observation or event that us used to make a claim as evidence
Appeal to authority
- The belief in an “expert’s” claim even when no supporting data or scientific evidence is present
- Biased expert?
Descriptive Research
- Descriptive data
- From observations
- No attempt to explain the ‘why’
What are descriptive research generated from?
- Case studies
- Naturalistic observation
- Surveys and questionnaires
Case Studies (Descriptive Research)
- Are in-depth reports about the details of a specific case
- Difficult to generalize findings
- E.g., Phineas Gage
- Drastic personality changes
Naturalistic Observation (Descriptive Research)
When psychologists unobtrusively observe and record behaviour as it occurs in the subject’s natural environment
Surveys and Questionnaires
(Descriptive Research)
Participant makes the observations
Correlational Research
- Involves measuring the degree of association between two variables
- Correlations have:
- Direction: Positive vs negative
- Magnitude: Correlation coefficient (-1 to +1)
Illusory Correlations
- Relationships that really exist only in the mind rather than in reality
- Crime increases when the moon is full
- Opposites attract
- Gamblers on a “hot streak”
- Stereotypes
Experimental Group
(Experimental Research)
Receives special treatment in regard to the IV
Control Group
(Experimental Research)
Similar subjects who do not receive the special treatment
Random Assignment
(Experimental Research)
A technique for dividing samples into two or more groups
Experimenter Control
(Experimental Research)
Variables are manipulated
Independent Variable
(Experimental Research)
Presumed cause
Dependent Variable
(Experimental Research)
Presumed effect
Confounding Variables
(Experimental Research)
Variables outside of the researchers control that might affect the results
Between Subjects Design
(Experimental Research)
- Participants who are in different groups are compared
- A large sample and random assignment makes equal groups likely, but not guaranteed
Within Subjects Design
(Experimental Research)
- All participants respond to all types of stimuli or experience all experimental conditions
- Order effects
- Separating measurements in time
- Counterbalancing
- Order effects
Quasi Experimental Method
- A research technique in which two or more groups that are compared are selected based on predetermined characteristics, rather than random assignment
- E.g., Comparing men and women
- Cannot determine cause-and-effect
Research Ethics Board (REB)
A committee of researchers and officials at an institution charged with the protection of human research participants
What are the potential stress to participants ?
- Physical stress
- Cognitive and emotional stress
- Writing about traumatic experiences
- Stress usually minor; benefits need to outweigh risks
Informed Consent
- A potential volunteer must be informed of the purpose, tasks, and risks involved in the study, and give consent to participate based on the information provided
- Topic
- Nature of stimuli
- Nature of tasks
- Duration
- Risks
- Steps taken to minimize risks
Deception
Misleading or only partially informing participants of the true topic or hypothesis under investigation
- Given enough information to consent
Full Consent
- Can refuse participation without fear of penalty
- Given equal opportunities
- The right to withdraw
- Withhold responses
Debriefing
Means that the researchers should explain the true nature of the study, and especially the nature of and reason for the deception
When are animals used?
- Treatments cannot be applied to humans
- Heritability studies require species with short lifespans
- Examining evolutionary origins of behaviour and cognition
Committes oversee ethical treatment
- Given appropriate housing, feeding, and sanitation
- Studies of sick or stressed animals do not provide generalizable results
- Risk and discomfort needs to be justified and managed humanely
Ethical Collection, Storage, and Reporting of Data
- Data kept for 3 to 5 years
- Replication
- Honesty with data paramount
- Acknowledge conflicts of interest
- Scientific misconduct has lasting repercussions
- The unfounded panic surrounding vaccines and autism
Descriptive Statistics
- Are a set of techniques used to organize, summarize, and interpret data
- Frequency
- Central tendency
- Variability
Frequency
The number of observations that fall within a certain category or range of scores
Central Tendency
A measure of the central point of a distribution (mean, median, mode)
Variability
The degree to which scores are dispersed in a distribution
Standard Deviation
- A measure of variability around the mean
- 75% +_ 10%
- This means that 68% of the class scored between 65%and 85%
Hypothesis test
A statistical method of evaluating whether differences among groups are meaningful, or could have been arrived at by chance alone
Statistical significance
Implies that the mean of the groups are farther apart than you would expect them to be by random chance alone
