Modules (1-3) Flashcards
What is the publication process?
it takes __ to ___ years
journals are the __ of science
What’s the process?
How long does it take to complete?
The rigorous process primary research articles have to go through to be published.
- It takes 2-4 years after the research is
conducted is it published in a journal (i.e.,
publication lag). - Technology has reduced the lag, but it still
is a slow process. - Journals are the gatekeepers of science:
there is an editor who decides what is
published (i.e., a scientist who in their
spare time is the editor of the journal).
The Process:
1. Do the research, write an article and
submit a manuscript to a journal for
publication.
2. The editor of the journal either:
a. Desk rejects article (i.e., article is rejected
before being sent for review; when the
article doesn’t fit the theme of the journal,
there is something fundamentally wrong
with it or it’s not interesting enough).
b. The article is sent out to well-known
researchers who specialize in the topic or
methods used in the target article for 3-4
researchers to critically evaluate it and
provide feedback.
3. The editor takes into account the
feedback provided and comes to a
decision about the article:
a. Accept.
b. Accept under the condition the author
makes small revisions to the article.
c. Revise and re-submit where the author
makes the necessary changes and then
resubmits their article to the journal where
it is sent out again to 3-4 researchers for
review (this process can happen multiple
times till it meets the standards of the
journal).
d. Rejected and author needs to submit the
paper to another journal with lower
standards.
*This process of revision takes between 6-
months to a year and a half to complete.
how can we identify a good research article?
the number of citations or the quality of the research journal it was published in.
what is a primary source?
Primary Research Literature:
- Articles on research that’s been conducted
by scientists in labs and published in an
academic journal.
- Publication is a key part of scientist’s job, it
determines promotions, grants, good PHD
students and if their students get
scholarships (incentives) but the key reason
is because they love doing it.
Not the last word!
Can a single study answer all the questions?
science is __ and ___
• A single study never answers the question
(i.e., specific to participants or variables
used; to answer questions we need
replication; each study is a single piece in
the puzzle).
• No study is perfect. There is always
limitations due to the trade-offs in
designing a study that is ethical and
practical.
• Scientific literature is cumulative and self-
correcting (to build a body of literature; self-
correcting through replication to support or
contradict existing theory; one study
doesn’t throw out a theory but as more and
more accumulate with similar findings
revisions to theory are made; science is
always progressing; they support theory
but do not PROVE anything).
• You do not get paid to publish, review or
edit research articles. It is just a part of your
job as scientists to contribute to the body
of scientific literature.
Two Types of Articles
Peer-reviewed scientific literature
- Empirical Research
• Report the results of a research project[s] (more
than one if combined paper with 3-4 studies
testing the same specific hypothesis)
• Their standard format is
Abstract/Intro/Method/Results/Discussion - Review Articles
• Combine the results of several (many) research
studies to summarise and draw conclusions.
• Narrative review (more like a story where the
author takes bits and pieces from different
studies to make a point on a psychological
phenomenon).
• Systematic review (the author has done very
careful library work to ensure they’ve
mentioned EVERY study that’s been published
on the specific question; nothing left out;
comprehensive review).
• Meta-analysis (combine data from quantitative
studies on the specific question to perform
analysis on and make a comprehensive
conclusion on the effect in question).
*still peer-reviewed
*commentary on how different studies have
addressed the same question with different
samples, methods, variables, analysis and
comment on any consistent patterns found.
Is it any good?
Peer-Reviewed Journal:
Is it in a “good” peer-reviewed journal?
• Use journal rankings to know which journals
have high standards and peer-review.
• You need to watch out for predatory journals
which are journals in it to make money. They
charge authors money to publish studies in
their journal.
Are conflicts of interest declared?
• Funding? Especially in medical research,
college, intervention/treatment research that
is funded by a company who wants to sell
that treatment if successful.
• Business interests? Financial interests in the
thing they’re writing about.
• These should be declared at the beginning
or the end of the article.
Read critically yourself
• What are the studies strengths and
weaknesses? How generalisable are the
findings? How does this fit in with other
studies on the same topic? Consistency
across studies increases our confidence that
the findings are reliable and valid.
What are secondary sources? Name three examples
Secondary Sources:
People who have read primary literature who have synthesized it into make It readable in a short period of time.
- Textbooks:
o A researcher who has knowledge on the
field and reviewed all the literature on the
topic to synthesize it and communicate it in
an easier to digest format. - Trade Books:
o These are books written by researchers on
their own work/experiences over the years. - Media/Journalism Sources
o Journalism, social media, news outlets,
blogs etc.
o Some is good but some of it is very bad
(sensationalism, misinformation, selling
something etc.).
o The media likes to exaggerate and draw
conclusions that are not supported by
science. This conflicts with scientist’s
tendency to draw tentative conclusions that
are fully supported by science.
Is it any good?
Text/Trade Books:
Is it any good?
Media/Journalism Sources:
Is it any good?
Text/Trade Books:
Is the author also a researcher? Read the
books bio to see if the writer is a
researcher or works at a university.
Is the author selling something? Do they
have financial interests, an agenda or
lifestyle they’re peddling?
Who recommends the book? Is it a valid
source like your lecturer, fellow researcher,
librarian or multiple recommendations like
reviews.
Does it cite sources? Do they have
citations in footnotes or at the end to
support that their claims are based on
scientific literature.
Is it out of date? Newer up to date
research is because it is cumulative and
self-correcting, so it changes overtime.
Read critically!
Is it any good?
Media/Journalism Sources:
Is it a reputable science news source?
Does it cite the source of the research
(authors, journal)? The best news sources
will provide links or citations to read the
original article.
Does it get opinion from an independent
scientist? An expert in the field’s opinion.
Small peer-review.
Does it confuse correlation and causation?
Language. Do they confuse correlation
with causation. Is their type of science
claim valid or supported?
Does it extrapolate from animals to
humans? Across cultures. Babies to adults
etc.
Read critically!
How do we understand and evaluate primary psychological research?
• Psychology is a science but, • Science is a human activity • Like all human activities, science is fallible • Errors • Biases • Chance results • Published ≠ proven • We must read research critically to identify its strengths and weaknesses.
A critical evaluation consists of understanding:
- What was done
- Why it was done
- What was found
- What this means
If evaluation is done properly, any issues/problems with the study will arise naturally while doing the critical evaluation.
Critical questions:
1) Research Question
broad and theoretical overview of what
the main question the study aims to
answer.
and, but, therefore
2) Hypothesis/Prediction
construct level operationalized into IV-
DV’s
If…. then.
3) Participants
how many? how were they sampled?
were there criteria for inclusion or
exclusion? what are the implications of
these criteria?
4) Procedure
what was the procedure?
5) Variables
theoretical, operational and type
(between or within-subjects)
6) Design
what type of claim does it make? how
were the variables manipulated?
i.e., mixed 2x factor anova
7) Comparisons
looking at the data, is the pattern within
the data what I expected?
8) Analysis
does the data support my hypothesis?
9) Conclusions
What conclusions are drawn?
Are they justified?
What limitations are identified?
What questions remain unanswered?
Four Validities:
- Construct Validity
How well do the operational variables map
onto the theoretical variables?
2. Internal Validity Are there other possible explanations for the findings? Are there confounds? Is it the best design to address the question?
- External Validity
Can the conclusions generalise to other
people, other stimuli, other contexts?
4. Statistical Validity How big is the effect? Does the study have sufficient power? Is the data treated appropriately? Are the statistical conclusions (e.g., significance) justified?
Internal Validity allows us to …. and requires…
Internal validity is needed to make causal claims.
Manipulation of IV and observe change in DV to assess temporal association between variables.
Internal validity is to do with how well we achieve control. We will discuss three ways that researchers may fail at establishing interval validity.
Three main types of internal validity concerns?
- Change over time
- Threats to internal validity which occur as a
result of time. - Selection Effects:
- These arise when two groups are not
matched on extraneous variables. - Expectancy Effects
- Two types of expectancy effects:
Participant vs Experimenter Expectations.
(5) change over time threats to internal validity:
(A) Maturation Effects
- People change naturally over time.
(B) History Effects
- Something in the world changes between
pre-test and post-test (not the people)
(C) Mortality Effects
- People drop out of your experiment.
(D) Testing Effects
- Pre-test biases responses to the
manipulation or responses on the post-
test
(E) Order Effects
- The order in which you do the tasks
matters.
- Practice – people get better over time.
- Fatigue – people get tired or bored over
time.
(4) Selection Effects threats to internal validity:
- Selection Effects:
- Threats to internal validity based on
participant selection, sampling and
assignment to conditions.
- These arise when two groups are not
matched on extraneous variables.
(A) Self-Selection:
- Participants choose their own
experimental condition
(B) Non-Random Assignment
- People don’t have an equal opportunity
to be in each experimental condition.
(C) Regression to the Mean:
- Any group at the extremes of a
distribution will shift to mean over time.
(D) Small Sample:
- Is unlikely to equate groups on all
extraneous variables