Critical Evaluation Of Research Final Flashcards
What is Evidence-Based Practice, EBP, (3)?
- External Research Evidence
- Internal Clinical Experience
- Fully Informed Patient Preference
What is the scientific method (5)?
Step 1: Observe behavior
Step 2: Form a Hypothesis:
a tentative explanation of the observation
Step 3: Rational Method:
generate a testable prediction (based on the hypothesis)
Step 4: Data Collection:
evaluate the prediction with systematic, planned observations
Step 5: Support or Nullify:
use the observations to support, refute, or refine the original hypothesis
Describe what it means to say that science is empirical, public, and objective.
3 important PRINCIPLES of the Scientific Method:
- Empirical: answers are obtained by making observations.
(science requires empirical verification to be accepted) - Public:
• AVAILABLE: observations are available for evaluation by others.
- REPLICATION: repetition of observation allows verification of the findings.
- PEER REVIEW: evaluated by the researcher’s peers in scientific journals
- EVALUATE: erroneous concl. can occur either by error, chance, or fraud
- Objective:
“The dispassionate search for knowledge”
(the observations are structured so that the researcher’s biases and beliefs do not influence the outcome of the study)
Hypothesis
A TENTATIVE EXPLANANTION that is intended to be TESTED and CRITICALLY evaluated.
Must be:
• Testable
• Refutable, and
• Positive Statements (existence of change/new),
bc you cant conclude something doesn’t exist bc it wasn’t found
4 Foreground Elements
P (patient/problem)
I (intervention)
C (comparison/control)
O (outcome/effects)
In a research paper, where do you find:
How many individuals participated in the study, and what are their characteristics?
Method: Participants
In a research paper, where do you find:
Why was the study done?
Introduction: Literature Review
In a research paper, where do you find:
Did the study produce a statistically significant result?
Results: Data
In a research paper, where do you find:
The limitations of the study and directions for future research?
Discussion: Limitations
Humans are called _____________
Participants
Nonhumans are called ____________
Subjects
Do studies PROVE results?
NO! evidence only SUPPORTS findings (for or against)
Science is ___________ and ___________
incremental & cumulative
What is the Nuremberg Code?
A set of 10 guidelines for the ETHICAL treatment of human participants in research
What are 4 major ethical concerns/issues with research?
- No Harm:
• Must outline risk/benefit
• Benefit must be greater - Obtain Informed Consent:
• Purpose of study (blinding?)
• Right to decline participation @ any time
• Risk/Benefits
• Provide contact info for concerns - Use of Deception & Debriefing
• Passive Deception (withholding/omitting)
• Active Deception (misinformation/telling
lies), uses Confederates (in on the study)
• Debriefing, after study concludes, explanation/
purpose of the study to participants to
counteract/minimize harmful effects by deception - Confidentiality & Anonymity
• Confidentiality (info obtained is kept secret
and private)
• Anonymity (name is not linked/used w/ info
obtained @ any point)
Name several places where you can search for peer-reviewed articles
- ASHA
- Cochrane Library
- Campbell Collaboration
- MEDLINE
- NYU Library
- PLOS
- PsycINFO
- Scopus
Define Construct
• An intangible broad CONCEPT/IDEA/TOPIC
that helps EXPLAIN or PREDICT behavior in a theory
e.g. self-esteem, motivation, hunger, beauty, intelligence,
personality, learning, gravity, pain, anxiety (abstract)
• Can be influenced by external stimuli (variables) and, in
turn can influence external behaviors (dependent).
e.g.,
Stimulus –> Construct –> Behavior
(reward) –> (motivation) –> (Performance)
Define Variable
- Factors/Characteristics/Conditions that influence a construct
- A measurable representation of a construct
Give examples of Abstract Variables
intangible, not directly observable
- thoughts
- feelings
- perceptions
Give examples of Well Defined Variables
(easily observed and measured)
- age
- sex
- race
- height
- weight
Define Operational Definition
• Defines constructs in terms of HOW they will be
empirically measured
e.g., temperature (construct) will be measured by
Fahrenheit (O.D.)
______________ is demonstrated when scores obtained from a new measure are directly related to scores obtained from an established measure of the same variable.
Concurrent Validity
______________ is demonstrated when scores obtained from a measure accurately predict behavior according to a theory.
Predictive Validity
______________ requires that the scores obtained from a measurement procedure behave exactly the same as the variable itself.
This type of validity is based on many research studies that use the same measurement procedure and grows gradually as each new study contributes more evidence.
Construct Validity
Define Validity of Measurement
The degree to which a measurement procedure is measuring what it claims to measure
Define Reliability of Measurement
The stability or consistency of a measurement
Describe the relationship between Validity and Reliability as they pertain to measures
Validity requires reliability (not valid w/o it) BUT
Reliability does not require validity
WHY?
Bc the concept of reliability is based on the assumption that the variable measured is stable/constant.
Why are Operational Definitions sometimes necessary.
When you need to convert an ABSTRACT variable into a CONCRETE entity that can be observed, studied, and replicated
How may you operationally define
RECEPTIVE VOCAB.
• PPVT-5 (Peabody Picture Voc. Test)
• ROWPVT (Receptive One-Word Picture Vocabulary
Test)
- Y/N Questions
- Story Retell
How may you operationally define
EXPRESSIVE VOCAB.
- EVT-2 (Expressive Vocab. Test)
- EOWPVT-4 (Expressive One-Word Picture Vocab. Test)
- Picture Naming
- MLU
- Narratives
How may you operationally define
HOARSENESS
- altered vocal quality
- laryngeal tenseness
- loudness
- pitch
- vocal effort
- Perception of audible aperiodicity in sound (raspy)
How may you operationally define
BREATHINESS
- altered vocal quality
- laryngeal tenseness
- loudness
- pitch
- vocal effort
- Perception of audible air escape
Define Test-retest reliability
The CONSISTENCY of SCORES when administered on MULTIPLE OCCASIONS (same test multiple times)
Define Inter-rater reliability
The consistency of scores when DIFFERENT OBSERVERS measure (or score) the measurement (degree of agreement between raters)
A SLP has developed a new test for measuring receptive language and would like to determine the validity of the test. The new test and an established measure of receptive language are both administered to a sample of participants
Describe the pattern of results that would establish concurrent validity for a new test.
A POSITIVE correlation pattern between the new test and the well-established test would establish concurrent validity (and an UPWARD direction in a scatterplot).
What are the 4 SCALES of measurement and give examples?
- Nominal:
CATEGORIES that represent QUALITATIVE differences
• Gender (1 = F, 2 = M)
• Subject group (1 = CWS, 2 = CWNS) - Ordinal:
CATEGORIES that have DIFF NAMES, but organized
SEQUENTIALLY (Directional relationship)
• Academic Grades A-F (Ranked Data)
• Surveys 1 -5 from agree to disagree (Likert Scales) - Interval:
CATEGORIES organized SEQUENTIALLY and all
Categories/intervals are the SAME SIZE
• Temperature
• SAT scores - Ratio
CATEGORIES organized SEQUENTIALLY and all
Categories/intervals are the SAME SIZE
• Height
• Weight
Interval Scale
• Has an ARBITRARY zero point (e.g., temperature, year)
Ratio Scale
• Has a TRUE zero (e.g, % of verbs uttered or # of SS)
What is a ceiling effect
The clustering of scores at the HIGH end of a measurement scale
How can a ceiling effect be a problem?
It hinders the ability to measure changes with little or no possibility of increases in value.
What are demand characteristics?
POTENTIAL CUES or features of a study that:
- SUGGEST to the participants what the PURPOSE/HYPOTHESIS IS
- INFLUENCE the participants to RESPOND/BEHAVE in a certain way
How do Demand Characteristics limit the Validity of the measurements obtained in a research study?
It triggers participant REACTIVITY, which modifies their natural behavior in response to the study and thereby INFLUENCES the MEASURES they PRODUCE
What is a population?
The large group of interest to a researcher
What is a sample?
• The small set of individuals who participate
in the study
• Must be representative of the population