WEEK 7) Principles of research critical appraisal Flashcards
Why is it imp to study research design?
3 main things, hint minimise and maximise
- Minimize potential for bias or unclear interpretation
- Maximize efficiency of resource utilization (dont want wasted resources). e.g. draw conclusions which are contrary to whats really going on. Wasted resources, made conclusions about a treatment that could be harmful.
- Minimise ethical issues with subjects
Distinguish btn basic research and applied research.
BASIC RESEARCH: you’re addressing a question but you
dont want to use the results of the study to relevant things in the real world
vs
APPLIED RESEARCH: attempt to apply to real world problems.
Why do we study critical appraisal of research?
Application of scientific method to human studies not always easy
Cannot treat research results as a black box in which we have uncritical faith
Practicing psychologists are the experts and need to decide for themselves whether a given study is to be believed e.g. psychologists need to be able to critically evaluate articles and then maybe apply to real world therapy
What is the scientific method steps for a normal experimental design?
Ask q Do background r construct hypothesis test with experiment analyse results/ draw conclusions Decide whether hypothesis is true or partially true or false Report results Think and try again (go back to construct h).
What is good about the scientific method?
- Conclusions based on quantifiable and reproducible evidence
- evidence-based psychology
- Control of variables other than that of interest ! confidence in attribution of effect
What is the scientific method steps for a non-experimental design?
Have idea do background r pose q collect data Analyse results/ draw conclusions Generate a hypothesis report results think again and go back to pose q.
What is Random sampling?
When every member of the population has an equal probability of selection
What is good about random sampling ?
- then your sample is truly representative.
- And should be easily generalisable.
If youve used random sampling, can you assume its randomised?
Why is this?
In most cases you cant ensure its completely random. Samples may be technically non-random but arbitrary.
In practice, samples are often obtained for convenience, higher chance of being non- representative e.g. all 300 level psych students.
(Applies to surveys as much as experimental studies)
What are four key measurement issues?
List their importance from lowest to highest importance
• Most closely match research question / hypotheses
• Best (psychometric) properties – Reliability/validity
– Minimum variance (you want to minimise error v).
– Most responsive (to experimental factor)
• Most feasible
• Least cost
This is what critical appraisal is not, tell me what it is then…
negative dismissal of any piece of research
balanced assessment of benefits and strengths.
This is what critical appraisal is not, tell me what it is then…
Assessments of results alone
assessment of results and r process
This is what critical appraisal is not, tell me what it is then…
based entirely on detailed statistical analysis
consideration of qual and quant aspects of r
This is what critical appraisal is not, tell me what it is then…
to be undertaken by expert researchers /statisticians
to be undertaken by all health professionals as part of their work.
True or false, good appraisal is inherently retrospective
true
What is CONSORT in critical appraisal of r?
CONSORT: way of assessing areas of a research report.
Ensures various criteria are met. Authors should address
this checklist.. This is used in randomised trials.
What are 10 questions one could ask to help you make sense of evidence / randomised controlled trials?
1. Did the study ask a clearly defined question
2. Is it a randomized controlled trial
3. Subjects appropriately allocated to groups
4. Staff and subjects blinded
5. All subjects who entered accounted for
6. All subjects followed-up consistently
7. Sample size adequate for power
8. How are the results presented and key message
9. How precise are the results
10. Were all important outcomes considered
What are some biases in these appraisal guidelines (the 10 q ones on randomised controlled trials).
- Primarily geared to intervention studies; CBT, pharmacotherapy, …
- Not so much attention given to other forms of research
- Seeking scientific evidence of potential effects attributable to intervention
- RCTs considered the holy grail
Your perspective will affect which aspects of appraisal you give most weight to. You MUST consider the three reasons why you are bothering to read the article. What could they be?
– Research direction
– Clinical practice
(implications for your own clinical practice.)
– Non-specific
(your purpose for reading article will affect the emphasis you place on parts. )
Is it bad to not explicitly state your hypothesis in an intro?
no not necessarily, so long as after the intro you have a clear sense of where the report is going, it should be fine. Even if aims arent explicitly stated.
It is a randomly controlled trial if
the participants are dbl blind =neither the participants nor the researchers know which participants belong to the control group, nor the test group.
e.g. random assignment.
(Double blind means both subject and investigator blinded to treatment allocation)
Should an article explicitly state that all subjects are accounted for?
It should, otherwise it should at least mention drop outs etc. Otherwise its suspicious.
can infer that this is suspicious. They should report
the completion rate e..g if one or two partiicpants dropped out.
Not reported which suggests suspicious missing data.
One must also state that the sample size is adequate for power. What should the report say?
Sample size specified which is good “there were 86 men as participants” but no reference to adequacy of statistical power
True or false you will have more power with a repeated measures design rather than a btn subjects design.
True.
