Surveys/Quest and Systematic review/meta-analysis (2) Flashcards
What is one of the most popular methods for collecting descriptive or subjective data and composed of a series of questions that are posed to a group of subjects?
surveys
Survey data can be used in what type of studies?
ALL THREE
experimental, exploratory or descriptive studies
Surveys can be conducted as what two things?
- Oral interview
2. Written/electronic questionnaire
What is a questionaire?
Structured surveys
self-administered
using pen / paper or electronic formats
Advantages of using a questionnaire: (4)
- More efficient - respondents complete on their own time
- Data gathered from large sample in a wide geo distribution in a short time
- standardized written forms are used - it reduces potential bias from interactions with an interviewer
- useful as a research method for assessing attitudes and values
Disadvantages of using questionnaire: (3)
- not as useful for studying behavior that require objective observation
- potential for misinterpreting questions or response choices
- accuracy or motivation of the respondent unknown
Common methods of distributing questionnaires: (3)
- Through the mail
- In-person distribution
- Electronic distribution (most common, economical)
Major concerns in regard to surveys and questionnaires: (2)
- Return rate low - 30-60% for most studies and lowered further by incomplete, incorrect, or missing data (survey samples large to obtain sufficient usable responses)
- Self-report - always potential for bias or inaccuracy in self-reports
recall bias -
when respondents are asked to remember past events
Self-report is generally valid for variables such as: (4)
- injury
- mobility function
- hypertension
- smoking habits
Self-report is only direct way to obtain info for: (4)
- perceptions
- fears
- motivations
- attitudes
Design of surveys (7 steps):
- Identify questions
- formulate hypothesis
- develop questionnaire outline
- Review existing instruments
- Design instrument
- Preliminary drafts
- Pilot testing and revision
When pilot testing/revising survey, who test it on and serves to do what?
- small rep sample of 5-10 subjects from target population
2. serve to check the validity (face validity and content validity)
How do respondents answer open-ended survey questions?
respondents to answer in their own words
What is open-ended survey questions useful to find?
useful for feelings and opinions without biases/limits imposed by the researcher
open-ended survey questions are often used in what part of the study?
pilot study to determine a range of responses, then converted to a multiple choice item
T/F open-ended survey questions are difficult to code and analyze
True, difficult to code and analyze because so many different responses can be obtained
Closed-ended survey questions -
provide multiple response choices
T/F Closed-ended survey questions are easily coded
True, great uniformity across responses
Closed-ended survey questions should be exhaustive, what does that mean?
they should include all possible responses that can be expected
Closed-ended survey questions should be mutually exclusive, what does that mean?
each choice should clearly represent a unique answer (not 2/4/6 but rather 1-2/3-4/5-6)
What is a scale?
ordered system for a series of questions that provide an overall rating
What were scales devloped to measure
- attitudes
- function
- health and quality of life
- pain
- exertion
- other physical, physiological and psychological variables
Categorical scales -
used with categorical variables such as gender; race; diagnosis
Continuous scales -
used with continuous variables such as age; blood pressure; years of experience
Ordinal scales -
used with ordinal variables such as:
- pain measured by minimal; moderate; severe
- function by independent; minimal assist; moderate assist; maximal assist; dependent
What are the steps to analyzing survey data?
- Enter collected responses into a computer program
- Code responses appropriately
- Analyze the data using appropriate statistical methods (descriptive stats to describe data and inferential stats to make decision about population group)
What type of stats for continuous variables?
mean and standard deviation
What type of stats for categorical variables?
frequency and percent
What is a research design that uses explicit methods to identify, select, appraise and synthesize results from similar but separate studies?
systematic review
T/F systematic review does not focus on a specific question
FALSE it does!
What is considered to be be the strongest level of evidence (gold standard)?
Systematic Review of RCTs
T/F not all review articles are systematic reviews
True
What would be a good source of information on the background of a specific topic?
Traditional narrative review
What is missing from a Traditional narrative review?
it doesn’t include a detailed description of the methods/criteria used to select and evaluate articles that are includes
What are Four steps that differentiate a systematic review from a narrative review?
- identify ALL evidence on the topic
- select them
- appraise the quality of these evidence
- summarize them
What is a statistical method of combining a large collection of results from individual studies called?
Meta-analysis
Meta-analysis develops what?
uses quantitative methods to develop a single overall estimate of the intervention effect
Why is a meta-analysis useful?
useful to provide important results when several smaller studies are not sufficient to demonstrate meaningful outcomes
T/F All systematic reviews use meta-analysis (explain)
False, some do
- it can be very powerful tool for synthesizing information across multiple studies
- it can also be misleading if the studies are not appropriately combined
some don’t
- systematic review may have to be sufficient to qualitatively synthesize the information when measurements are inconsistent or comparisons do not make sense
What are the subjects of a systematic review?
The studies themselves
What types of requirements do selection criteria specify?
Inclusion and exclusion requirements for studies to be used for the review
What are selection criteria based on?
Based on the types of:
- studies
- participants
- interventions
- outcomes
What type of studies are considered gold standard and that authors restrict systematic reviews of interventions to?
RCTs
Put in order from highest level of evidence (1) to lowest:
Case-control/RCT/Case/Cohort studies
- RCTs
- Cohort studies
- Case-control studies
- Case series
T/F Systematic reviews have to set reasonable ranges for subject characteristics
True, because study populations can be so variable
Selection criteria based on types of interventions must do what?
indicate definitions of interventions and comparison treatment (if relevant) because studies are rarely consistent in the application of interventions
Selection criteria based on types of outcome measures must do what?
must indicate the choice of outcome measures because included studies will often have a variety of outcomes
Mean difference as type of outcome measure -
effect measures for randomized controlled trials
Odds ratio (OR) / Relative Risk (RR) as type of outcome measure -
measures of association for observational studies
Standardized Effect Size (SES) as type of outcome measure -
– it evaluates how big or small an effect is when the units of measurement aren’t intuitive
– it compares the results across studies
What is the goal when searching for studies?
The goal is to find a comprehensive list of relevant documents that should be considered (must indicate in report which databases were used and specific search terms entered)
Issues of publication bias with available literature: (@)
Issues are:
- researchers may fail to submit the results of studies where the results are not statistically significant
- editors may decline to publish such studies
- > bias against negative or inconclusive results
Why must Evidence of no effect must be part of the picture?
because the value of systematic review is to broaden the evidence for or against a proposed outcome
What are 4 possible bias in conducting a systematic review?
- Selection bias
- Performance bias
- Attrition bias
- Detection bias
Include studies that adhere to the _____ for clinical trials or the ______ for observational studies
consort statement for clinical trials
strobe statement for observational trials
What rating scales can be used to assess the quality of a study? How many reviewers?
JADAD scale / PEDro scale / QUADAS scale
-> at least 2 primary reviewers will independently assess and rate
T/F When synthesizing the data, you must determine the degree of heterogeneity or homogeneity in the included studies
True, [Heterogeneity – dissimilarity in specific aspects of the studies]
What is inclused in the discussion and conclusions section?
- overall conclusions based on the quality of evidence that was obtained
- Integrate the findings to clarify the state of knowledge in a clinical context
Meta-analysis results are presented in what type of plot? Helps to do what?
Forest plot - Helps combine the results quantitatively and obtain a single summary result
Forest plot can answer questions of:
- what is the direction of effect
- what is the size of effect
- whether the effects are consistent across studies
T/F Forest plot is only as good as the studies in it
True
Where can you find find independent, high quality evidence of systematic review
Cochrane Library
- with topics including movement disorders, wounds, stroke, neuromuscular disease, back problems, developmental and learning problems, bone and joint trauma, and HIV/AIDS
- with rigorous standards developed to ensure these reviews provide accurate and thorough appraisals of the literature
