EVP6 D&B Tool

Question 1

Is the hypothesis/aim/objective of the study clearly described?

Answer 1

Y/N

No hint

Question 2

Are the main outcomes of the study clearly described?

Answer 2

Y/N

If the main outcomes are first mentioned in the Results section, the question should be answered no.

Question 3

Are the characteristics of the patients included in the study clearly described?

Answer 3

Y/N

In cohort studies and trials, inclusion and/or exclusion criteria should be reported. In case-control studies, a case-definition and the source for controls should be provided

Question 4

Are the main findings of the study clearly described?

Answer 4

Y/N

Simple outcome data (including denominators and numerators) should be reported for all major findings so that the reader can check the major analyses and conclusions. (This question does not cover statistical tests that are considered below)

Question 5

Does the study provide estimates of the random variability in the data for the main outcomes?

Answer 5

Y/N

In non-normally distributed data, the inter-quartile range of results should be reported. In normally distributed data, the standard error, standard deviation, or confidence intervals should be reported. If the distribution of the data is not described, it must be assumed that the estimates used were appropriate and the question should be answered yes.

Question 6

Have actual probability values been reported (e.g. 0.035 rather than <0.05) for the main outcomes except where the probability value is less than 0.001?

Answer 6

Y/N

If no p values are presented, the question should be answered ‘no’. If p values presented and there is a mixture of reporting (some presented as < or > specific figures, some as equality, e.g. p = 0.034), question should be answered ‘yes’

Question 7

EXTERNAL VALIDITY

Were the subjects asked to participate in the study representative of the entire population from which they were recruited?

Answer 7

Y/N/Unable to determine

• The study must identify the source population for patients and describe how the patients were selected
• Patients would be representative if they comprised the entire source population, an unselected sample of consecutive patients, or a random sample.
• Random sampling is only feasible where a list of all members of the relevant population exists
• Where a study does not report the proportion of the source population from which the patients are derived, the question should be answered as unable to determine

Question 8

EXTERNAL VALIDITY
Were those subjects who were prepared to participate representative of the entire population from which they were recruited?

Answer 8

Y/N/Unable to determine

• The proportion of those asked and agreed to participate should be stated
• Validation that the sample was representative would include demonstrating that the distribution of the main confounding factors was the same in the study sample, and the source population

Question 9

INTERNAL VALIDITY

Were the statistical tests used to assess the main outcomes appropriate?

Answer 9

Y/N/Unable to determine

• The statistical techniques used must be appropriate to the data
• For example, non-parametric methods should be used for small sample sizes
• Where little statistical analysis has been undertaken but where there is no evidence of bias, the question should be answered ‘yes’
• If the distribution of data (normal or not) is not described, it must be assumed that the estimates used were appropriate and the question should be naswered yes

Question 10

INTERNAL VALIDITY

Were the main outcome measures used accurate (valid and reliable)?

Answer 10

Y/N/Unable to determine

• For studies where the outcome measures are clearly described, the question should be answered yes
• For studies that refer to other work or demonstrates the outcome measures are accurate, the question should be answered as yes