Study Designs

Question 1

What are case reports and series studies?

Answer 1

A case report is the most basic type of descriptive study and documents an individual’s medical experience. A clinician may see an interesting case and describe what he or she has seen.

A case series is an extension of a case report and is based on a small group of individuals. Case reports and case series are useful for hypothesis generating, but because the is no comparison if group, their use is limited and statistical relationships between exposure and outcome cannot be assessed.

Question 2

What are the advantages of case reports and series?

Answer 2

Advantages:

• allows reporting on unusual medical cases
• results used to generate hypotheses
• generates evidence of possible new diseases
• rapid feedback of current events int he medical community

Question 3

What are the disadvantages of case reports and series?

Answer 3

Disadvantages:

• cannot be used to assess statistical associations
• could just be reporting a medical oddity

The rest of the study designs can all look at associations between exposure (or treatment) and outcome/disease by comparing outcome between different levels of exposure.

Question 4

What are ecological studies?

Answer 4

Ecological studies look at the association between exposure an disease on a population or area level rather than on an individual level. They look at questions like ‘do populations or areas with high levels of exposure have high rates of disease?’, rather than ‘do individuals with higher exposure have a higher risk of disease?’

For example to look at the association between smoking and cardiovascular disease, data were collected for every state in the US on ‘average number of cigarettes sold per person’ and ‘rate of cardiovascular mortality’. It was found that death rates from cardiovascular disease are highest in the states where the greatest amount of cigarettes are sold.

The benefit of ecological studies is that data on a population level are often readily available and published routinely (eg death rates, national food consumption data, cancer statistics, hospital admission data, census data etc), and such studies can therefore be done quickly and inexpensively.

A drawback is confounding which can be a major problem and data on potential confounders are often not available. For example, those states with high rates of cardiovascular mortality may share many other characteristics than just high levels of smoking, eg high social deprivation, poor diet, high proportion of elderly etc, any one of which could be the true explanation for the high rates of cardiovascular mortality.

Question 5

What is meant by ecological fallacy?

Answer 5

With ecological studies any associations seen are on a population level, and we cannot assume that this transfers to an individual level. To assume that associations seen one population level apply to the individual level is called the ecological fallacy. The unit of analysis is a population and it is at this level that the interpretation should be conducted. In the smoking and CHD example, we cannot assume the individuals who smoke are the ones who are more likely to die from cardiovascular disease.

Question 6

How should you go about analysing data from ecological studies?

Answer 6

Data from ecological studies are analysed on a group level and generally presented in a scatter diagram with exposure on the x-axis (horizontal axis) and outcome on the y-axis (vertical axis). Each point on the scatter diagram represents an area/population. A correlation coefficient (r), which can vary between +1 (perfect positive correlation) and -1 (perfect negative correlation), is then computed.

Ecological studies are a good first step in investigating possible exposure-disease relationships, especially when there are restraints on time or money, and generating hypotheses. They are also good when investigating an exposure which has little variation between individuals within a population/area, but large variation between populations/areas (eg some dietary factors). However any relations seen need to be investigated further in an individual-based study where data on confounders are collected.

Question 7

What are the advantages of ecological studies?

Answer 7

Advantages:

• inexpensive and quick to conduct
• exposure and disease information is often more readily available by area
• differences in exposure is often greater between areas than between individuals in one area

Question 8

What are the disadvantages of ecological studies?

Answer 8

Disadvantages:

• results cannot be extrapolated to the individual level
• systematic differences between areas in recording disease frequency can occur
• quality of diagnosis
• differences in classification
• completeness of reporting
• sampling on the population can distort the results of an individual
• data often not available on confounders

Question 9

What is a cross-sectional study?

Answer 9

A cross-sectional study collects observations on individuals at one point in time, thereby providing a ‘snap shot’ of the health of the population. These may be observations on disease status to measure prevalence of disease, or some continuous measure such as blood pressure, level of protein in serum etc. As people are surveyed at one time point only, cross-sectional studies are relatively cheap but only provide information on disease prevalence and not incidence. Study subjects should be selected so they are representative of the target population, eg if target population is adults in Nottingham, the study population may be defined as all adults registered with a GP in Nottingham, and a 1 in 4 random sample may be taken from the GP registers to provide the sampling population. Data on exposure variables are usually collected as well so that associations between exposures and disease can be explored. Confounding can occur in this study design, but as long as data on potential confounders are collected, they can be dealt with at the analysis stage (stratification or multiple regression).

Cross-sectional studies only consider prevalent case of disease (ie current cases) so any risk factors identified will be determinants of ‘having the disease’, of which survival as well as incidence are components. For example a cross-sectional study showing deprived people have a lower prevalence of heart disease than more affluent people, does not necessarily imply they get less disease. It may be that they develop heart disease at the same rate but the deprived people don’t survive as long.

Question 10

How should you go about analysis of cross-sectional studies?

Answer 10

The outcome variables should be summarised, using statistics appropriate to the type of variable. Associations can be initially assessed by computing an appropriate measure of effect (odds ratio, mean difference etc) and 95% confidence interval, and statistical significance determined from the appropriate test (eg chi-squared test, t-test, non-parametric test etc).

Question 11

What kinds of things are cross-sectional studies appropriate for?

Answer 11

This method is appropriate for investigating some health outcome of interest eg prevalence of a disease, and as a first step in identifying risk factors for a disease.

It is not suitable when the outcome or exposure of interest is rare as you may end up with very few (or even no) people in your sample with the outcome/exposure and therefore cannot determine associations. More than one disease or exposure can be assessed in the same study, although as with any study, an a priori primary hypothesis should be stated. Changes in prevalence can be assessed by carrying out a series of cross-sectional studies. This study design is not suitable for looking at incidence or natural history of a disease.

Question 12

What are the advantages of cross sectional studies?

Answer 12

Advantages:

• can be used to examine how much disease there is in a population, and look at cross-sectional associations between exposure and disease
• can look at more than one disease and more than one exposure
• can be relatively inexpensive and quick to conduct and drop out is not a problem as no follow up

Question 13

What are the disadvantages of cross-sectional studies?

Answer 13

Disadvantages:

• disease and exposure are measured at the same time therefore no temporal association can be made
• not suitable for studying rare exposures or rare outcomes
• high possibility of recall/reporting bias

Question 14

Describe what case-control studies are.

Answer 14

The case-control study is a useful study design as is it suitable for looking at risk factors for rare diseases. However, it cannot look at how much disease there is (prevalence/incidence), only at whether associations with exposures exist.

In a case-control study subjects are selected on the basis of the presence or absence of disease. A group of individuals with the disease (cases) are selected, along with a comparison group of individuals without the disease (controls). This method of sampling reduces the number of disease-free people needed to be studied (hence good for rare diseases). The exposure of interest is then measured in the two groups (this may be past or current exposure) and compared. The effect of exposure on the risk of disease is estimated using the odds ratio.

Disease frequency cannot be measured because subjects are chosen or samples according to their disease status. To obtain a prevalence estimate, a cross-sectional study is needed, and to obtain an incidence estimate a cohort study should be conducted.

Question 15

When is a case-control study appropriate?

Answer 15

A case-control study is appropriate when you have a single disease of interest that may be rare, and you want to look at associations with one or more exposure(s) that are relatively common.

Question 16

Describe what things need to be considered when choosing your cases for a case-control study.

Answer 16

When choosing your cases the following needs to be considered:

• what are the case definition criteria? In other words what criteria will be used to define the outcome / disease of interest, eg clinical diagnostic criteria, laboratory results, coding on a death certificate etc. A poor choice could result in misclassification (ie cases not all truly having the outcome or controls including some diseased).
• what are the eligibility criteria for the selection of individuals for the study, eg are you interested in just adults or just children. The criteria may be to restrict the study to those potentially at risk of the outcome of interest - for example if you were studying cervical cancer, then you would only want to include women. Criteria may also be chosen to restrict to individuals who were potentially at risk of exposure - for example, if your exposure was oral contraceptive use, you may exclude women who are post menopausal or pregnant.
• what is the source of the cases? Case control studies are usually either hospital-based or population-based. In a hospital-based study we might choose our cases from those attending a particular hospital. In a population-based study cases are taken from a defined population (eg geographical area) over a fixed period of time. Population based studies are often more difficult to conduct, and to identify all the cases in a particular population it may be necessary to use more than one source. The choice of hospital or population based will often depend on the severity of the disease of interest. Remember that the cases in your study should be representative of all those with the disease in you target population. So suppose you want to study the association between an exposure and disease in Nottingham city, choosing cases from one health centre will not be representative of all cases in Nottingham.
• Are prevalent or incident cases to be included? Prevalent cases are all those with disease at a particular point (or short period) in time. Incident cases are new cases that arise within a fixed period of time. Prevalent cases differ from incident cases in that they will include individuals who have had the condition for some time. Such cases are likely to be the ones that survive longer or take longer to recover. Furthermore they may have changed their exposure (eg diet, exercise, smoking habit) because of the disease diagnosis, leading to incorrect ascertain end of exposure.

Question 17

Describe what needs to be considered when choosing your controls for a case control study.

Answer 17

When choosing your controls, the following need to be considered:

• controls should be free from the disease of interest
• controls should fulfil the same eligibility criteria as cases. For example, if cases were restricted to pre-menopausal women aged 18 to 45, then controls should also be pre-menopausal women in this age group.
• the source of the control group, which is not always obvious, is dependent on the source of the cases. Controls are often chosen wrongly and this is where selection bias is introduced. The basic rule is that controls should be drawn from populations that gave rise to the cases. In other words, if the control had developed the disease, he or she would have been included in the study as a case.
• population based controls: if the cases are a population based sample of all incident cases over a specific time period, the. Controls should be selected from this same population during this time period. If the cases are identified through hospital admissions or other health facility, and the hospital has a defined population base and sees all cases in that population, then again, that population would provide the controls.
• hospital controls: often a hospital does not have a defined population base from which the cases it sees come. If there is some selection process that affects whether a case reaches the hospital or not, then it may be more appropriate to choose your controls from other hospital patients. When choosing hospital controls, special care needs to be taken. The purpose of controls is to represent exposure in the population the cases came from, but it is easy to inadvertently pick a group of hospital patients as controls who have a particularly high or low prevalence of exposure. For example, if we were looking at the association between alcohol consumption and liver disease, choosing controls from A and E (who are likely to be higher alcohol consumers than the general population) would give us a biased result. As a general rule, the selection of hospital controls should exclude those individuals who are identified by medical conditions (or backgrounds) that are known to be associated with the exposure of interest.
• sometimes researches include more than one control group, eg a community based control group and a hospital based control group. This can be tempting to do when it is not obvious who the controls should be. However this can lead to problems with interpretation if the two analyses produce different results.

Question 18

What are the possible problems that may be associated with case-control studies?

Answer 18

Because in case control studies the disease status is already known at the time exposure data is collected, information bias can be a problem. As described for cross sectional studies, recall or reporting bias can be introduced. Another type of information bias common in case control studies is observer or interviewer bias. This arises when the investigator knows who cases are, which influences the way in which data is collected. For example, the interviewer may probe more deeply for information or prompt the respondent if the subject is a case. To overcome this, the investigator should be blind to the hypothesis under study and the case/control status of each subject, and the same forms/questions should be used for cases and controls.

As with cross-sectional studies, reverse causation is a possibility. Often data on past exposure are collected which helps eliminate the possibility, but without data on the timing of exposure and disease onset, it is difficult to eliminate completely.

Question 19

Describe how you should go about analysing case-control data.

Answer 19

Data from case control studies are initially analysed by cross-tabulating the outcomes (case-control status) against the exposure.

When computing percentages previously we have computed the percentage diseased on each exposure group. Whilst this makes sense for data sets in which the subjects have been randomly selected (eg cross sectional survey or trials), case control studies are different because of the way in which the diseased and disease-free are selected. Therefore in case-control studies, we actually want to know about the percentage exposed amongst controls, not the percentage diseased amongst exposed and unexposed. To test statistical significance the chi-squared test is used since the variables are categorical.

The only measure of effect suitable for case-control data is the odds ratio. This is because the design of case-control studies means that the risk of disease, and hence the risk ratio, cannot be estimated. The odds ratio and 95% confidence interval around this should also be computed to tell us how precise this estimate is likely to be. This can be done in SPSS, but remember to always compute the odds ratio first by hand and check they match with that displayed in SPSS (and if necessary recode to get the cross-tab table on the right format, or take reciprocals if the values displayed). Odds ratios can be interpreted in the same way as risk ratios.

I’m most studies there are potential confounders that no to be considered since they may be distorting the relationship between the exposure and disease. So what we would actually like is an estimate of the odds ratio which has had the effect of the confounder(s) removed. In other words, we want an adjusted odds ratio. There are sophisticated statistical ways of getting an adjusted odds ratio called multivariate models. When the outcome is binary, such as in case-control studies, multiple logistic regression is the appropriate multivariate method.

Question 20

What are matched case-control studies?

Answer 20

Often in case-control studies all identified cases are selected for inclusion and a random or systematic sample of potential controls. However, sometimes controls are selected so they are matched to a case. This involves the pairing of one (or more) control to a case based on specific variables (other than those under investigation) such as age, sex, or place of residence. The specific variables are factors thought to be confounders in the relationship between exposure and disease.

There are two types of matching, individual matching where each case is individually matched to one or more controls based on matching variables, and frequency matching where controls are chosen to ensure roughly the same number of controls fall in each category of the matching variable as there are cases.

Matching is carried out to help control for confounding, but this can only be achieved if matching is accompanied by a matched analysis. The analysis needs to take account of the matching.

The result of matching and carrying out a matched analysis is that the precision of the estimated odds ratio can be increased. There is no simple answer as to whether to match or not. However matching without properly thinking it through can lead to complications, and if we match in a variable, we cannot then look at the association between that variable and the disease.

Question 21

What are the advantages of case-control studies?

Answer 21

Advantages:

• there is no need to follow people up over time, which can be costly, time consuming and subject to drop outs
• it reduces the number of disease free people that need to be studied. This is particularly important when the disease of interest is rare
• it is possible to study multiple exposures

Question 22

What are the disadvantages of case-control studies?

Answer 22

Disadvantages:

• no estimate of disease frequency can be obtained
• bias can be a problem, particularly selection bias
• it is possible to look at one outcome only

Question 23

What are the 6 main types of study design?

Answer 23

The main types of study design are:

1) . Descriptive: case reports and case series
2) . Ecological
3) . Cross-sectional
4) . Case-control
5) . Cohort (longitudinal)
6) . Intervention or clinical trials

These designs are ranked by power according to the level of evidence that they provide, from case reports/series which are simply a description of cases, to the gold standard design, the randomised controlled trial, which provides the strongest evidence of an association. The first 5 designs are called observational studies and the last is an experimental design. Within this section we will cover case-control studies, cohort studies and clinical trials in the most detail.

Question 24

Describe cohort studies.

Answer 24

Cohort studies measure incidence of disease. Since incidence measurements are considered as the gold standard in epidemiology, these studies are held in high esteem. In the traditional prospective cohort study, the study population is ‘free of disease’ at the beginning and the exposure variables are measured and then the population is followed through time to determine their disease outcome, and to compare the risk of disease in those who are exposed and unexposed.

Question 25

Contrast to cohort and case control study designs.

Answer 25

Cohort studies involve more time and effort than the case control studies. Case control studies have the advantage that many more cases can be collated over a much shorter period of time since you are identifying cases with, and controls without, the disease at the beginning of the study, and the interviewing them about previous exposures. Additionally, the total number of people that need to be recruited and interviewed in a case control study is a lot less than would be needed for a cohort study, since a cohort study needs to have sufficient numbers of events (people with disease) at the end of the study by ensuring enough people are recruited at the beginning. However, the cohort study has the advantage that more detailed information can be collected about the exposures of interest at the beginning and throughout the study by having frequent follow-ups. Additionally, the effects of the exposures can be assessed for any particular disease, or for all diseases, since other diseases. May be recorded at each follow up.

Question 26

Describe alternative designs for cohort studies.

Answer 26

The design of cohort studies can vary quite a bit.

Longitudinal studies can be formed through a series of cross sectional surveys recruiting the study population to participate over time. For example, the 1970 British birth cohort study for which exposure and outcome data were collected just at birth, and ages 5, 10, 16 etc. These cohorts don’t truly give us a measure of incidence, but give us measures of prevalence. The 1970 British birth cohort started with 17,000 children at birth, 13,000 of whom were followed up at age 5, about 11,000 at age 10 and just over 7,000 at birth, 5, 10 and 16. This shows one of the common problems of (prospective) cohort studies; loss to follow up. We have to think carefully, and assess as best we can, what bias might have been introduced through loss to follow up.

One way to overcome the length of time it takes to conduct a cohort study is to conduct what is sometimes called a retrospective cohort. In this type of study we would define a cohort for which there is existing data on exposure and disease. For example, consider researchers who were interested in the effect of metal working on a lung disease called CFA. To avoid having to follow up a cohort of metal working for a long period of time to wait for disease to develop, they actually used existing death records on a (occupational) cohort of metal workers, and looked retrospectively at their exposure and their incidence (mortality) of CFA.

The term retrospective and prospective are often used in describing cohort studies and these terms can provide some confusion, therefore it is best to describe the methods used to collect information to avoid confusion. The term retrospective is generally used when the exposure and outcome have already occurred when the study has started. This contrasts with prospective studies where the exposure may or may not have occurred but the disease outcomes have not occurred yet. Retrospective design or historical cohorts can often be conducted more quickly and less expensively, especially for diseases that have long latency periods between exposure and disease. In these cases it can be difficult to obtain accurate exposure data; because this data generally needs to be collected from pre-existing records and they do not always contain complete and accurate information resulting in misclassification of the exposure. Another problem with historical cohorts is attaining measurements of confounding variables because this information is not always available.

Occupational cohorts can be a good way of examining the effects of rare exposures, for example those that occur in occupational setting such as working with a certain chemical. In the example above we used a cohort of metal workers to look at the effects of exposure to metal dust on lung disease. Careful thought needs to be put into choosing an appropriate comparison group in these situations. A working population is a healthier population than the general population, simply because they have the ability to work. The general population contains people who are healthy and working and people who are too ill to work; therefore the working population has a lower total mortality and morbidity than the general population. The general population has higher rates of disease than the working population, and when comparing rates of disease between an occupational cohort and the general population it will underestimate the effect. This is known as the healthy worker effect.

Another type of cohort design is set up with no one specific exposure in mind, such as the Framingham Heart Study. Exposure is determined by a whole series of baseline measurements such as blood pressure level or amount of physical activity. In the Framingham study, the community was chosen because it had a broad range of occupations and it was a population that was considered stable, so it was easier for a long term follow up period. This community also had one major hospital that was utilised by the majority of the population. Another e ample is the Black Women’s Health Study, which started from questionnaires sent out to all the subscribers of the magazine aimed toward black women to assess health status of black women and has now evolved to a cohort of approximately 65,000 black women that are being followed up for a number of disease outcomes and a variety of exposures.

Question 27

What are the possible sources of bias in cohort studies?

Answer 27

As with any study all possible bias should be assessed and commented on in the discussion of the paper. Selection bias is not as big a problem for cohort studies since the selection of subjects occurs before the disease develops.

Two possible sources of bias in cohort studies are loss to follow-up or ascertainment bias and the healthy worker effect.

Question 28

What are the advantages of cohort studies?

Answer 28

Advantages:

• measurement of incidence of disease
• multiple disease outcomes can be examined relating to one exposure
• often multiple exposures for one disease can be examined
• exposure is measured before the development of disease so we can assess the temporal relationship between exposure and disease
• good assessment of level / duration of exposure and therefore allows us to look at dose response relationships between exposure and disease
• extremely useful for measuring the effect of rare exposures
• selection / recall bias bound potentially be very low

Question 29

What are the disadvantages of cohort studies?

Answer 29

• prospective cohort studies are time consuming and extremely expensive
• retrospective studies rely on availability of good and accurate records
• results can be effected due to loss of follow-up (response rate)
• can be inefficient for evaluation of rare diseases
• can be inefficient for exposure / disease relationship with a long latency period (ie it takes a long time for the exposure to influence disease or for the disease to develop after exposure)
• ascertainment of outcomes could be influenced by exposure

Question 30

How can we go about measuring incidence for cohort or longitudinal studies? What are the two different ways of expressing the incidence of disease?

Answer 30

Cohort studies measure the incidence of disease. Measures of incidence are concerned with quantifying the number of new cases of disease within a population over time. Incidence measures are only derived from cohort or longitudinal studies. There are two ways of expressing the incidence of disease, as a risk or rate, and the difference between them is important.

Question 31

Describe how we can measure incidence risk.

Answer 31

Incidence risk is the probability that a disease will occur within a defined healthy population over a set period.

Incidence risk = number of new cases during a time period / total population initially at risk

If our initial population contains 100 healthy people all of whom are followed up for exactly 1 year and after that time 10 have the disease then the incidence risk is 10/100 ie 0.1. In order to calculate the risk we need to have a cohort of people followed for a set period of time. Such ‘closed cohorts’ occur in clinical trials but in epidemiological studies they are unusual. In observational studies it is more usual that people will be recruited into cohorts at different time periods and followed up for different times - so called ‘open cohorts’.

Question 32

Describe how we measure incidence rate.

Answer 32

With open cohorts we cannot calculate risks, however, we do the time that each person has been followed up for and whether or not they got the disease. With these data we can calculate the incidence rate, a measure of incidence that relates the number of new cases to the person-time at risk, thereby taking account of changes in the size of the population during the follow-up period.

Incidence rate = number of new cases of disease / total person-time at risk

The denominator is the sum of the time each person in the study remains at risk during the study period.

Incidence rates are typically presented as per 100 or 1000 or 10000 person years, as the number per person-year is usually very small.

With this method we can also include data for people lost to follow up assuming we know when they were last followed up and that they did not have the outcome at the last point of follow up.

Question 33

What is the effect measure used for cohort studies which look at incidence rates?

Answer 33

For cohort studies which look at incidence rates, the effect measure used is the rate ratio:

Rate ratio = rate of disease amongst exposed / rate of disease amongst unexposed

The rate ratio tells you how much more likely the exposed group are to develop the disease than the unexposed group. So again 1 means no association, anything greater than 1 means the exposed have a higher rate of disease, and anything less than 1 means the exposed have a lower rate of disease, ie exposure is protective.

Question 34

In longitudinal studies such as cohort studies what confounding may exist?

Answer 34

In longitudinal studies such as cohort studies, confounding may exist because the exposure of interest may also be associated with other (confounding) exposures that are also risk factors for the disease. For example, a diet high in vegetables and fruits may appear to protect agains colorectal cancers in an observational study, but this may well be because non-smokers eat more fruits and vegetables than smokers - here smokers is the confounder.

Question 35

Describe randomised controlled trials.

Answer 35

Controlled trials (also called clinical trials, field trials, intervention studies) are human experiments where the investigator implements the intervention (or exposure) of interest and determines which individuals are exposed and which are unexposed.

Randomisation is often used within trials to ensure that the exposed and unexposed groups are similar with respect to all other factors (including other risk factors for the outcome). A randomised controlled trial (RTC) is thus equivalent to a laboratory experiment where an investigator changes one variable at a time keeping all other variables constant.

In observational studies, it is possible that an observed association between an exposure and an outcome is due to differences between the exposed and unexposed groups with respect to other risk factors for the outcome (eg if an observational study found that high exercise levels were associated with a reduction in lung cancer, an alternative explanation could be that people who exercise a lot are more likely to be non smokers than those who do less exercise). We must adjust (or control) for these confounders in the design and analysis of observational studies, however it is difficult to remover their effects completely. It is often difficult to obtain precise measurements of known confounders, so there may be residual confounding even after adjustment. There may also be unknown confounders that have not yet been identified and it is not possible to control for these. The randomisation part of a clinical trial enables one to overcome confounding and as a result of this and other advantages, they give a higher level of evidence than observational data, which makes them central to driving changes in public health policy. Therefore, they are the main tool of evidence-based medicine because they have the potential to establish causation. Trials are often the last step in the research chain after observational studies have established hypotheses of interest. It is important to remember, however, that RCTs cannot be used for all questions in epidemiological research.

Question 36

How do clinical trials fit into epidemiology?

Answer 36

Ultimately the aim of epidemiology is to improve the health of the public and the research should always be directed towards this. To examine how clinical trials might fit into the scheme of epidemiological research consider the question - ‘do fish oils protect against prostate cancer?’ The series of studies might be:

1. An ecological study to determine whether it is a reasonable idea. Is the incidence of prostate cancer lower in countries that eat more fish?
2. Do a cross-sectional survey. Survey a community for diet and symptoms of prostate cancer.
3. Do a case-control study. Cases of prostate cancer and controls and assess diet (fish intake) or measure red cell membrane fatty acids.
4. Identify a cohort of males with high and low fish oil intake and follow them to estimate the rate ratio for prostate cancer.
5. Do a randomised control trial to assess the impact of dietary supplementation with fish oils versus placebo on the rate of prostate cancer.
6. Implement policy.

The best observational studies will be designed to minimise bias, confounding and chance, but even with well designed studies we can never be absolutely certain that we have excluded bias and confounding. It is this drawback that limits the amount of weight we can give to observational data in terms of changing policy. The role of a good quality randomised control trial is to exclude altogether the effects of bias, confounding and chance and this is why they are he,d in such high regard and considered to be the ‘gold standard’ design. Since randomised controlled trials are experimental it is the researcher who determines who takes part and how the outcome is measured. Because of the control the researcher has with the clinical trial design he/she is able to use powerful means of eliminating confounding, bias and chance.

Question 37

In randomised controlled trials because of the control the researcher has with the clinical trial design he/she is able to use powerful means of eliminating confounding, bias and chance. What are these means?

Answer 37

• chance is minimised by estimating study power, ie studying enough people
• confounding is removed by randomisation
• bias is removed by blinding

Question 38

If clinical trials are so good why do we use other designs?

Answer 38

The reason is that with clinical trials we can only test beneficial interventions - we can do a trial of whether fish oil supplementation reduces incidence of prostate cancer but not whether smoking increases the incidence of prostate cancer. To assess the adverse effects of exposure we have to rely on observational data. Remember also that clinical trials are expensive, take a long time and since the sample of people in the trial might be restricted to certain individuals (eg high risk groups) their results may not be generalisable to other populations.

Question 39

What are the objectives of random controlled trials?

Answer 39

As in all epidemiological studies, we must be clear of the objectives in the initial study protocol. Randomised controlled trials are conducted for two main reasons:

1. To test a specific casual hypothesis regarding a disease (eg to investigate if increased exercise is associated with a reduction in breast cancer, we could randomise groups to receive or not receive regular exercise programmes and investigate their rate of breast cancer).
2. To measure the effect of a particular intervention (eg drug or treatment, or a preventative intervention such as a vaccine or health promotion campaign). These studies can be classified as:

a. Explanatory trials, where the objective is to measure the efficacy of an intervention under optimal conditions (eg initial testing of a new drug before it is licensed).
b. Pragmatic trials, where the objective is to measure the effectiveness of an intervention when implemented in practice (eg evaluation of a health promotion programme or a screening programme, or a field trial of a vaccine or a drug when used in regular medical practice).

Question 40

What do we need to consider with regards to study population and sample size in randomised controlled trials?

Answer 40

As in all epidemiological studies, generalisability is important so we must choose a study population (population in which the study is conducted) that is adequately representative of our target population (population in which the results of the study are intended to apply). Exclusion criteria are often more strict when choosing a study population for testing the efficacy of a new drug (eg exclusion of pregnant women or people with other illnesses) and may therefore not be completely representative of the reference population in which the drug will be marketed.

When designing an RTC we must estimate the study power. We need a sufficiently large study population in order to detect an effect of the intervention while excluding the possibility of chance findings, so we must always calculate a sample size prior to conducting our study.

We have seen previously that when we do analyses what we actually do is set up a null hypothesis and look for evidence to reject it. For example if you want to determine whether a new drug reduces the chance of getting influenza then the null hypothesis would be that the drug has no impact on the chance of getting influenza. We then do our trial comparing active treatment with control (placebo) and examine our results. We test our null hypothesis by asking, what is the probability of getting these results if the null hypothesis is true. If the probability is low, by convention less than 5%, we reject the null hypothesis as being unlikely and conclude the alternative explanation that the drug is having an effect. This is a useful approach but there are two potential errors. Firstly we could reject the null hypothesis when it is true and secondly we could not reject the null hypothesis when it is false. As mentioned above we use our p value from our statistical test to guard against rejecting the null hypothesis when it is true (called a type I error). We only reject the null hypothesis when the probability of our result occurring if the null hypothesis is true is less than 5%. We can make our probability value smaller to make sure we have a smaller chance of having a type I error (eg use a p value of less than 1%). Note: just by chance we will reject some null hypotheses that are true (type I error).

How do we guard against not rejecting the null hypothesis when it is false? The reason why this error occurs is because we do not have enough evidence - usually because our study is too small in sample size.

Question 41

With regards to randomised controlled trials what is randomisation and why does it work?

Answer 41

In the clinical trial setting randomisation is the process by which the people we want to study are allocated the intervention/treatment, for example how we decide who gets the new drug and who gets the control (placebo). How does randomisation break confounding? In observational studies confounding exists because the exposure of interest may also be associated with other (confounding) exposures that are also risk factors for the disease. For example a diet rich in fruit may appear to protect against lung cancer in an observational study, but this may well be because non smokers eat more fruit than smokers - here smoking is the confounder. If we allocate a dietary supplement to people randomly then smokers and non smokers would be equally likely to get the supplement. There is then no association between smoking and the supplement and so there can be no confounding by smoking. Therefore we can assess the supplement and not worry about confounding due to smoking. Randomisation is an extremely powerful way to break confounding. Importantly randomisation will break confounding with exposures we know about and also with those we don’t. It is important to remember that it works because subjects are allocated to their treatment group by chance, so it breaks the association between the confounder and the exposure.

Question 42

Describe some methods of generating randomisation sequences.

Answer 42

Central to randomisation is the concept that people are allocated to the treatment group by chance. This can be achieved in a number of ways which are listed below:

• tossing a coin - reasonable but slow. Could involve bias. There is a computer program to do this which will adjust the probability of the next patient being allocated to a group if the numbers are getting unbalanced - sometimes called the bias coined method.
• random numbers - a common method. Use random number tables, start at random point, all odd numbers could get active drugs, evens could get control (placebo).
• permuted blocks - this is common for multi-centre studies as it ensures that at any point in time there are roughly equal numbers in each group. Instead of randomising patients one at a time we randomise patients in groups/blocks, within which there are equal numbers of control (placebo) and active drug allocations. For example if there are two treatments (A=active drug and B=control) we can set up groups of letters or blocks as follows: AABB ABAB BBAA BABA BAAB. We can then select these using random numbers to draw up a randomisation list for each centre. For example AABB chosen if random number ends with 1, ABAB if ends with 2, BBAA if ends with 3 and so on until BAAB if ends in 6 - ignore random numbers ending in 7, 8 and 9. In some studies the block length will vary, sometimes randomly.
• minimisation - this is a more complex procedure that uses computer programmes to allocate patients to each group and aims to ensure many characteristics of the patients in each group are similar. Minimisation is a specialised technique that is useful for small trials. But redundant for large trials.

Differences between the two groups, however, may still occur by chance, and we can use statistical methods to assess whether these observed differences can be observed by chance. This means that, even though randomisation can ensure comparability of the two groups, baseline data should be collected on all study subjects. This generally includes demographic variables such as age, sex, occupation and education, as well as variables thought to be associated with the intervention and the outcome of interest. If we want to measure changes in the outcome over time (eg change in weight or blood pressure), we also need to make initial measurements of the outcome of interest.

Question 43

Describe allocation concealment for clinical trials.

Answer 43

It is very important that the treatment is allocated after the subject has been recruited to the study and after baseline measurements have been taken. If subjects are allocated to the treatment groups in advance and these are known by the subject or the investigator, or both, this may affect whether the individual is recruited to the study, leading to considerable bias. Similarly, knowledge of treatment group may influence measurement of the baseline data in a biased manner. Most commonly, sequentially numbered opaque sealed envelopes contain the treatment allocation, or a Clinical Trials Unit are used to keep the sequence of randomisation list secret from the investigators and the subjects until after the subject has agreed to participate in the study.

Systematic randomisation (based on odd/even hospital numbers, day of the week, or time at entry or hospital admission) is unsatisfactory, because allocation concealment is difficult to ensure and bias may be introduced when using time of day or weekday of hospital admission. The system of randomisation may become known to the investigators which may introduce bias in their subject recruitment, even if unintentional.

Question 44

Give a brief summary of randomisation.

Answer 44

• eliminates the role of confounding, both known and unknown
• many different methods can be used to generate randomisation sequences
• make sure the method used to generate the randomisation sequence is truly random and won’t introduce bias into the study
• make sure that the investigators and future subjects in the study are not aware of the next allocated treatment from the randomisation sequence

Question 45

Describe the measurement of the outcome and blinding for clinical trials.

Answer 45

The outcome(s) may be measured continuously as a single follow-up after a fixed time interval (eg admission to a hospital at 9 months following treatment), or as a series of repeated measurements (eg weekly measurements, or six monthly clinical visits). It is therefore important to minimise the number of subjects lost to follow up, which can be difficult if the follow up period is long or of there are invasive or time consuming procedures in the collection of outcome measurements (eg blood sampling, complete health check visits). In RCTs loss to follow up is an important source of bias, since those lost may have important differences which are related to the outcome. Procedures to reduce loss to follow up should be considered from the initial planning of the study. These may include re-visits when subjects are not available or reminders by post or phone.

Validity of comparison of the outcome between the two groups must be ensured by measuring the outcome in the same way for the treatment and the control groups. If the subject or the investigator is aware of the treatment group, reporting or measuring the outcome may be influenced by this knowledge, which may lead to ascertainment bias. An important too, in RCTs is the use of blinding to reduce this bias. Blinding ensures that a subject’s treatment group allocation remains unknown.

Four main players in a study may be blinded:

1. The study subject - knowledge of their treatment allocation may affect their behaviour (eg they may be less careful of taking other preventative measures if they are in the intervention group)
2. Those recruiting subjects and delivering the intervention (e.g. clinician) - knowledge of the treatment group may influence their behaviour in recruitment or delivery of the intervention (e.g. clinician may be more likely to provide an alternative treatment to an individual in the control group).

If they are also assessing the outcome measurements, blinding will avoid biased measurements (eg clinician may be mor vigilant in measuring outcomes precisely for those in the intervention group).

3. Those conducting the evaluation (eg interviewer or laboratory personnel) - this avoids bias in outcome measurement.
4. Those doing the analyses (primary investigator or statistician) - this avoids bias in the analyses and how it is investigated. It is not always possible to blind all four players and studies are often single-blinded or double-blinded. However, it is difficult to establish sometimes which of the four players are blinded. Double blinding does not always mean the subject and clinician.

It is usually easiest to blind those conducting the evaluation (e.g. It is almost always possible to blind laboratory personnel and interviewers). The first two players can be blinded if the control group uses a placebo that is indistinguishable from the intervention, which is possible when delivering certain drugs or vaccines. Quite often the main analyses are conducted before the code is broken to ensure a more objective approach. Do not always assume a trial is double blind just because people say they are - be sure you believe the blinding. Two capsules could look identical one with fish oil and one placebo, the fish oil could potentially cause a person to burp up fish oils (and taste differently) whereas the placebo would not have that sort of side effect.

Question 46

Discuss ethics and consent with regards to clinical trials.

Answer 46

Ethical considerations that apply to any research, also apply to trials. All trials should be reviewed and approved by an ethics committee. The following are some issues that are specific to trials:

• informed consent (or assent for very ill subjects) is especially important in trials because participants (or their relatives) must understand that they may receive an inactive product in a placebo-controlled trial. Subjects murs also understand that the extent of protection is unknown in preventive trials so participants should continue to tame other preventive measures.
• the intervention given to the control group, if any, should be considered carefully; control subjects should not be given deprived or effective interventions that they would have received if they had not participated in the trial. Therefore, if there are existing interventions with proven effectiveness that are available, it is not considered appropriate to use placebo controls.
• another general principle is that random allocation is only justified when there is genuine uncertainty as to the relative benefits of the two treatment conditions (the principle of equipoise). If data show clearly that one of the intervention treatments is superior early on in the study, it is possible to incorporate interim analysis into the design so that the study can be terminated early, however special statistical techniques are needed for this.

Question 47

What are the advantages of randomised controlled trials?

Answer 47

Advantages of RCT:

• rigorous evaluation of a single exposure or treatment
• prospective design (can assess causation)
• potentially eradicates bias and confounding

Question 48

What are the disadvantages of random controlled trials?

Answer 48

Disadvantages of RCT:

• expensive and time consuming hence:
  - most trials never done, or completed trials are too small or have follow up periods which are too short
  - most trials are funded by large research bodies or drug companies (who may dictate research agenda)
• risk of ‘hidden bias’ - did the blinding actually work?
• limited generalisability - often exclude high risk groups such as the elderly, children or pregnant women

Question 49

Give examples of situations where a randomised control trial might be impractical.

Answer 49

• where it would be unethical to randomise subjects
• where the number of patients needed to demonstrate a clinically significant difference between groups is prohibitively high
• where the length of the study would be too long eg breast feeding and adult onset diabetes