L23-L24 Data Acquisition Approaches

Question 1

Compare & contrast primary versus secondary data.

Answer 1

Primary Data | Secondary Data:

1) Collection purpose: Specifically for research qn at hand | For other purposes
2) Data variables: Exact data variables needed to answer research qn collected | Limited to existing data variables collected
3) Missing data: Can be minimised as data collection process is controlled by researcher | Often a problem as actual data collection is NOT w/in researcher control
4) Accessible population: May be less representative of target population | More representative of target population
5) Collection process: Involved | Rapid and easy
6) Collection cost: Costly | Less costly
7) Collection time: Time-consuming | Less time-consuming

Question 2

List the three main primary data collection

methods.

Answer 2

1) Experimental Method:
- Laboratory-based experiment
- Controlled trial

2) Observation Method (NOT to be confused with observational study design!)

3) Survey Method
- In-depth interview
- Focus group discussion
- Questionnaire

Question 3

Describe how data is collected via the experimental method.

Answer 3

Research method used to test a hypothesis by seeking to establish a causal relationship between dependent & independent variables

• Involves manipulation of independent variable (e.g. Tx) while trying to keep all other variables constant, so as to measure the effect of independent variable on one or more dependent variables (e.g. outcome)
• Data collection via case report form (CRF)

Question 4

Describe how data is collected via the observation method.

Answer 4

Research method enabling researchers to systematically observed & record people’s behaviour, actions and interactions.

Either via non-participation observation or participant observation:

1) Non-participant: Researcher conducts observation w/o participating in activities that he/she is observing (from outsider’s perspective)
2) Participant: Researcher observes through exposure to or involvement in the day-to-day or routine activities of study subjects (from insider’s perspective)

Question 5

Discuss the strengths & limitations of using the observation method to collect primary data.

Answer 5

(+) Independent of subjects’ willingness to respond
- Relatively less demanding of subjects’ active cooperation as compared w/ survey method
(+) Useful for studies that involve subjects who are NOT capable of giving verbal report of their feelings
(+) Allows researcher to directly see what people do rather than relying on what people say they did (i.e. prevent reporting bias)

(-) Time-consuming
(-) Does not increase your understanding of why people behave as they do
(-) Required skilled observers
(-) Researchers need to refrain from interpretation
(-) Susceptible to Hawthorne effect (or observer effect)
- Hawthrone effect: a phenomenon where study subjects change their behaviour due to awareness of being observed

Question 6

Discuss what considerations you have to take when using the observation method to collect primary data.

Answer 6

1) What should be observed?
- Type of note-taking: Recording sheet & checklist, observation guide or field notes

2) How the observations should be recorded?
- Pen-and-paper, audio or video recording

3) How the accuracy of observation can be ensured?

Question 7

Describe how data is collected via in-depth interviews under the survey method.

Answer 7

One-to-one method of data collection that involves an interviewer & an interviewee discussing specific topics in depth

• A conversation with a purpose to gain insight into certain issues
• Less of a discussion, BUT interviewer needs to remain neutral
• Attempt to guide interviewee to answer & NOT judge

Question 8

Discuss the strengths & limitations of using in-depth interviews under the survey method to collect primary data.

Answer 8

(+) Gains information on people’s personal experiences, stories, feelings etc.
(+) Useful for sensitive topics
(+) Gains in-depth information

(-) Time-consuming
(-) Susceptible to interviewer bias
(-) Needs skills to establish rapport, use motivational probes, listen & react to interviewees
(-) Flexibility needed to change topic order in interview guide following interviewee’s story
(-) Lots of transcription required

Question 9

Discuss what considerations you have to take when using in-depth interviews under the survey method to collect primary data.

Answer 9

1) Using a semi-structured interview guide to prompt data collection
- Establishing rapport between interview & interviewee first with general, simpler questions
- Asking questions in an open, empathetic way
- Motivating interviewee by probing
- Close w/ open, general questions to “hop” out of intrusive approach

2) Research instrument: Are the interview questions posed in a language & using words understandable to interviewee?
- Consider translating before interview & conduct back-translation to verify accuracy; NOT on-the-go translation!
- Pilot-testing for newly crafted research instrument / interview guide

3) Pilot-testing:
- Are all questions understood as intended?
- Do any questions need to be reworded to improve clarity?
- Is the order of questions logical?
- Will the information gained help to answer the research question?
- Is the research instrument too long / too short?

4) Data collection:
- Manual note-taking
- Audio recording
- Video recording
- Combination of the above

Question 10

Describe how data is collected via focus group discussion under the survey method.

Answer 10

Involves a focus on specific issues, with a predetermined group of people, conducting an interactive discussion

• Aimed to gain a broad range of views on the research topic
• 6 to 8 pre-selected participants comprising of a moderator & a note-taker
• As long as one participant does NOT consent to recordings, NEED to use manual note-taking of opinions & audio/visual cues!

Question 11

Discuss the strengths & limitations of using focus group discussions under the survey method to collect primary data.

Answer 11

(+) Large volume of information
(+) Identify a wide range of views
(+) Identify new & unanticipated views
(+) Useful for exploratory, explantory & evaluative research

(-) Susceptible to interviewer bias, based on interviewer’s interpretation / guidance to certain answers
(-) Discussion can be dominated or sidetracked by a few individuals
(-) Required skilledmoderator to conduct the group discussion & manage group dynamics
(-) Less confidential
(-) Influence of social pressure
(-) Responses are not independent; thus does NOT provide valid information at the individual level (i.e. information is specific to group, rather than individuals w/in focus group, since others may introduce bias in their opinions)
(-) Information is NOT representative of other groups w/in study

Question 12

Discuss what considerations you have to take when using focus group discussion under the survey method to collect primary data.

Answer 12

1) Composition of focus groups:
- Homogeneity among participants: more likely to share their views & experiences with others who are similar to themselves
- Level of familiarity among participants: Recruiting a group of strangers is often preferred as greater anonymity may allow participants to contribute more freely to discussion

2) Discussion guide:
- Funnel design: Intro (cognition) -> broad opening qn (rapport) -> specific qn (data collection) -> closing qn (closure) -> post-discussion qn (addn info.)
- Type of discussion: Moderated-dominated discussion or interactive group discussion

3) Research instrument: Are the interview questions posed in a language & using words understandable to interviewee?
- Consider translating before interview & conduct back-translation to verify accuracy; NOT on-the-go translation!
- Pilot-testing for newly crafted research instrument

4) Pilot-testing:
- Are all questions understood as intended?
- Do any questions need to be reworded to improve clarity?
- Is the order of questions logical?
- Will the information gained help to answer the research question?
- Is the research instrument too long / too short?

5) Data collection:
- Manual note-taking
- Audio recording
- Video recording
- Combination of the above

Question 13

Describe how data is collected via questionnaires under the survey method.

Answer 13

Consists of a number of questions printed or typed in a definite order on a form or set of forms.

Designed with the purpose of seeking specific information from respondents.

Question 14

Under what situations are questionnaires useful for primary data collection?

Answer 14

1) Limited resources & require data from many people
2) Gather data about knowledge, beliefs and attitudes
3) Important to protect the privacy of participants

Question 15

Discuss what considerations you have to take when using questionnaires under the survey method to collect primary data.

Answer 15

1) Requires good question design principles
- Answers to questions become data for analysis, thus it is crucial that good question design principles are utilised in questionnaire construction or selection
- Be concise & clear; AVOID big words, technical jargons, double negatives; include time frame at beginning of question
- Be complete in providing options in closed-ended qn (e.g. do not have gaps, do not overlap categories)
- AVOID double-barreled questions (e.g. Do you like A and/or B?
- AVOID bias and leading questions

2) Use previously well-tested instruments, if available
- Save development time of instrument or adopt parts of it before developing additional qn

3) Pilot-test / Pretest questions well before adopting in study
(a) Are the interview questions posed in a language & using words understandable to interviewee?
- Consider translating before interview & conduct back-translation to verify accuracy; NOT on-the-go translation!
- Cross-cultural adaptation will require pre-testing!
(b) Pilot-testing for newly crafted research instrument:
- Are all questions understood as intended?
- Do any questions need to be reworded to improve clarity?
- Is the order of questions logical?
- Will the information gained help to answer the research question?
- Is the research instrument too long / too short?

4) Administration of questionnaire: Self-administered or interviewer-administered?
5) Delivery of questionnaire: Mail/e-mail, telephone or in person?

Question 16

Outline the general format of a questionnaire.

Answer 16

1) Introductory statement/paragraph
2) Demographic question
- Emotionally neutral qn for easier response / building rapport
3) Factual question
4) Opinion questions
- Emotionally charged qn
5) Closing statements & return instructions

Question 17

Discuss the strengths & limitations of using open-ended response questions in questionnaires under the survey method to collect primary data.

Answer 17

(+) More detailed answers elicited
(+) Opportunity to study respondents’ interpretations expressed in their own words (for qualitative research)
(+) To minimise successful guessing when asking knowledge-based questions
(+) Useful as a wrap-up question at the end

(-) Less structured
(-) Difficult to ensure systematic recording of responses (if interviewer-administered)
(-) Difficult to adequately probe for more complete / understandable answer w/o being biased (if interviewer-administered)
(-) May be more difficult for respondent than selecting an option in closed-ended response
(-) More time taken by respondent to answer
(-) ~75% of respondent will leave such question blank (if self-administered)
(-) Difficult to code information for data analysis

Question 18

Discuss the strengths & limitations of using closed-ended response questions in questionnaires under the survey method to collect primary data.

Answer 18

(+) Tightly structured
(+) Ensures standardisation of responses
(+) Responses easily encoded & analysed
(+) Less time taken to collect responses

(-) Less ‘depth’ in answers
(-) Imposes researchers priorities on respondent
(-) May bias responses if range of options are not exhaustive
(-) Presentation formats may affect responses (e.g. top-down listing has inherent ranking vs left-right listing)

Question 19

For ranking type of closed-response questions, what is the maximum options you should provide to minimise the difficulty for respondent to choose from?

Answer 19

Do NOT exceed more than 5 options!

Question 20

To avoid neutrality when using Likert scales as part of closed-response questions in a questionnaire, what type of scale should be used?

Answer 20

Even-numbered options on a scale to determine participants’ tendency

Question 21

Discuss the strengths & limitations of using self-administered questionnaires under the survey method to collect primary data.

Answer 21

(+) Cheap to administer
(+) Less susceptible to interviewer bias
(+) Can be administered by mail or e-mail

(-) Lower response rate
(-) Difficult to elicit detailed responses
(-) Less control over how questionnaire is filled out

Question 22

Discuss the strengths & limitations of using interviewer-administered questionnaires under the survey method to collect primary data.

Answer 22

(+) Higher response rate
(+) More detailed responses can be elicited
(+) Greater control over filling out of questionnaire

(-) Expensive to administer
(-) More susceptible to interviewer bias
(-) Time-consuming

Question 23

How do you calculate response rate?

Discuss the concerns of a low response rate.

Answer 23

Response rate = no. of participants who complete questionnaire / total no. of eligible persons who were asked to participate x 100%

Non-response bias (i.e. type of selection bias)

• Respondents may differ from non-respondents in their baseline characteristics
• Answers of respondents may differ from potential answers of those who did not answer; thus compromising on internal validity & weakens external validity of survey results

Minimum of 70% response rate will be optimal to minimise non-response bias.

Question 24

Discuss the uses of different modes of delivery of questionnaire (mail or e-mail, telephone or in-person).

Answer 24

Mail / Email useful when:

• Sample is geographically dispersed
• Directed to specific groups
• Research budget & manpower is limited
• Respondents need time to think about answer
• Questions are closed-ended, simple & clear
• Respondents need to be given privacy when answering

Telephone useful for:
- Open-ended questions
- Complex interviews
(+) Quality & speed when using computer-assisted telephone interviews

In-person useful for:
- Open-ended questions
- Complex interviews
- Using visual aids
(+) Quality & speed when using computer-assisted personal interviews

Question 25

Discuss the practical considerations of different modes of delivery of questionnaire (mail or e-mail, telephone or in-person).

Answer 25

Mail / Email:

• Accessibility of sample
• Response rate usually low = 5 - 30%
• Requires reminders, incentives, deadlines & return envelop with postage (for mail) to potentially improve response rate & turnaround time for data collection
• Should be avoided when asking about knowledge (can ask Google)

Telephone:

• Accessibility of sample
• Response rate generally good = 65 - 75%
• Turnaround time for data collection = (no. of surveys to be collected x duration for each interview) / (no. of interviewers x no. of hours each interviewer works per day)

In-person:

• Accessibility of sample
• Response rate generally good = 70 - 80%
• Turnaround time for data collection = (no. of surveys to be collected x duration for each interview) / (no. of interviewers x no. of hours each interviewer works per day)

Question 26

What formula is used to calculate the minimum sample size for sufficient statistical power, when using questionnaires as primary data collection?

Answer 26

Cochran’s formula:
For large populations, n0 = (Z^2)(pq) / (e^2), where,
n0 = sample size
Z = z-score of 1.96 at 95% CI
p = estimated proportion of an attribute present in population (assumed 0.5 for maximum variability)
q = 1 - p
e = desired level of precision

Adjustment for small populations, 
n = n0 / [1 + {(n0 - 1) / N}] when n0 => 0.05N, where, 
n = sample size
n0 = sample size for large populations
N = population size

Question 27

Under what condition will there be a need to adjust the use of Cochran’s formula in calculating sample size for questionnaire data collection?

Answer 27

Small population of n0 is at least 0.05N, where:
n0 = sample size for large populations
N = population size

Question 28

What formula is used to calculate the minimum sample size for sufficient statistical power, when using questionnaires as primary data collection, assuming 95% CI and p = 0.5?

Answer 28

Yamane’s formula (simplified formula of Cochrane’s formula):
n = N / [1 + N(e^2)] where,
p = estimated proportion of an attribute present in population (assumed 0.5 for maximum variability)
n = sample size
N = population size
e = desired level of precision

Question 29

List all random and non-random sampling methods used for questionnaire administration?

Answer 29

1) Random / Probability sampling methods:
- Simple random sampling
- Systematic random sampling (i.e. every kth item selected from every N/n group size)
- Stratified random sampling (i.e. divided into relevant strata before random sampling)
- Cluster sampling (i.e. population divided into clusters & a subset of clusters is randomly selected, where all units within clusters are surveyed | e.g. SG school children -> cluster by school -> randomly select classes within selected schools and survey all children within selected classes)
* Requires FULL list of population to use these methods, though more representative.

2) Non-random / Non-probability sampling methods:
- Convenience sampling -> selection bias introduced
- Quota sampling (i.e. reserve a certain proportion of participants to particular types of ppl)
- Snow-ball sampling (i.e. use some selected participants to nominate others whom they know could be interviewed | e.g. drug addicts) -> lead to not-so-diverse sampling due to similar characteristics

Question 30

What are some examples of secondary data sources?

Answer 30

Impt:

1) Medical / prescription / dispensing records
2) Registry data

3) Spontaneous reporting surveillance data
4) Claims database
5) Cross-sectional survey data
6) Large prospective cohort data

Question 31

Discuss the strengths & limitations of using medical / prescription / dispensing records to collect secondary data.

Answer 31

(+) Clinical data (including lab & image data) are often considered as the best source for disease outcomes
(+) Efficient: data already collected, large sample size & long follow-up
(+) Prospectively collected data
(+) Detailed information on prescriptions, including dose, duration & indication from diagnoses
(+) Can study many drugs in relation to many outcomes
(+) Can conduct nested case-control studies

(-) Drug use is based on prescription and/or dispensing information: not on actual drug use, compliance / adherence unknown
(-) Non-prescription drugs missed (e.g. OTC, CAM, herbal medications, vitamins & supplements)
(-) Diagnosis based on codes and/or text (i.e. international classification of diseases by WHO)
(-) Incomplete information on habits, past history & other potential confounders
(-) Uncertain completeness of data from other physicians & sites of care

Question 32

List the types of registry data we have in Singapore.

Answer 32

1) Disease registry or clinical registry in SG:
- Cancer
- Acute MI
- Renal
- Donor care (kidney & liver)
- Stroke
* all part of National Registry of Diseases Office (NDRO) of Singapore

2) Drug use registry
- e.g. pregnancy registries for antiseizure medicine uses

Question 33

Discuss the strengths & limitations of using registry data to collect secondary data.

Answer 33

(+) Detailed & systematically collected clinical data
(+) Usually representative of patients or drug users
(+) Inexpensive if using data already created for other purposes

(-) Minimal information on drug use (in clinical registries)
(-) No appropriate control or comparison groups
- Could use other drug users as a comparison or other disease as a control
(-) Expensive if you have to create a registry
(-) Incomplete information on habits, past history & other potential confounders

Question 34

What are some considerations in the collection or use of secondary data for research purposes?

Answer 34

1) Reliability
- Who collected the data?
- What were the sources of data?
- How were the data collected?
- When were the data collected? And for how long?

2) Suitability
- What were the objective, scope & nature of the data
source?
- Is the data suitable to scientifically answer the research question at hand?
- Is the information exactly what is needed for the
research question? Suitable as proxy?

3) Adequacy
- Is the level of accuracy of the data adequate for the
research question at hand?
- Is the data set complete?
- Are there confounding issues?

4) Ethical considerations
- May need special approval (sign user agreement) to
obtain data variables for research
- Compliance with requirements of the Singapore’s
Personal Data Protection Act 2012
- De-identified data