9.0 Quantitative - data collection and analysis

Question 1

Types of data collection in quantitative research

Answer 1

Physiological measurement
Observation
Interviews
Questionnaires
Records or other documents

Question 2

Interview schedule (structured) and questionnaire differ only in that

Answer 2

the interview involves interviewer asking the questions, and with the questionnaire the respondent reads themselves.

Question 3

Three ways to administer a questionnaire

Answer 3

Mail/Web
Collective/captive audience
In public place

Question 3

Considerations in choosing between an interview or questionnaire

Answer 3

Topic (e.g. a sensitive issue)
Where the respondents live
Who the study population is

Question 4

Advantages of using a questionnaire for data collection

Answer 4

Less expensive
Anonymity

Question 5

Disadvantages of using a questionnaire for data collection

Answer 5

Application is limited
Literacy respondents
Low response rate
Self-selecting bias
Opportunity to clarify issues not there
Does not allow for spontaneous response
Response to a question may be influenced by other responses.
Opportunity to consult with others.
Cannot be supplemented with other information (i.e. + observation)

Question 6

What is secondary data in research?

Answer 6

Existing data/Second hand information.
Examples:
Earlier research, census data, personal records, government publications
Government registries including cancer registries, hospital morbidity data, the mental health register etc.
Clinical records

Question 7

Surveys and questionnaires - purpose:

Answer 7

A structured way to collect information.
Surveys are typically conducted through the mail (electronic or surface), phone, or internet.

Purpose:
To collect standardized information from large numbers of individuals.
When face to face meetings are inadvisable.
When privacy is important or independent opinions and responses are needed

Question 8

What’s the difference between a survey and questionnaire?

Answer 8

Survey – method - “a descriptive research method where respondents are asked a series of questions in a standard manner so that responses can be easily quantified and analysed statistically.”

Questionnaire - tool - a specific type of written survey made up of a structured series of questions. Questionnaires usually have highly standardised response options so that data can be easily analysed and compared.”

Question 9

Research design steps

Answer 9

Decide who should be involved in the process.
Define content.
Identify your respondents.
Decide on the survey method.
Develop the questionnaire.
Pilot test the questionnaire and other materials.
Think about analysis.
Communicate about your survey and its results.
Develop a budget, timeline, and management process

Question 10

Representativeness

Answer 10

• refers to how well the sample drawn for the questionnaire research compares with (eg, is representative of) the population of interest.

Question 11

Questionnaire structure

Answer 11

Title

Introductory paragraph (ethics, confidentiality)

Content questions:
Section
Filter questions (Go to question…)
Place easiest questions upfront
Avoid providing answers to later questions

Finish:
How to return questionnaire
Link to press

Acknowledgement

Question 12

Response rate

Answer 12

The proportion of people who respond: divide the number of returned surveys by the total number of surveys distributed.

If 50 questionnaires are distributed and 25 questionnaires returned the response rate is 50%.

High response rate - promotes confidence in results
Lower response rate - increases the likelihood of biased results.

Question 13

How to increase response rate

Answer 13

Generate positive publicity for your survey.
Over sample.
Ensure respondents see the value of participating.
Make (multiple) follow-up contacts.
Provide incentives.
Provide 1st class postage/return postage.
Set return deadlines.
Make the survey easy to complete.

Question 14

Good questionnaires are NOT EASY

Answer 14

Developing a good questionnaire takes time, time and more time.
Multiple drafts may be involved before the questionnaire is ready.
It’s important to involve others in writing the questionnaire.

Question 15

Questionnaire design − Considerations

Answer 15

Kind of information: What do you want to know?
Is the information already available?
Wording of questions and responses
Formatting the questionnaire
Pre-testing
Cover letters and introductions
When/where will the questionnaire be distributed? How will returns be managed? How will the data be analyzed?
Who is responsible for each task?

Question 16

Steps to take if using questionnaires in research

Answer 16

Step 1: What information is needed?
Step 2: Sample
Step 3: Develop questionnaire
Step 4: Plan distribution, return, follow-up
Step 5: Pilot test
Step 6: Revise and revise

Question 16

Open-ended questions

Answer 16

− allow respondents to provide their own answers
N
umeric open end (e.g. please state AUD$_______ )
Text open end (sometimes called “verbatims”). How could you increase the proportion of household income used on healthcare?

Question 17

Closed-ended questions

Answer 17

− list answers and respondents select either one or multiple responses
Questions: How much do you spend on health care per year?
multiple choice (<100, <300, <500, <700,+1000)

Question 18

Question design

Answer 18

Avoid vague questions and answers.
Avoid ambiguous words or phrases.
Avoid questions that may be too specific.
Avoid making assumptions.
Avoid leading questions.
Biased questions
- Influence people to respond in a certain way – “loaded questions”
- Make assumptions about the respondent – “leading questions”
- Use language that has strong positive or negative appeal

Write questions through your respondent’s eyes
Will the question be seen as reasonable?
Will it infringe on the respondent’s privacy?
Will the respondent be able and willing to answer the question?
Be selective and realistic when writing questions.
Order and wording of questions can affect responses and bias the results (Bowling, 2002).

Question 19

Types of questions

Answer 19

Behaviour questions - ask about what people do – do you currently smoke cigarettes?

Belief questions – ask about whether people belief something to be true or false - Do you think peer pressure or parental example is more influential in determining smoking uptake among adolescents?

Attitude questions – seek to establish what respondents think is desirable- Do you agree with the statement ‘Airports should provide a smoking room for travellers.’

Knowledge questions - seek to determine what people know about particular topics. – what do you know about the effects of smoking on women?

Attribute questions –information about more objective characteristics of respondents - age, gender, place of residence etc.

Question 20

Reliability is …

Answer 20

the extent to which a measurement instrument is dependable, stable, and consistent when repeated under identical conditions.

Question 21

Validity is …

Answer 21

the extent to which the scale measures what it is supposed to measure (content and construct validity).

Question 22

Likert scale

Answer 22

Commonly 5 or 7 point scales.
Most commonly used scaling method in health research.
Has typically high reliability.

Question 23

Demographic data collection questions

Answer 23

AgeGender
Ethnicity
Marital status
Family size
Occupation
Education
Employment status
Residence
Previous contact with organisation
Prior knowledge of topic
First-time participant vs. repeats
How you learned about the program

Question 24

Pilot testing a questionnaire

Answer 24

Always!
With people as similar to respondents as possible:
Do they understand the questions? The instructions?
Do questions mean same thing to all?
Do questions elicit the information you want?
How long does it take?
Revise as necessary

Question 25

Data Processing involves:

Answer 25

1. Editing data - includes procedures for detecting and correcting errors in the raw data.
2. Coding data
3. Analysis of data - computers play a major role here for analysing quantitative data.
   Quantitative data analysis software packages: SPSS, SAS, Stata, R, JMP, MATLAB

Question 26

There are four levels of measurement:

Answer 26

nominal, ordinal, interval and ratio

Question 27

Nominal (a level of measurement)

Answer 27

Variables are divided into two or more categories and assigned arbitrary numbers. These numbers have no rank order.

Gender: male 1, female 2; Marital status: single 1, married 2, separated 3, divorced 4, widowed 5

Question 28

Ordinal (a level of measurement)

Answer 28

Reflects a rank order among the categories, but we do not know how much greater than or less than.

Pain: ranked on a scale of 1 to 10 with 10 being the worst pain and 1 being no pain.
Anxiety: ranked on a 5-point scale from low to high.

Question 29

Interval (a level of measurement)

Answer 29

Are made up of ‘real’ numbers that allow us to order the numbers and to know the distance between those numbers. The intervals between the categories are equal.

Time: hours, minutes and seconds are precise intervals)
Temperature: interval between the numerical values of 76 and 77 degrees is the same as the interval between the values of 44 and 45 degrees.

Question 30

Ratios (a level of measurement)

Answer 30

Ratios have the same properties as interval data, except that the measurement scale for the variable possesses a meaningful zero. This means that when zero is reached on the scale, the variable is absent.

Weight, height: zero means no weight or no height.

Question 31

Types of statistical analysis

Answer 31

1. Descriptive
2. Inferential

Question 32

Descriptive statistics

Answer 32

Describe or summarise information obtained from sample (e.g. frequency, percentage, mean, mode, median)

Question 33

Inferential statistics

Answer 33

use sample results to draw conclusions regarding the relevant population.

Statistics that allow a researcher to make inferences about whether relationships observed in a sample are likely to occur in the wider population from which that sample was drawn. Inferential statistics use logic and mathematical processes in order to test hypotheses relating to a specific population based on data gathered from a sample of the population of interest.

Allow you to test a hypothesis or assess whether data is generalisable to the broarder population.

Question 34

Difference between descriptive and inferential statistics

Answer 34

Descriptive statistics summarize the characteristics of a data set.

Inferential statistics allow you to test a hypothesis or assess whether your data is generalizable to the broader population.

Question 35

Mode

Answer 35

Most frequently occurring score in a frequency distribution. Only measure of central tendency where data are nominal, however it can be used with all levels of measurement.

Question 36

Median

Answer 36

Middle score or where 50% of scores are above it and 50% below it.

Question 37

Mean

Answer 37

Most widely used measure of central tendency.
Average of all scores. Used with interval and ration data.

Question 38

Measures of variability

Answer 38

Range
Variance
Standard Deviation
z scores
Quartiles and the interquartile range
Percentil

Question 39

Standard Deviation

Answer 39

Standard Deviation - meausre of average deviation or distance of each score from the group mean in a normal distribution. Must always be reported with the mean.
68% of the sample will fall within 1 SD from the mean.
Small DF = less variability within the sample and the more similar the scores to the mean and to each other.

Question 40

Variation

Answer 40

Variance = sum of squares / number of values
Small value for variable = values are very close ot the mean and therefore similar to each other.
Larger variance = values are very spread out around the mean and from each other.

Question 41

Standard Deviation

Answer 41

Standard Deviation - measure of average deviation or distance of each score from the group mean in a normal distribution. Must always be reported with the mean.
68% of the sample will fall within 1 SD from the mean.
Small DF = less variability within the sample and the more similar the scores to the mean and to each other.

Question 42

z scores

Answer 42

Used to compare measurements/values in standard units. Takes account of mean and SD of the distribution

Each score is converted to a z score and then the z scores are used to examine the relative distance of the scores from the mean - process called standardising the score.

z score = 1.5 = observation is +1.5 SD above the mean
z score = -2 = observation is -2 SD below the mean

Question 43

Quartiles

Answer 43

Cuts the observations into 4 equal amounts / sections. Q1 - 25th percentile , Q3 = 75 percentile.

Distance between Q1 and Q3 = interquartile range and indicates the range of the middle 50% of scores. More stable than range because it is less likely to be changed by a single extreme score.

Question 44

Percentile

Answer 44

Represents the percentage of cases a given score exceeds.
A score in 90th percentile is only exceeded by 10% of the scores.

Question 45

Skewness

Answer 45

Refers to asymmetry of a distribution of interval or ration scores.

Question 46

Kurtosis

Answer 46

Related to peakness or flatness of distibution

Question 47

Inferential statistics

Answer 47

Hypothesis testing
Probability and level of significance
95% CI
Odds ratio
Errors in statistical inference: type 1 and type 2 errors
Power anaylsis
Effect size
Tests of significance (parametric or non-parametric)

Question 48

The term statistically significant means

Answer 48

that the result is unlikely to have occurred due to chance fluctuations in sampling.

Levels of significance - alpha levels
a = 0.5 (researcher willing to accept a 5% risk that the results are in error) - minimal levels acceptable for all scientific disciplines.
a = 0.01 (1% risk of error)
a = 0.001

Question 49

Confidence interval

Answer 49

identifies a range of values that includes the true population value or a particular characteristic at a specified probability level.

If CI passes through 0 = no realtionship

e.g. 95% CI of a mean indicates if we took 100 similar samples and calculated their means, 95 of those samples would contain the ‘true’ population mean, and 5% wouldn’t.

Mean _+ SD
6.12 +- 2.54
95% probability that in the target population mean would be between 5.53 and 6.70.

Question 50

Odds ratio

Answer 50

Another way of presenting probabilities.

Summary statistic that estimates the odds of an event occuring in one group compared to another. Measure of strength of association.
OR of 1 = that either event is likely
OR > 1 = event is more likely to happen
OR <1 event is less likely to happen

Can be obtained in chi square analysis, logisitc regression where outcome is dichotomous, used in meta-analyses.

OR = 1.60 (can interpret at 60% increase in risk)

Statistic used to assess the risk of a particular outcome and is widely used in the healthcare literature—particularly epidemiology because it can be calculated for case-control studies, in studies using logistic regression, and as a way of presenting the results of a meta-analysis.

Question 51

Type 1 and Type 2 errors

Answer 51

Two types of errors in statistical inference.

Type I - false positive, e.g. stats say treatment is effective when it isn’t).

Type II - false negative.

Impossible to eliminate these errors however researchers try and minimise through conducting appropriate power analysis.

Question 52

Power analysis

Answer 52

quantitative method that allows us to determine the sample size required to detect an effect of a given size with a given degree of confidence.

Question 53

Effect size

Answer 53

equates to the magnitude of the effect of an intervention or treatment.

Question 54

Degrees of freedom

Answer 54

Represent the number of data points in any given set of data are are free to vary.

Question 55

Parametric tests

Answer 55

type of inferential statistic that involves the estimation of at least one parameter. Such tests require either interval or ratio data and involve a number of assumptions about the variables under investigation including the fact that the variable is normally distributed.