Statistical Reports

Question 1

What is sampling error?

Answer 1

Variation in survey percentages due to sampling

Question 2

What is the Margin of Error?

Answer 2

A number used to give an indication of the amount of uncertainty due to sampling error

Question 3

Why do we use the Margin of Error?

Answer 3

It’s a way for accounting for the variation in survey percentages due to sampling

Question 4

What is the “rule of thumb” method for calculating the Margin of Error?

Answer 4

MoE = 1 / (√n)

where n = sample size

x 100% to convert into a percentage

Question 5

When can you apply the “rule of thumb” method?

Answer 5

When you are calculating the MoE of a survey percentage that is between 30% and 70%

(note: sometimes if they ask you to calculate the MoE for a whole survey then you’ll use the rule of thumb but then only use that number for survey percentages between 30% and 70%)

Question 6

How can you use the Margin of Error to calculate a Confidence Interval for a single group?

Answer 6

X% +/- MoE

e.g. 45% with an MoE of 5%

45% +/- 5%

Confidence interval (40%, 50%)

Question 7

How can you interpret a confidence interval of a single group? e.g. (40%,50%)

Answer 7

• If there is a claim being made, base your interpretation of that e.g. if it says over 45% of NZers watch cricket, then you would say “because the confidence interval contains 45%, then we cannot make the claim that over 45% of NZers watch cricket”
• If there is no claim being made, make a statement like “With 95% confidence, I estimate that the percentage of New Zealanders who watch cricket is somewhere
  between 40% and 50%”

Question 8

How do you calculate the MoE of the difference between two groups when the two groups are from the same survey?

Answer 8

MoE (difference) = 2 x MoE

Question 9

How do you calculate the MoE of the difference between two groups when the two groups are from different surveys?

Answer 9

MoE (difference) = 1.5 x ((MoE1 + MoE2) / 2)

((MoE1 + MoE2) / 2) is the average MoE

Question 10

How do you construct a confidence interval when the two groups are from two different surveys?

Answer 10

(X2-X1) +/- MoE(difference)

e.g. if two groups are 23% and 28% and the MoE(difference) is 5%:

(29%-23%) +/- 5%
6% +/- 5%

(1% , 11%)

Question 11

How do you interpret a confidence interval when the two groups are from two different surveys?

Answer 11

If the confidence interval contains 0, then we cannot make the claim that the two groups are different

If the confidence interval does NOT contain 0, then we make the claim (with a 95% confidence) that the two groups are different

Question 12

What is non-sampling error?

Answer 12

The error that arises in a data collection process as a result of factors other than taking a sample.

Question 13

When thinking about non-sampling error, what kind of things can you think about?

Answer 13

• Who did they sample? Is the sample representative of the population that the claim is being made about?
• How did they sample? e.g. Telephone, Cell Phone, Email?
• When did they sample?

Question 14

What is a self-selecting sample? What is wrong with it?

Answer 14

When the sampling process allows individuals to select themselves. Individuals with strong opinions about the survey issues or those with substantial knowledge will tend to be over-represented, creating bias

Question 15

What is an “under representative sample”? What is wrong with it?

Answer 15

When the sampling process is such that a specific group is excluded or under-represented in the sample, deliberately or inadvertently. If the excluded or under-represented group is different, with respect to survey issues, then bias will occur.

Question 16

What do we mean by the response rate? Why is it important?

Answer 16

The response rate is the proportion of the sample that takes part in a survey. If the response rate is low, bias can occur because respondents may tend consistently to have views that are more extreme than those of the population in general

Question 17

What is a “double barrelled” question?

Answer 17

A question which is made up of two questions but only requires one answer.

This means the responses might not be accurate, as people might be saying “yes” to only one of the costs, but there is no way to know this from their response.

Question 18

What is the explanatory variable vs the response variable?

Answer 18

The explanatory variable causes the response variable.

e.g. if the two variables were amount of sleep and score on a test,

“amount of sleep” - explanatory variable
“score on test” - response variable

Question 19

What is a confounding variable? Why is it a problem?

Answer 19

A variable that affects both of the explanatory and response variables, but is either not known or is not acknowledged.

e.g. “amount of sleep” and “score on test” could have a confounding variable of “level of anxiety”.

If we don’t control for the confounding variable, then the relationship between the explanatory and response variables could be invalid, overstated or understated.

Question 20

What is an observational study? What is the main limitation?

Answer 20

In an observational study the researcher has no control over the value of the explanatory variable; the researcher can only observe the value of the explanatory variable for each individual and, if necessary, allocate individuals to groups based on the observed values.

The main limitation is that you cannot claim causation from an observational study because the experimenter cannot control the explanatory variable

Question 21

What is an experimental study?

Answer 21

In an experiment the researcher controls the conditions by allocating individuals to groups and allocating the value of the explanatory variable to be received by each group. A value of the explanatory variable is called a treatment.

In a well-designed experiment the allocation of subjects to groups is done using randomisation. Randomisation attempts to make the characteristics of each group very similar to each other so that if each group was given the same treatment the groups should respond in a similar way, on average.

Experiments usually have a control group, a group that receives no treatment or receives an existing or established treatment. This allows any differences in the response, on average, between the control group and the other group(s) to be visible

Question 22

Can you make a causal claim from an experimental study?

Answer 22

Yes, because the experimenter can control the explanatory variable.

Question 23

What do we mean by “correlation vs causation”?

Answer 23

Just because their is a statistical relationship between the explanatory and response variables, does not mean that one causes the other.

Question 24

What is an issue with inappropriately extending the results?

Answer 24

We cannot extend our claim to groups that aren’t investigated in the study.

e.g. We cannot say “all students who study 10 hours of week are more likely to pass” if the study involved only students from Wellington.

Question 25

What is an issue with using the past of a source of data?

Answer 25

People may overestimate or underestimate measurements when they think about the past e.g. How much have you used Facebook in the last month?

To deal with this, you could get all participants to use a log book and record when they go on social media instead, rather then asking them to remember the last month.

Question 26

What is a blind study?

Answer 26

An experiment where the subjects do not know what level of the explanatory variable they are getting.

Question 27

What is a double blind study?

Answer 27

An experiment where both the researchers and subjects do not know what level of the explanatory variable they are giving/getting.

Question 28

What is a placebo?

Answer 28

When a subject in an experiment gets zero of the explanatory variable