Statistics: Lecture quiz (15/15) Flashcards
Which of the following problematic practices are likely to constitute lying with statistics? Mark all correct options.
A) Making a claim for which there is absolutely no statistical basis.
B) Formulating the conclusion of a statistical study using a term such as “average”, without specifying, for instance, whether the median or mean is intended.
C) Using accurate data and valid statistical methods to draw a false conclusion.
D) Using inaccurate data to draw a true conclusion.
B) Formulating the conclusion of a statistical study using a term such as “average”, without specifying, for instance, whether the median or mean is intended.
C) Using accurate data and valid statistical methods to draw a false conclusion.
Which of the following claims about the choice and justification of statistical methods are true? Mark all correct options.
A) The use of a certain statistical method is a conventional matter, and the choice of this method can be left to a statistical program.
B) The available statistical methods will all yield the same result if applied to the same data.
C) The choice of statistical methods does not matter if you know which conclusion you need to arrive at.
D) The justification of statistical methods is an integral part of science.
D) The justification of statistical methods is an integral part of science.
Imagine that you are a researcher in psychology who is interested in developing a statistical test of the positive impact on one’s ability to perform well on standardized tests, by being exposed to so-called “intelligence-related words” (for example “smart”, “clever”, “genius” and so forth). To determine this, you measure people’s performance on such standardized tests before and after having been exposed to intelligence-related words. You note that 75 % of those who were subjected to intelligence-related words improved their score the second time they took the test. Is it true or false that one can conclude this result to be statistically significant solely on the basis of this information?
A) True
B) False
B) False
In the example from the lecture, the conclusions derived from the data set of the European Central Bank allowed for two opposing conclusions; namely that Germany was the worst off in terms of household income on average, and that they were, on average, somewhere in the middle. How is such a seeming contradiction possible? Mark the correct option.
A) It is possible because ‘average’ is ambiguous, and two different senses were used.
B) It is possible because there is no truth of the matter.
C) It is in fact not possible. People simply did not understand the data.
A) It is possible because ‘average’ is ambiguous, and two different senses were used.
Which of the following are examples of good statistical methodology? Mark all correct options.
A) Carefully choosing the statistical method that is the most suitable one to get a significant result, given the type of study you are performing.
B) Carefully choosing the statistical method that is the most suitable one to get a true result, taking into consideration the type of study you are performing.
C) Using any of the methods that are implemented in some high-quality statistics software.
B) Carefully choosing the statistical method that is the most suitable one to get a true result, taking into consideration the type of study you are performing.
Which of the following situations might occur depending on how we interpret ‘average’ in the sentence “this student’s test score is below the average score”? Mark all correct options.
A) If there are some low-scoring outliers (and no high-scoring outliers), then the student’s test score might be above the mean score but still be below the median score.
B) If there are some high-scoring outliers (and no low-scoring outliers), then the student’s test score might be below the mean score but still be above the median score.
C) If there are some low-scoring outliers (and no high-scoring outliers), then the student’s test score might be above the median score but still be below the mean score.
D) If there are some high-scoring outliers (and no low-scoring outliers), then the student’s test score might be below the median score but still be above the mean score.
A) If there are some low-scoring outliers (and no high-scoring outliers), then the student’s test score might be above the mean score but still be below the median score.
B) If there are some high-scoring outliers (and no low-scoring outliers), then the student’s test score might be below the mean score but still be above the median score.
What are some common sources of misunderstanding connected to the choice of statistical format? Mark all correct options.
A) It may be hard to understand how absolute risk reduction is connected to the “number needed for treatment” format.
B) A relative risk increase of e.g. 200% may be perceived as alarmingly large if one doesn’t understand that the initial risk was very low.
C) A relative risk reduction may be overestimated when compared to the same risk reduction in absolute numbers.
A) It may be hard to understand how absolute risk reduction is connected to the “number needed for treatment” format.
B) A relative risk increase of e.g. 200% may be perceived as alarmingly large if one doesn’t understand that the initial risk was very low.
C) A relative risk reduction may be overestimated when compared to the same risk reduction in absolute numbers.
Under what circumstances do we have reason to evaluate a hypothesis statistically? Mark all correct options.
A) When the hypothesis is stated in terms of statistical notions, such as ‘mean’ and ‘median’, we should always apply statistics.
B) We should always do so. Otherwise, the testing procedure is unscientific.
C) For some hypotheses, the only relevant implications are stochastic. The truth of these implications can be determined precisely only with the help of statistical tools.
D) In order to account for measurement errors and similar disturbing influences, one might want to determine the acceptable error of a test. Quantifying this error requires statistical methods.
E) In order to correctly determine how much a set of observational data changes one’s confidence in a hypothesis, statistical methods are useful.
C) For some hypotheses, the only relevant implications are stochastic. The truth of these implications can be determined precisely only with the help of statistical tools.
D) In order to account for measurement errors and similar disturbing influences, one might want to determine the acceptable error of a test. Quantifying this error requires statistical methods.
E) In order to correctly determine how much a set of observational data changes one’s confidence in a hypothesis, statistical methods are useful.
Which of the following claims about p-values are true (in the context of Fisher’s version of a significance test)? Mark all correct options.
A) P-values are set by convention.
B) Given a hypothesis H (with a test statistic T) and a data set D, the p-value is the probablity of D (or any more extreme outcome) given H. That is, it is the probability of getting the outcome one actually got, or a more extreme one, given that one’s hypothesis is true.
C) P-values are compared with a significance level
D) Given a hypothesis H (with a test statistic T) and a data set D, the p-value is the probablity of H given D. That is, it is the probability that one’s hypothesis is true, given the outcome of one’s test.
B) Given a hypothesis H (with a test statistic T) and a data set D, the p-value is the probablity of D (or any more extreme outcome) given H. That is, it is the probability of getting the outcome one actually got, or a more extreme one, given that one’s hypothesis is true.
C) P-values are compared with a significance level
Imagine a researcher who is investigating whether changing the materials used to conduct electricity in central processing units (CPUs) positively affects its efficiency by a factor of 1.12. The researcher formulates the hypothesis “the effect of changing the materials is less than 1.12”. When failing to reject this hypothesis, the researcher decides (against good research practice) to perform p-value abuse. Which of the following alternatives constitute such abuse? Mark all correct options.
A) Expanding the sample in order to find a data set that (in isolation) has a significantly low p-value in relation to the stated hypothesis, and then choose to report only that data set.
B) Purposely choosing a significance level that is higher than the p-value calculated for the test data.
C) Rejecting the stated hypothesis without reporting the p-value of the data.
D) Using the same data sample and, with the only reason being to generate a statistically significant result, modifying the value in the hypothesis to 1.15, which gives a significant p-value.
A) Expanding the sample in order to find a data set that (in isolation) has a significantly low p-value in relation to the stated hypothesis, and then choose to report only that data set.
B) Purposely choosing a significance level that is higher than the p-value calculated for the test data.
D) Using the same data sample and, with the only reason being to generate a statistically significant result, modifying the value in the hypothesis to 1.15, which gives a significant p-value.
Neyman-Pearson’s method of hypothesis testing allows researchers to choose between two competing hypotheses. What is true about this method? Mark all correct options.
A) The hypotheses must be mutually exclusive and jointly exhaustive.
B) Usually, one of the hypotheses is taken to be the main hypothesis. The other can be just about any given hypothesis, provided that it functions as an auxiliary hypothesis in some possible empirical test of the first hypothesis.
C) Committing a type I error in a test excludes the possibility of a committing a type II error at the same time.
D) Rejecting one hypothesis entails the acceptance of another.
A) The hypotheses must be mutually exclusive and jointly exhaustive.
C) Committing a type I error in a test excludes the possibility of a committing a type II error at the same time.
D) Rejecting one hypothesis entails the acceptance of another.
Suppose you have set up a Neyman-Pearson test with the hypothesis Hi: “the new advertising campaign doesn’t increase the sales of the advertised products”. What is true about the power of the test? Mark all correct options.
A) If your test is high-powered and the campaign does in fact increase the sales, then there is a high probability that Hi will be rejected by the test.
B) If your test is high-powered and the campaign does in fact increase the sales, then there is a high probability that Hi will be accepted by the test.
C) Collecting data from more people in the area targeted by the campaign will make your test less probable to reject Hi in situations when Hi is in fact true.
D) Collecting data from more people in the area targeted by the campaign is one way to increase the power of the test.
E) Collecting data from more people in the area targeted by the campaign will make your test less probable to accept Hi in situations when Hi is in fact false.
A) If your test is high-powered and the campaign does in fact increase the sales, then there is a high probability that Hi will be rejected by the test.
C) Collecting data from more people in the area targeted by the campaign will make your test less probable to reject Hi in situations when Hi is in fact true.
D) Collecting data from more people in the area targeted by the campaign is one way to increase the power of the test.
E) Collecting data from more people in the area targeted by the campaign will make your test less probable to accept Hi in situations when Hi is in fact false.
In a sentence, what is the focus of Bayesian statistics? Mark the correct option.
A) Calculating the probability of the hypothesis being true, given the evidence.
B) Calculating the probability of observing the evidence, given that the hypothesis is true.
A) Calculating the probability of the hypothesis being true, given the evidence.
Three different problems with Bayesian statistics were mentioned in the lecture: (1) the problem of determining priors, (2) the problem of old evidence, and (3) the problem of uncertain evidence.
Match each problem description (A-D) to the correct problem (1-4)
A) Bayesian statistics require that we quantify prior probabilities, yet does not specify how such probabilities should be determined. Thus, people with radically different priors might not be able to come to an agreement on whether a hypothesis is more likely to be true or more likely to be false - even if they have access to the same data.
B) Prior probabilities should not be determined on the basis of old evidence - yet, we often have nothing else to go on.
C) Sometimes, it is hard to distinguish the evidence used in determining the prior probability of a hypothesis from the evidence used in a Bayesian evaluation of the probability of that hypothesis.
D) Bayesian statistics typically assumes that the probability of the evidence is 1. Yet, in reality, we are rarely (if ever) justified in being so certain about the strength of our evidence.
1) The problem of determining priors
2) The problem of old evidence
3) The problem of uncertain evidence.
4) Non of the three problems
A - 1
B - 4
C - 2
D - 3
What is true about the differences between Fisher’s significance testing, Neyman-Pearson hypothesis testing and Bayesian hypothesis testing? Mark all correct options.
A) Neyman-Pearson hypothesis testing avoids some kinds of p-value abuse that Fisher significance testing is threatened by.
B) Fisher tests and Neyman-Pearson tests include two hypotheses that are tested simultaneously, but Bayesian tests don’t.
C) Bayesian hypothesis testing can mirror the idea that humans have prior beliefs about the truth and falsity of hypotheses. This cannot be done with Fisher and Neyman-Pearson.
D) As opposed to a Neyman-Pearson test, the hypotheses in a Bayesian test need not be mutually exclusive.
E) Neyman-Pearson testing is the only one out of the three where the test can reject a hypothesis.
A) Neyman-Pearson hypothesis testing avoids some kinds of p-value abuse that Fisher significance testing is threatened by.
C) Bayesian hypothesis testing can mirror the idea that humans have prior beliefs about the truth and falsity of hypotheses. This cannot be done with Fisher and Neyman-Pearson.
