Data Analysis in Real Life

Question 1

What are Some methods for helping reports have a more clear narrative include

Answer 1

1. eliminating jargon and focusing on interpretability
2. critiquing significant effects by coming up with potential alternate explanations
3. focusing on simpler models and parsimony

Question 2

What is Version control software used for

Answer 2

Keeps track of checked in versions of code, data and reports

Question 3

What are Some easy things to double check reports include

Answer 3

1. verifying the signs of effect are in the obvious direction
2. checking magnitude of effects by comparison with other known effects
3. putting units on graphs and coefficients and generally keeping track of units

Question 4

What do Reproducible report writing tools like knitr and ipython help with

Answer 4

1. by automating the report writing process
2. by organizing ones thinking by blending the code and the narrative into a single document
3. by documenting the analysis code with the project narrative
4. by advancing the goal of reproducibility

Question 5

What are two components that make for good final data products that are ubiquitous across all settings

Answer 5

making the report reproducible and

making the report and code version controlled.

Question 6

The reason you get a null result it may be due to

Answer 6

low power

that the null hypothesis is actually correct

Question 7

A study with a very low sample size will likely have

Answer 7

low power

Question 8

Calculating power after the study has been done and analyzed is

Answer 8

problematic and should only be done by people well versed in the issues

Question 9

Ideally, a surrogate variable for variable of interest will

Answer 9

1. have a known or estimable variance around the desired measurement variable
2. be unbiased

Question 10

What is the idea of power

Answer 10

Power is the probability of rejecting the null hypothesis when it’s false.

Question 11

What can you do In the absence of any calibration data to evaluate your surrogate

Answer 11

either modeling via assumptions or sensitivity analyses.

Question 12

What to do if your surrogate variable is such an unreliable of an estimate of your actual outcome

Answer 12

one must come to the conclusion that it’s better to not conduct the study at all.

Question 13

Potential problems with testing lots of hypotheses until a significant one is found include

Answer 13

Declaring effects that are not significant as significant by chance

Misrepresenting the strength of the findings

Question 14

What is Comparing your effects to familiar ones useful for

Answer 14

Mentally calibrating the size of an effect or its significance when a variable under study is not well understood

Question 15

Negative control analyses are useful for

Answer 15

for evaluating processes to see if spurious effects are obtained

as a validity check of an effect of interest by looking to see if similar effects occur with the same analysis on variables where an effect is known not to be present

Question 16

A good negative control analysis will

Answer 16

have a negative control that is known not to have an effect but is otherwise similar to the variable under study

Question 17

What is hypothesis testing

Answer 17

In hypothesis testing, we use a statistic to decide between two hypotheses. We set one as the default hypothesis (null hypothesis) and the other as the alternative.

Question 18

The result of a hypothesis test is summarized with a p-value.What is p-value?

Answer 18

A small p-value (close to 0) supports the alternative while a large one (close to 1) supports the null. We reject the null if our p-value is less than 0.05 if we want to control the probability of incorrectly rejecting the null at 5%.

Question 19

What are potential problems with multiple comparisons

Answer 19

The probability that we see apparently significant findings simply by chance even though they’re not actually significant increases.

Question 20

In A/B testing randomization of a treatment is used for

Answer 20

To make groups as comparable as possible

Attempt to balance potential unobserved confounding variables

Question 21

Three strategies to combat sampling bias are

Answer 21

Random sampling

Modeling

Weighting

Question 22

It is generally a good idea to consider possible confounders when considering a significant effect

Answer 22

TRUE

Question 23

It’s possible for a regression effect to reverse itself after the inclusion of another variable into the model

Answer 23

TRUE

Question 24

What are Blocking and adjustment are tools used for

Answer 24

Account for variables potentially impacting the estimation of the effect of interest.

Question 25

If we see an association between two variables, it would be a good idea to

Answer 25

consider the possibility that the association is explained by a confounding third variable.

Question 26

You see an effect of ice cream sales on the number of heat exhaustion cases. The effect is likely due to:

Answer 26

The hot weather as a confounder.

Question 27

Associations can imply causality

Answer 27

under a set of strict assumptions often as a result of design choices.

Question 28

What is confounding

Answer 28

Confounding occurs when you want to compare two things and a third gets in the way.

Question 29

Define causal effects

Answer 29

We define causal effects as the difference between the outcome for a subject observed at a particular treatment minus the outcome observed as a a control

Question 30

What is Casual Inference

Answer 30

The study of how to estimate causal effects using data is called causal inference.

Question 31

What must The residuals consider:

Answer 31

the difference between the response and the fitted value

Question 32

Summary tables should include

Answer 32

Quantiles

Means

Standard deviations

Question 33

In leading digits of data that follow Benford’s, the digits (0-9) are all equally likely

Answer 33

FALSE

Question 34

Merging (linking two datasets via a common index) errors can have a strong impact on subsequent analyses

Answer 34

TRUE

Question 35

How do you keep on top of data quality without being in the trenches?

Answer 35

The construction of summary tables.
Regression diagnostics.
Residuals -