Statistics

Question 1

Can you ever be sure about disproving a hypothesis?

Answer 1

No you cannot be completely sure, however you can be arbitrarily sure if the results are statisitically significant

Question 2

What does it mean for a result to be statistically significant?

Answer 2

A result is called statistically significant if it is unlikely to have occurred by chance. Normally meaning that the p-value is less than 0.05 (5%). (However can alter the threshold)

Question 3

What is the p-value?

Answer 3

The p-value is the probability of obtaining the given results if the null-hypothesis is true.

Question 4

What is a null hypothesis?

Answer 4

A null hypothesis is what is assumed to be true and is being tested against to be disproven. Functionally meaning that both data sets are from the same mechanism, wheras we are trying to prove they are different aka the alternate hypothesis.

Question 5

How to prove/disprove something with stats

Answer 5

• It is not possible to prove/disprove something with stats
• You can only reject the null hypothesis given enough statistically significant data
• Otherwise the test “didn’t find a statistically significant difference” and “fails to reject the null hypothesis”

Question 6

What is a research question?

Answer 6

A statement that identifies a phenomenon to be studied.
Ex: I believe that rewards improve memorization skills

Question 7

What is a hypothesis?

Answer 7

• A statement of the predicted relationship between at least two experimental variables.
• A provisional answer to a research question
• Ex: group chocolate will have a higher memorisation score than group with no reward

Question 8

Independent vs dependent variable

Answer 8

The dependent variable is the event studied and expected to change whenever the independent variable is altered.

Question 9

What is a controlled variable

Answer 9

The variables that are** kept constant** to prevent their influence on the effect of the independent variable on the dependent. Ideally everything besides dependent and independent variable is controlled.

Question 10

What is a confounding variable

Answer 10

Extraneous variables that correlates with both the dependent variable and the independent variable.
Example: Weather temperature correlates with both ice-cream sales and murders.

Question 11

The goal of experimental design

Answer 11

Experimental design aims at maximizing your chances of finding the signal and not the noise (noise being randomness, confounding variables etc, that may show correlation not causality)

Question 12

Within vs. between subjects

Answer 12

Within = All participants do the same thing (everyone does A and B)
Between = Certain participants do only certain conditions (certain people do A, certain people do B)

Question 13

Comparison of within vs. between experiments

Answer 13

Within pros:
+ Less user variation (between groups)
+ Statisical power with less participants

Between pros
+ No baises from other conditions (eg. transfer of learning from doing A before B)

Question 14

What is counterbalancing?

Answer 14

A method of avoiding confounding among variables/
Presenting conditions in a different order

Question 15

How is a latin square used for counterbalancing?

Answer 15

A latin square is an n × n array filled with n different Latin letters, each occurring exactly once in each row and exactly once in each column, where each letter corresponds to treatment/condition. Varying the order in this way avoids counfounding variables and transfer of learning.

Question 16

Sweet spot of number of trials.

Answer 16

Ideally as many trials as possible but 30 - 40 is the sweet spot.

Question 17

What does a t-test measure?

Answer 17

T-statistics tell us how many standard errors away from the mean our observed difference is.

Question 18

What is bonferroni correction?

Answer 18

To reduce type I error when testing n hypotheses, test each one against 0.05/n. This is because when conducting n tests, the chance of one of them being invalid increases by a factor of n (0.05 * n)

Question 19

Use of a t-test

Answer 19

Comparing two groups

Question 20

What is an ANOVA test used for?

Answer 20

An ANOVA is an analysis of varience and is used to compare multiple variables. Often ANOVA tests prove there is a significant difference and follow-up t-tests show where there difference is.

Question 21

What is regression?

Answer 21

A machine learning technique for determining the statistical relationship between two or more variables where a change in a dependent variable is associated with, and depends on, a change in one or more independent variables. Basically drawing a line which minimises the distance to each data point.

Question 22

Ways to determine the goodness of fit for a regression

Answer 22

1. Standard error
2. R squared

Question 23

How to calculate standard error

Answer 23

Lower is better

Question 24

How to calculate r squared

Answer 24

Percentage, higher is better.

Question 25

What is a CHI-Square used for

Answer 25

When both the independent variable and dependent variable are categorical. Can have multiple variables if needed

Question 26

CHI-Square formula

Answer 26

Use a table to find p from X^2

Question 27

CHI-Square degrees of freedom (contingency table)

Answer 27

(number of
rows−1)∗(number of columns−1)

Question 28

How to operate a CHI-Square contigency table

Answer 28

1. We compute the sums in all direction
2. For each cell, multiplying that cells row and column totals and dividing by our total sample size
   e.g. case (sport, male)= (29 * 50) / 75 e.g. case (family, male= (46 * 50) / 75 …
3. Use the Chi-square formula
4. Calculate degree of freedom as DF =(number of
   rows−1)∗(number of columns−1) (here = 1)
5. Use the Chi-square table to conclude!