Research 2 Exam 2 Study Guide Flashcards
What was the point of the video we watched?
The point of the video we watched was to show us an example of how a researcher had broken ethics in the way of fraud/fabrication of data. A student showed this to Lewandowski and it is a more interesting way to teach the topic of open science.
What’s the difference between replication and reproducibility? Which is more likely to work? Why?
Replication means that we can use the data they provide, run what they ran, and receive the same results as the study. This is a check to make sure that they did their analyses correctly and did not complete an underhanded act of breaking ethics.
Generally, what are questionable research practices?
Practices that are sketchy. They seem suspicious and the choice is not appropriate to a researcher.
What is HARKing? b) Distinguish between testing hypotheses & exploratory findings.
Harking is hypothesizing after collecting data and analysis. In this questionable practice, they find data and then say oh yeah I expected this to happen. Liar, you just looked for a pattern and then claimed it all along. When our studies go through the IRB, we have to submit our hypothesis among other information in advance to any data collection. Testing your hypothesis means that you formulate your research question and then you set up your experiment, collect data, and then in your findings, determine if your hypothesis was supported or rejected. Exploratory findings are not created before the experiment, but done after you have the data. They are just some thing you wanted to look into or explore but not a hypothesis you test.
What is p-Hacking? What are the 3 ways of doing that?
P-hacking is probability hacking, which means you affect your probability.
1.Cherry picking
2.Run your study until it works (e.g., analyze every 5 participants.
3.Play with variables to get them to work (e.g., selectively drop/include items)
How often does falsifying data/fraud happen?
Not very often, it was reported 0.6 the BTS nerds who check the data found 1.7, although doubled, is still pretty low.
What is preregistration? Why do we do that? Will this stop fraud/fabrication?
stating your hypothesis before conducting your study. Often in a public forum. We do it to show that out hypothesis was in fact created prior to the testing. It will prevent not stop fraud because there are multiple ways to commit fraud, this would only be stopping people from testing the study and then claiming that they knew it all along. It would not stop people from faking data.
What are the three tenets of open science? Describe each. Which one helps with reproducibility? Which one helps with replication?
preregistration (keeps you honest), open data (every one can analyze your exact same data) reproducibility and open materials (replicability) can redo your study with their own participants.
a) Give your own example of a double negative (including what it actually says). b) How does this apply to hypothesis testing?
a) I don’t disagree with you (I agree with you), you ain’t going nowhere(you’re going somewhere)
b) fail to reject the null
a) I am not never going to study. ( I am going to study)
b)In hypothesis testing, in order to reject the null we make a double negative. The null is that there is no difference, we show that there is not no difference (there is a difference) and we reject the null.
What is the null hypothesis?
states there is no effect (change, difference) and the populations are the same The opposite of what we are hypothesizing. SImilar to a straw man argument we are setting it up to hopefully tear it down and support our hypothesis.
Hypothesis test require that we start off assuming we’re wrong (with the null hypothesis). Give your own example of how this approach could be beneficial in another context.
This approach could be beneficial if we were in an ethical dilemma. If I thought it was possible my boss was committing fraud, but I did not have proof. It is better to assume I am wrong, and she is not committing fraud. THen I would talk to my boss and ask what they are doing. If it is fraud then I would report it, but if it isn’t then I end the process there. (If i assumed there was fraud when there was not (type 1 error) then they would have done all the extra processing and investigating to turn up with no fraud.
a) List the 5 steps of hypothesis testing? b) In your words, describe what we’re generally doing in each. (we talked a bit about this in class, but also refer to the Table on pg 188)
a) First you have to label(establish) your population and your hypothesis. You need to know who you want to test/take a sample from and what you are looking for.
Then you need to build a comparison distribution
Third you establish the critical value cutoff
FOurth you determine the sample results
Finally you decide and interpret
b)state who you want to test/take sample from and what you are looking for (hypothesizing)
Then you need context so you have to see what or who you are comparing to. You should create a normal distribution.
Next, you establish the cutoff you must state what value or percentage you must reach in order for your result to count.
After that, you determine your sample results, conduct your study and receive the findings
Last, you compare the sample results to the cutoff and make a decision about the study.
Distinguish between Type 1 and Type 2 error. Which is a false positive? Which is a false negative?
A type 1 error is when we claim something is true when it is false. (We say there is a ghost, when there is not ghost). This is a false positive because we think it is true but in actuality it is not. Fake true.
A type 2 error is when we claim something is false when it is true. We fail to reject the null when we should. (We say there is nothing there when there actually is a ghost). This is a false negative because we think it is false but in actuality it is not. Fake False.
Interpreting p values. is this significant?
a) 5.43 4.47 N=1000
most definitely
Interpreting p values. is this significant?
b) 5.43 4.27 N=63
probably not if 63 is small sample
Interpreting p values. is this significant?
8.21 8.64 N=1,999,999
Most Definitely
Interpreting p values. is this significant?
8.64 8.21 N=6
Definitely Not
Interpreting p values. Is this significant?
5.55 5.73 N= 200
definitely not
Interpreting p values. Is this significant?
6.78 9.91 N=200
most definitely
Interpreting p values. effect size? significant? meaningful?
a) 5.43 4.27 p=.03
small effect, significant, not meaningful
Interpreting p values. effect size? significant? meaningful?
5.43 4.27 p=.78
small effect, Not significant, not meaningful
Interpreting p values. effect size? significant? meaningful?
8.21 8.64 p=.002
small effect size, significant, not meaningful
Interpreting p values. effect size? significant? meaningful?
8.64, 8.21 p=.10
small effect size, Not significant, not meaningful
Interpreting p values. effect size? significant? meaningful?
5.55 3.98 p=.000
Big effect size, impossible significance, meaningful?
Interpreting p values. effect size? significant? meaningful?
6.78 6.03 p<.001
small effect size, significant, not meaningful
Interpreting p values is this significant?
d=.15 N=6000
most likely
Interpreting p values is this significant?
d=.85 N=58
Most likely
Interpreting p values is this significant?
d=.05 N=100
probably not
Interpreting p values is this significant?
d=.62 N=1547
most likely
How does the mean of the distribution of means compare to the mean of the population of individuals? Why does this happen?
Equal to the mean of popul - all scores from og population. Taking samples plot for distribution, the mean is
Explain in your own words how to create a Distribution of Means.
To create a distribution of means one would take the mean of every 5 people in a sample, over and over and then plot those means. Doesn’t always have to be five, taking samples mean of sample over and over.
In your own words, what does a t-test for dependent means do/tell us? (i.e., looking at the formula, what is it doing? What does it compare?)
Compares within-subjects groups to see if there is a change from the first time the group is tested to the second time the same group is tested. For the t-test formula, we have the average difference subtract the comparison distribution to find difference, then divided by the average variability (with standard deviation)
What types of variables to the t-test for Dependent means test analyze?
Nominal/categorical independent variables, interval ratio continuous dependent variables
What research design does the t-test for Dependent means test analyze?
Strawman, like a z-score test? It is a t-test for dependent means THIS IS A WITHIN SUBJECTS DESIGN
t-test for dependent means Give 2 examples of a research question that fits this design (one related to everyday life and one related to research)
Everyday life: Does coffee improve test scores? Research: Are people more willing to break the rules after doing so once?
a) What is a difference score? b) How do you calculate it?
A difference score is the subtraction of the post to the pre or after to before. You calculate this by looking at the first time the group was tested and subtract that by the second time the same group was tested.
a) What is the comparison distribution for the t-test for dependent means? b) What is the mean of this comparison distribution? c) Why?
a) The comparison distribution is a distribution of means of DIFFERENCE SCORES “sampling distribution”
b) The mean is 0
c) because it always is, it represents the null hypothesis that there is no change from the first time the group was tested to the second time the same group was tested.
Do you want to have a lot of degrees of freedom, or fewer degrees of freedom? Why?
Yes, you want to have a lot of degrees of freedom. The degrees of freedom are the number of the participants in the sample minus 1. We want our sample size to be large and more representative. A larger sample would mean a larger degree of freedom. Additionally, by having more df, our Distribution becomes MORE NORMAL.
You run a study and find the mean difference score in your sample is 3.44. Is that a lot? How would you know?
Maybe? We do not yet know. The only way to know is with the context of the typical difference score. (If the typical is 3, than our sample is not that difference, if the typical is 1, than yeah our sample is pretty large.)
You run a study and your sample’s average difference score is 2.78. The comparison distribution has a mean of____and a standard error of 2.59. Is the difference in your sample big or small?
0; Small because we would be just slightly higher than one standard deviation from the mean. It is likely that our difference score is not big enough to pass the cutoff.
a) If your study fails to reject the null hypothesis, what does that mean? b) What two things can you do to avoid this happening?
Our hypothesis was not supported by the results of the study. We did not reach the cutoff we needed for our test to be significant. increase sample size and increase effect size.
When running a study, why do we include items that we don’t plan on analyzing?
Some times we place distractor items within our study?
When communicating results, what is the key with everyday language?
Write it as if your grandma or uncle could understand exactly what you are talking about without any statistics knowledge.
t (53) = -2.27, p = .03, d = .42.
Which number(s) is/are the degrees of freedom?
53
t (53) = -2.27, p = .03, d = .42.
Which number (s) is/are the significance level? Is it significant?
.03, yes
t (53) = -2.27, p = .03, d = .42.
Which number(s) is/are the effect size?
.42