Class 8: Research methods in cognition, levels of analysis, generalisability, control, and open science Flashcards
1
Q
What was the finding in Dasgupta and Greenwald’s study that Brian Nosek spoke of?
A
The study was investigating whether exposing people to positive black individuals would reduce their implicit race bias – measured by IAT. Joy-Gaba and Nosek believed that it was likely associative between ‘good’ and ‘black’ exemplars and acted to reduce implicit race bias. They did not get the original result with replication due to failure of replicating an important moderator.
2
Q
The________ is the process of conducting a study again, with the goal of verifying the original findings.
A
Replication
3
Q
Is there is persuasive evidence that individuals know what things are more likely to replicate than others?
A
Yes. Nosek highlighted in his talk ‘What is Replication’ that there were prediction market priced bets placed based on probability of replication with the results indicating that individuals have some insight in likelihood of replication. Some prediction of probability of replication is said to help sharpen our theoretical expectations.
4
Q
Mook (1983) says that the representativeness of a sample is a) always the most important factor in designing an experiment, b) especially important if the research is not in survey form, or c) depends on the purpose of the research
A
c) He says the representativeness of the sample is more or less important depending on the purpose of the research. For instance if it is a survey, it is very important, if it is a case study then it may be less so.
5
Q
Fill in the blank: “External invalidity can occur when the specific conditions of the __________ do not match those of the real world.”
A
Study
6
Q
A common definition for replication is “repeating a study’s procedure and observing whether the prior results recur.” Nosek defines replication as “an attempt to reproduce a previously observed finding with no a priori reason to expect a different outcome”. What is the key difference in these definitions?
A
In Nosek’s definition, nothing about procedural similarity is included. His definition is centred around theoretical expectations as he suggests that claiming that something is a replication is itself a theoretical commitment.
7
Q
Why should we not discard all research due to external validity?
A
Misplaced preoccupation with external validity can result in dismissing good research for which generalisation to real life was not intended or meaningful.
8
Q
True/False Argyle dismissed Milgram’s work on obedience as absurd and unethical
A
FALSE
9
Q
According to Brian Nosek, replication is not necessary for scientific progress because it simply confirms what we already know.
A
False. According to Brian Nosek, replication is critical for scientific progress because it helps to establish the reliability and validity of research findings.
10
Q
Define the term “External Validity”.
A
External Validity - The extent to which the findings of a study can be generalized to other populations, settings, and times beyond the specific context of the study.
11
Q
True or False: Parents react to the form of a child’s speech, not its content.
A
Answer: False Explanation: Direct observation of parent-child interactions reveals that parents react to the content of a child’s speech, not to its form.
12
Q
What does Mook (1983) suggest as a more appropriate criterion for evaluating psychological research?
A
Instead of focusing on external validity, Mook (1983) proposes that researchers should emphasize internal validity and the contribution of their work to the understanding of psychological processes and theories. He believes that well-designed studies with strong internal validity can provide valuable insights into human behavior, even if they do not directly generalize to all populations or situations. By building a robust body of evidence through numerous studies, researchers can gradually enhance the external validity of their findings.
13
Q
How is ‘replication’ defined in Brian Nosek’s talk?
A
Repeating a study’s procedure and observing whether the prior finding recurs.
14
Q
What does Brian Nosek propose the revised definition of replication is?
A
Attempting to reproduce a previously observed finding with no a priori reason to expect a different outcome.
15
Q
The purpose of replication is to increase the reliability of research findings.
A
TRUE
16
Q
True or False: The concept of external validity was developed with a specific type of research in mind, namely experiments designed to evaluate the effectiveness of new teaching procedures or enrichment programs on culturally deprived populations.
A
TRUE
17
Q
Instead of predicting the real world from lab experiments and results, psychologists are instead predicting …?
A
Psychologists are testing what occurs in labs, then refining their theory to test if it will occur in lab settings
18
Q
What is the revised definition of Replication? (As mentioned in the video)
A
It is the process of attempting to reproduce a previously observed finding with no a priori reason to expect a different outcome.
19
Q
How can we try to reduce the impact of false negatives (or the likelihood)?
A
We can minimise false negatives by obtaining original materials.
20
Q
What are the reasons provided by Nosek for why procedural similarity is a pragmatic solution currently in replication research?
A
’- We lack theoretical specificity - There is a lack of understanding of the methods of studies and what modifications will produce differing results
21
Q
How can a study’s external validity be enhanced?
A
A study’s external validity can be enhanced by conducting the study in multiple settings, recruiting a diverse sample of participants, and varying the temporal characteristics of the study.
22
Q
What are some of the benefits of open science practices, as advocated by Nosek?
A
Some benefits include increased transparency in research, improved collaboration, and greater reproducibility of findings.
23
Q
When building evidence about theoretical expectations, generalisability is where we have low/high commitment. When committing to something as a replication, we have low/high commitment.
A
When building evidence about theoretical expectations, generalisability is where we have LOW commitment. When committing to something as a replication, we have HIGH commitment.
24
Q
What is external validity according to Mook’s article “In defense of external invalidity”?
A
Answer: External validity refers to the extent to which research findings can be generalized to real-world settings and populations beyond the sample used in the study.
25
Mook (1983) suggests that external validity is not important in experimental research, as long as the findings are internally valid.
Mook (1983) argues that external validity is important in experimental research, and that researchers should aim to balance both internal and external validity in their studies. Mook states that sacrificing external validity for internal validity can limit the usefulness of findings in real-world settings, and suggests that researchers use a variety of research methods to increase the generalizability of their findings.
26
What is external invalidity?
External invalidity refers to the inability to generalise the results of a study to other populations, settings, or time periods.
27
True or False: Experiments that were conducted in real-life settings will still have some limits in their generalizability.
TRUE
28
True or False? Justifications for limited external validity may include experimental control, repeatability, and the need for specific research contexts.
TRUE
29
According to Brian Nosek, should replication be framed in conceptual or procedural terms?
Conceptual terms
30
What does Mook state claim is the biggest source of confusion regarding the debate around External Validity?
The purpose of collecting data in the laboratory is to pred t real-life behaviour in the real world--this is not always the case.
31
What did the original discussion of EV by Campbell and Stanley(1967) reveals?
They reveal that the experimental investigation they had in mind was a rather complex mixed design with pretests, a treatment and a posttest
32
Which of the following statements best summarizes the main argument of Mook's article "In defense of external invalidity"? A) External validity is a minor concern in research and can be overlooked in favor of internal validity. B) The concept of external validity is flawed and should be abandoned altogether. C) External validity is crucial for the practical application of research findings and should be given more attention. D) Researchers should prioritize the use of laboratory experiments over field studies to maximize external validity.
Answer: C) External validity is crucial for the practical application of research findings and should be given more attention.
33
In what kind of research is representativeness important, according to Mook?
Survey research
34
What must be considered in the checklist of decisions, and what does Mook mean when he talks about the specifics of the sample and setting?
(A) We Must consider what conclusion wants to be drawn and (B) whether the specifics of our sample or setting will prevent us from drawing it. Sample > Are you trying to (A) estimate sample characteristics to those of a particular population, or (B) trying draw conclusions not about a population but rather a theory that specifies what these subjects ought to do, or ( C ) if any subject does or can be made to do this or that Setting > is the intention to predict (A) what would happen in a real-life setting or (B) "target" class of such settings
35
What are some strategies proposed by Mook (1983) to enhance external validity in research studies?
Mook suggests several strategies to enhance external validity, including the use of more realistic and diverse samples, the replication of studies across different settings and populations, and the examination of the generalizability of findings through meta-analyses and other methods. He also emphasizes the importance of reporting effect sizes and statistical power to evaluate the practical significance of study findings.
36
What are some of the challenges associated with replicating studies in psychology?
Some of the challenges include differences in sample characteristics, variations in experimental procedures, and variations in measurement instruments.
37
True or False: According to the study by Brown and Hanlon (1970), parents react to the content of a child's speech rather than its form?
True.
38
Ecological Validity
The realism with which a design of evaluation setup matches the user's real work context; how accurately the design or experiment reflects the relevant characteristics of the ecology of interaction
39
True or False: According to Mook (1983), context-specific research is less valuable than research that can be generalized to multiple settings and populations.
Answer: False Explanation: Mook argues that context-specific research can provide valuable insights into specific situations or populations. Rather than aiming for generalizability, these studies seek to gain a deeper understanding of a particular context. In some cases, Mook suggests that external invalidity can even be a strength, as it allows researchers to uncover unique insights that may not emerge from more generalizable studies.
40
Mook argues that one origin of the misconceptions about EV, its goals and why we should absolutely reach it lies in the “feeling” that comes with words such as “validity” and “invalidity” a priori. True or False?
True. Cf “Warm Fuzzies and Cold Creepies” paragraph
41
Why are some areas of psychology further ahead at addressing the issues surrounding replication compared to others?
Areas such as social psychology are further ahead as experiments are easier to replicate. Areas such as infant or clinical psychology are further behind because it is harder to replicate studies.
42
What are the two reasons highlighted in the youtube video that experiments are conducted through procedural similarity rather than altering the procedure but ensuring the expected result is the same.
Lack of specificity Lack of understanding the methods
43
True/False The assumption that laboratory settings should resemble real-life ones as much as possible is true.
FALSE
44
Three questions to consider when interpreting a failure to replicate
is the original study a false positive, is the replication a false negative, are both true effects.
45
Fill in the blank: Mook argues in his article, In Defense of External Invalidity, that a misplaced preoccupation with external ________ can lead us to dismiss good research for which generalization to real life is not intended or meaningful.
Validity
46
Why are conclusions that cannot be generalized important?
They can contribute to the understanding of the process. It can reveal what can occur, not necessarily what happens in real-life conditions.
47
Brian Nosek believes that all scientific studies should be replicated. True or False?
False : Brian Nosek believes that not all studies need to be replicated, and that replication efforts should be focused on studies that are particularly important, influential, or controversial.
48
What are some potential limitations of using college students as research participants and how might these limitations affect the external validity of research findings?
College students may not be representative of the broader population and may limit the external validity of research findings. They are often more educated, younger, and more diverse than the general population, and they may have different life experiences and attitudes that affect their behaviour. This can limit the generalizability of research findings beyond the college student population.
49
What is grandiose narcissism?
A subtype of narcissism characterized by an inflated sense of self-importance, a need for admiration, and a lack of empathy for others.
50
In discussing external validity, Mook (1983) says that in experiments we are not ‘making generalisations’, but rather we are a) examining them b) testing them c) appraising them d) evaluating them
b) testing them
51
How can experiments which lack EV (and also provide seemingly silly results) still provide value?
Findings, no matter its apparent value, can contribute towards an underlying theory of the world
52
According to Mook 1983, external validity "asks a question and invites us to think of prior questions", what is external validity NOT?
an automatic desideratum
53
Fill in the blank: Every replication, because they're never identical, is a test of _.
Generalizability
54
What were cambell and Stanley (1967) aims
To evaluate the effect of an enrichment program for the culturally deprived
55
Why is external invalidity sometimes considered to be less important than internal validity?
External invalidity is often considered less important than internal validity because it does not threaten the causal conclusions drawn from the study.
56
To what kind of experiment does Mook (1983) acknowledge the importance of the external validity problem?
Applied experiments.
57
Brian Nosek argues that procedural changes are necessary in replication studies so the definition of replication should focus on conceptual terms. How is this justified?
By conceptualising replication in the realm of theoretical expectations, changing what procedural differences are necessary with no prior expectation that such changes will produce a different result. Relies on researchers theoretical understanding of the concept being tested.
58
What are three ways to minimise false negatives in studies?
Obtain original materials. Use high-powered designs. Peer review in advance.
59
What is external invalidity, as defined by D.G. Mook in his article "In Defense of External Invalidity"?
External invalidity refers to the extent to which research findings can be generalized to real-world settings and situations outside of the laboratory.
60
Why does Mook suggest that researchers should prioritise external validity?
To make research more relevant and applicable to real-world situations
61
True or False: The goal of replication is to copy the methodology and procedure of a previous study and observe if we get the same results.
False. The goal is actually to attempt to reproduce the finding of a previous study with no a priori reason to expect a different result. I.e., with this new sample/context/setting, these are the adjustments I need to make to the procedure, in order to have no reason to think that it will produce a different result.
62
Why is replication important in science?
Replication is important in science because it helps to confirm the results of studies and to build confidence in scientific findings.
63
What is external validity?
External validity is the extent to which our findings can be generalised in other populations, settings, variables, etc.
64
According the Brian Nosek, every replication, because they are not identical, is a test of ___?
Generalisability
65
Why does the experiment by Harlow raised a threat of external validity?
One of the reasons is because the sample used was not deemed as representative of the population. In this case, Harlow’s baby monkeys were not representative of the population of monkeys in general as they were born in captivity and orphaned.
66
Why is there a focus on Procedural Similarity?
There is a lack of Theoretical Specificity There is a lack of understanding the Methodology. Having the same procedure is a substitute for Theoretical & Methodological Understanding
67
How does Brian Nosek view Replication?
Brain Nosek views Replication as a conceptual understanding. Replication provides us with an opportunity to confront current theoretical understandings of how we obtain research findings.
68
Why is there a current focus on procedural similarity in psychological research?
A lack of theoretical specificity and of understanding of methods. Using procedural replication (rather than conceptual) is a substitute for theoretical and methodological understanding.
69
Multiple Choice: According to the study by Brown and Hanlon (1970), what do parents react to in a child's speech? a) Its form b) Its content c) Its grammar d) Its tone
Answer: b) Its content Explanation: Direct observation of parent-child interactions reveals that parents react to the content of a child's speech, not to its form.
70
According to Mook (1983), what is the importance of external validity in research?
External validity ensures that research findings have practical implications and can be applied to real-world situations.
71
When considering internal and external validity, what does Mook suggest should be the sacrificed one if required?
Sacrifice internal validity
72
What are the three ways Brian Nosek described to interpret a failure to replicate?
The three ways are (1) the original finding was a false positive, (2) the replication is a false negative, or (3) neither, that there are important substantive differences that explain the different results
73
What is usually lacking when someone believes replication can only occur if it has the exact procedure of what it is replicating?
This often demonstrated a lack of theoretical understanding, and shortcomings in understanding the methods of the original study.
74
Criticisms of external validity may include issues such as sample characteristics, ecological validity, and ________ of results.
generalization
75
true/false Theoretical specificity doesn’t enable effective replication
false
76
What are the two things that Mook suggests can lead to the compromise of a study when focusing on external validity?
Rigor and control
77
What is the "agricultural" concept of the aims of research?
The assumption that the experiment is intended to be generalised to similar subjects, manipulations, and settings.
78
Fill in the Blank: According to the article, the fact that eyeglasses can influence our judgments of intelligence says something about us as __________.
Answer: judges Explanation: The fact that eyeglasses can influence our judgments of intelligence, though it may be quite devoid of real-world application, surely says something about us as judges.
79
True or False: Harlow concluded that wild monkeys in the jungle would choose terry-cloth over wire mothers.
False: Harlow did not generalise his findings of an artificial experiment involving monkeys born in captivity to monkeys in the wild. Harlow actually concluded that the hunger-reduction interpretation of mother love would not work
80
What is the difference between statistical generalization and analytical generalization, and how do they relate to external validity?
Statistical generalization refers to the use of probability and sampling techniques to make inferences about a population, while analytical generalization involves the transfer of theoretical concepts and propositions to different contexts, and both are important for achieving external validity.
81
True or False: According to Mook (1983), experiments that lack external validity are worthless.
FALSE
82
How is generalisability explained in Mook's (1983) research article?
To what populations, settings, treatment variables, and measurement variables can this effect be generalized.
83
Why is it that claiming that a study is a replication is a theoretical claim?
No two studies can be identical; Even if the same sample, environment and context were the same, the conditions have changed.
84
What is the main argument that Mook (1983) makes in his article "In Defense of External Invalidity"?
Mook (1983) argues that external invalidity is not necessarily a fundamental flaw in psychological research, as it is often criticized. He contends that the focus should be on the internal validity and theoretical implications of a study, rather than its direct generalizability to other populations or situations.
85
TRUE/FALSE: An individual study with enough power, theoretical strength, and significant findings can act as conclusive evidence for or against a theoretical concept.
FALSE. A key part of Nosek’s definition of replication is that no individual study is ever conclusive evidence for or against our theoretical understanding. Evidence is accumulated slowly across many studies in order to get an understanding of a phenomenon.
86
What was the goal of the Many Labs 2 replication work described by Nosek?
To evaluate the current assumption in much of research that when we don't get the same finding during replications it is due to changes in the sample, setting, or context, and whether we do or should use this as a default explanation.
87
What are the two types of validity that Mook discusses in his article?
Mook discusses internal and external validity. Internal validity refers to the extent to which an experiment's results are due to the manipulation of the independent variable, whereas external validity refers to the generalizability of the experimental results to other situations or populations.
88
In "In Defense of External Invalidity" by Douglas G. Mook, what were some benefits for the use of artificial and controlled laboratory conditions in research.
Some benefits include: - Showed what can occur, rather than what typically does happen - could illustrate the power of a phenomenon by demonstrating that it even happens under unnatural conditions that prevent it from happening - control and produce conditions that does not happen in real life
89
True/False Milgram's work on obedience was conducted in a laboratory setting.
TRUE
90
True or False: According to Argyle (1969), wearing glasses has a significant effect on judgments of intelligence, regardless of the duration of exposure to the stimulus person.
False. According to Argyle, wearing glasses has a significant effect on judgments of intelligence when the stimulus person is seen for 15 seconds, but has no effect when seen for 5 minutes of conversation.
91
True or false. Based on the video, Brian Nosek claims that there is no such thing as an exact replication because every time we do a replication its in a new context.
True.
92
According to Brian Nosek, the process of __________ is a critical part of scientific research that involves reproducing findings from previous studies.
Replication
93
What is Brian Nosek's definition of replication?
Attempting to reproduce a previous observed finding with no a priori reason to expect a different outcome.
94
What are three productive ways that one can reduce the chances of a false negative according to Brian Nosek?
1. Obtain original materials 2. High powered designs 3. Peer review to occur in advance
95
The validity of these generalizations is tested by their success at prediction which has nothing to do with 1,________ 2,________3,_________
1, naturalness, 2, representativeness, 3, nonreactivity
96
The external validity question asks, in short: are the sample, the setting, and the manipulation of a study so artificial that the class of "target" real-life situations to which the results can be generalized is likely to be trivially small? When the answer is yes, what is Mook's general reasoning behind why, sometimes, this same research can still be considered well-done, meaningful, or informative?
He argues that generalisation to real life is sometimes not intended or meaningful. In these cases, a study's external validity shouldn't lead to the research being dismissed altogether.
97
When the setting and manipulation of variables are too artificial in an experiment, what does this impact?
External Validity
98
Why are experimental settings made artificial in the words of Mook?
To control for extraneous variables and to permit separation of factors that do not come separately in Nature-as-you-find-it.
99
What is Mook's Checklist of decisions?
What conclusion do we want to draw? Does our sample's specifics prevent us from drawing it?
100
What is external validity?
The extent to which research findings can be generalized to real-world settings and populations.
101
Define external invalidity
The extent of how well the conclusion of the study can be generalised to other contexts
102
What are the levels of analysis in psychology?
The levels of analysis refer to different perspectives from which psychologists can examine behaviour and mental processes, including biological, cognitive, social, and cultural.
103
Because every replication is never identical, it is a test of generalisability. True or false?
True.
104
True or False: Brian Nosek used the example of Schnabel and Nadler's (2008) study and how it was revised for an American sample to argue for the original definition of holding the procedure constant when it comes to replication.
FALSE: In fact, this article was used by Brian Nosek to argue for a new definition regarding replication as he noted that holding the procedure constant would mean keeping the study replicated in the same language and context as the original study for an Israelian sample, which wouldn't be applicable to an American sample; he argues that replication is in fact concerned with conceptual matters over procedural matters.
105
Which of the following is NOT a way to minimise false negatives: (A) Obtain original materials (B) Change the original procedure to fit the new context of the replication (C) Peer review in advance (D) Use high-powered designed
(B) is not a way to minimise false negatives. Although procedural changes may be necessary for a replication, it does not minimise false negatives.
106
True or False: Mook (1983) argues that external validity should always be prioritized over internal validity in psychological research.
Answer: False Explanation: Mook contends that the importance of external validity has been overemphasized in psychological research. He argues that internal validity should often be prioritized, as establishing causal relationships is crucial for scientific progress. Mook also suggests that the relevance of external validity depends on the type of research being conducted, with applied research benefiting more from generalizability than basic research.
107
What are the three major conclusions that may be drawn by a failure to replicate according to Brian Nosek?
1. The original is a false positive 2. The replication is a false negative 3. Neither of the findings are true (violation of expectations)
108
How did Mook use what was happening 'in the sample' to make predictions about a theoretical proposition?
Although Mook understood that sample bias gave rise to issues of generalisability. By considering the theoretical proposition 'parents respond to the grammar of their children's utterances (either through approval or correcting mistakes). They can predict that it ought to be observed that parents do this. However, as evidence showed parents reacted to content rather than grammar thus disconfirming the prediction. It was then taken further to show that these children went on to acquire the approved syntax even if these parents did not react to grammaticality, which disconfirmed the theory further illustrating that this theory was analysed by looking at what happened in the sample rather than trying to generalise from the sample to the population.
109
What are some of the causes of narcissism discussed in the video?
Parental overvaluation, neglect or abuse, cultural values, and individual factors such as temperament and genetics.
110
What are 2 reasons that many psychology experiments are accused of "failure to generalize to the real world"?
Because they have sample bias or artificiality of setting
111
What is the reason that Brian Nosek justifies why we should think of replication as conceptual replication, not procedural replication?
Procedural replication, properly done, would just be the same study. We are not able to replicate the same procedure, with the same participants, at the same time, in the same context as the original study. Therefore, we have to consider replication as conceptual, because the theoretical concept is what we are attempting to replicate, not the procedure.
112
True or False? Representativeness of sample is important for certain purposes, but for other purposes it is a trivial issue. Therefore, one must evaluate the sampling procedure in light of the purpose—separately, case by case.
TRUE
113
True or false: Mook (1983) mentions that a possible cause for the confusion about EV regarding sampling bias stems from survey researchers.
True. For most survey researchers, one main goal is to generalize the sample findings to the population.
114
What is the main argument presented by Mook in his article "In Defense of External Invalidity"?
Mook argues that external invalidity, or the limited generalizability of experimental results to other situations or populations, is not necessarily a problem. He believes that researchers should focus on understanding the underlying processes and mechanisms in a particular context, rather than trying to generalize their findings too broadly.
115
Define External Invalidity?
The extent to which the findings of a study can be generalized to other populations, settings, or contexts beyond the specific conditions of the study.
116
True or False: External validity refers to the extent to which research findings can be generalised to real-world settings and populations outside the research context.
True.
117
According to Brian Nosek, what is replication?
The process of repeating an experiment to confirm its findings.
118
What are some of the reasons why external validity is often neglected in research, according to Mook's article "In defense of external invalidity"? A) It is difficult to achieve both internal and external validity in the same study. B) Researchers often prioritize theoretical significance over practical applicability. C) The use of convenience samples limits the generalizability of findings. D) All of the above.
Answer: D) All of the above.
119
According to Mook (1983), which of the following is NOT a common way in which researchers sacrifice external validity for the sake of internal validity in experimental research? A) Using highly selected samples that are not representative of the population of interest. B) Using contrived settings and procedures that do not resemble real-world situations. C) Using multiple measures to increase the reliability of the results. D) Using simple and straightforward experimental designs that do not capture the complexity of real-world phenomena.
C) Using multiple measures to increase the reliability of the results. According to Mook (1983), using multiple measures to increase the reliability of the results does not necessarily sacrifice external validity for internal validity. However, using highly selected samples, contrived settings and procedures, and simple experimental designs can limit the generalizability of the findings to real-world settings and populations.
120
what are four reasons someone may do an experiment, if not to predict real-life behaviour?
1. They May be asking whether something can happen, rather than whether it typically does happen. 2. their prediction may specify something that ought to happen in the lab, so they go to a lab to see whether it does. 3. They may want to demonstrate the power of a phenomenon by showing that it happens even under unnatural conditions that ought to preclude it. 4. To produce conditions that have no counterpart in real life at all, so that the concept of "generalising" to the real world" has no meaning
121
According to Mook, why is external validity not a necessary condition for experimental research?
Mook argues that internal validity is more important than external validity because it is necessary to establish a cause-and-effect relationship between the independent and dependent variables.
122
What is the definition of external validity?
External validity refers to the extent to which research findings can be generalized to other populations, settings, and conditions beyond the specific study.
123
What are some potential benefits of developing theories that are broadly applicable and what are some potential challenges in developing such theories?
Broadly applicable theories allow researchers to understand how psychological principles work across different populations, settings, and contexts, which can lead to more effective interventions and policies. However, developing broadly applicable theories can be challenging, as it requires taking into account the complexity and diversity of human behaviour and psychology. It also requires a willingness to challenge assumptions and preconceived notions about human behaviour, and a willingness to incorporate diverse perspectives and experiences into psychological research.
124
What is an interaction effect?
An interaction effect occurs when the effect of one variable on the outcome of an experiment depends on the level of another variable.
125
Why are many psychological investigations accused of “failure to generalise to the real world”?
Because of sample bias or artificiality of setting.
126
According to Brian Nosek's talk, what is the defiinition of replication?
Repeating a study's procedure and observing whether the prior finding recurs.
127
According to Mook (1983), what can the pursuit of internal validity lead to?
The pursuit of internal validity can lead researchers to conduct studies that are artificial and disconnected from the real world.
128
In the video "What is Replicability?" by Brian Nosek, what is the revised definition of replication that he provides?
Replication is attempting to reproduce a previously observed finding with no a priori reason to expect a different outcome.
129
According to Mook what are the implications of a stripped-down setting?
Although tasks, variables and settings have no real-world counterparts it allows us to understand real-world phenomena for example how the visual system works. Through the processes dissected in the laboratory, there are found to also operate in the real world, such that the results apply to any setting in which it is relevant. Therefore, showing that the findings do not represent a class of real-world phenomena they define one.
130
What is external validity in psychology research?
External validity refers to the extent to which research findings can be generalized beyond the specific context in which the research was conducted.
131
Why is it impossible to perfectly replicate a study?
You can never exactly replicate the people were who did the original study.
132
TRUE or FALSE: To ensure robustness and relevancy of a persons research, they should strongly focus on external validity.
False, researchers should aim for a balance between internal and external validity.
133
How can researchers balance internal validity and external validity in their study designs?
Researchers can balance the trade-off between internal validity and external validity by using a multi-method approach that combines different types of evidence and research designs, conducting pilot studies to test and refine the methods, reporting effect sizes and confidence intervals, and considering the practical significance of the findings in addition to the statistical significance.
134
Which of the below options is NOT an implication of conceptual replication according to Brian Nosek?
A) Theoretical specificity enables effective replication B) Replication promotes specificity C) Understanding whether a study is a replication can change over time D) Procedural replication Furthers the progress of scientific research.
135
What are some threats to external validity, and how can they be minimized?
Some threats to external validity include selection bias, reactivity or measurement effects, experimenter bias, multiple-treatment interference, and situational specificity. These threats can be minimized by using random sampling, randomized treatment allocation, standardized measures, blinding procedures, replication across different settings, and ecological validity.
136
minimise false negatives by
obtaining original materials, create high power designs, peer review in advance
137
What is applied experiments
Research that is applied to a theory that relates to real life issue with a practical outcome
138
Define generalisability
Generalisability is the extent in which the findings of a study can be applied in different contexts, populations and conditions.
139
Why is it that some experiments that is "clearly applicable to the real world" have limits to its generalisability?
Unknown factors would be present and no single study is able to discover them all.
140
What problem does Mook (1983) think that questioning the external validity of studies can help explore?
We focus too much on whether results found in studies are representative of the subjects/participants that we test, and not enough on whether the environments and how we brought about these findings represent what might be found in the real world
141
True or false: According to Mook (1983), external invalidity is a serious problem in experimental research.
FALSE
142
How does Brian Nosek define replication?
Attempting to reproduce a previously observed finding with no a priori reason to expect a different outcome.
143
What does Brian Nosek believe should be the focus of replication in the context of psychological research?
Replication should be focused on the conceptual (not procedural) recreation of a study to test a "known" physiological phenomenon for its validity.
144
Among all of these Douglas G . Mook’s view about external validit,which one is correct? A,EV is only relevant in certain types of research, specifically applied experiments aimed at evaluating the effectiveness of a particular intervention (e.g., a new teaching method or a fertilizer on crop growth). B,Got a tendency to focus too narrowly on subject representativeness and not enough on representativeness of settings and manipulations in experimental research. C, Not all experiments are intended to be generalized in this way. Some experiments are designed to test specific hypotheses rather than to make broad generalizations. In these cases, EV may not be as relevant a concern. D,ALL of these above
D,ALL of these above
145
Why does Brian Nosek argue that we need replication?
Replication can provide an opportunity to challenge the current theoretical understanding of the way a result is obtained.
146
Fill in the blank: "The generalization of experimental results to the real world depends on __________."
External validity.
147
What is external validity?
It refers to the extent in which research findings can be generalised/applied to real-world contexts or populations beyond that specific context of the study.
148
Why do an experiment if we are not using our experiments to predict real-life behaviour?
1. Asking the question of CAN something happen rather than does it typically happen 2. Prediction may specify something that ought to happen in the lab, so this is then investigated in the lab 3. Show the power of a phenomenon if it were to happen under unnatural conditions that ought to prevent it 4. To simply contribute to an understanding of the processes going on.
149
According to Mook, External Validity is a concept that should be banned from the scientific jargon since it is impossible to reach. True or False?
False. EV is useful for some applied contexts (agricultural conception)
150
What is external validity?
The extent to which research findings can be generalised to real-world settings
151
What is the main argument presented in "In defense of external invalidity" by Mook (1983)?
Mook argues that external validity is an essential component of scientific research and that it should not be neglected in favor of internal validity. He proposes that external validity should be viewed as a separate research goal and that studies should be designed to maximize both internal and external validity.
152
true or false, The characteristics of the sample or population studied have no impact on external validity.
False. The characteristics of the sample or population studied can have an impact on external validity.
153
What is external validity and why is it important in research?
External validity refers to the extent to which research findings can be generalized to other settings, populations, and situations beyond the one in which the research was conducted. It is important in research because it helps to ensure that the findings have real-world relevance and can be applied to practical situations.
154
What is external invalidity
Validity of applying conclusions of a study to a population (ability to generalise)
155
What is the importance of replication in experimental research?
Replication is important because it helps to establish the generalizability of experimental results and to verify the validity of the findings.
156
true or false, External validity refers to the extent to which the findings of a study can be generalized to other settings or populations beyond the study.
TRUE
157
True or false: Mook (1983) contends that generalizability is not a concern in experimental research.
FALSE
158
True or False? Social processes observed to occur within a laboratory setting accurately reflect natural social settings
FALSE
159
True or False: Mook argues that context plays a negligible role in understanding human behaviour and its relationship to external validity.
False. Mook emphasises the importance of context in understanding human behaviour and the impact it has on the external validity of research findings.
160
What are the three components of experimental investigation as mentioned in the original discussion of external validity by Campbell and Stanley (1971)?
Pretests, treatment imposed/withheld, and posttest.
161
True or False: according to a study by Mook (1983), an experiment conducted by Harlow raised a threat of external validity, in which the sample was not representative of the population of monkeys in general as Harlow’s baby monkeys were born in captivity and orphaned.
TRUE
162
What is the recommended approach for researchers when it comes to prioritizing internal and external validity in their studies?
Researchers should prioritize both internal and external validity in their studies, as they are both important for advancing scientific knowledge and developing practical applications.
163
What is external validity? Is it better to have higher or lower validity and why?
External validity refers to the extent to which the findings of a study can be generalized to other populations, settings, and contexts. It is better to have high external validity to ensure that the findings of a study have real-world implications.
164
T or F: Mook (1983) argues that external validity is often prioritised over internal validity in experimental research.
False. In Mook's (1983) article he argues that internal validity is often prioritised over external validity in experimental research.
165
True or false: Mook (1983) suggests that researchers should focus on maximizing internal validity at the expense of external validity.
FALSE
166
What are some ways researchers can enhance the external validity of their research?
Researchers can enhance the external validity of their research by using diverse samples that are representative of the population of interest, using real-world stimuli and settings, and using designs that allow for the manipulation of variables in ways that mimic real-world situations. For example, researchers can use field studies, naturalistic observations, and quasi-experimental designs to increase the external validity of their research.
167
Why are many psychological investigations accused of "failure to generalise to the real world"?
The failure to generalise to the real world stems from sample bias or artificiality of the setting.
168
By the study conducted by Higgins and Marlatt (1973) on the tension-reduction view of alcohol consumption. Dring alcohol can absolutely decrease people's tension after they have been eclectric shocked,is this statement true or wrong?
Wrong
169
True or false? A misplaced preoccupation with external validity can lead us to dismiss good research for which generalization to real life is not intended or meaningful.
TRUE
170
TRUE or FALSE: Harlow's monkey experiment applies to some criteria of external validity.
FALSE: Harlow's experiment is deficient or unevaluable on every criterion of external validity
171
According to Mook 1983, for what reason do researchers make experimental settings artificial?
We do it to control for extraneous variables and to permit separation of factors that do not come separately in nature-as-you-find-it.
172
According to Mook (1983), what are 3 limits that can still be found in a study's generalizability even if it had been conducted and proved applicable in the real world?
Cultural, historical, and age-group limits.
173
Name at least one way (out of three) to reduce the likelihood of false negatives.
You could have said: * Obtaining original materials. * Developing high-powered designs. * Peer reviewing in advance.
174
How does Mook (1983) address the criticism of laboratory experiments' lack of external validity?
Mook (1983) acknowledges the limitations of laboratory experiments in terms of external validity but argues that it is more important to understand the underlying psychological principles and mechanisms in a controlled environment. He suggests that the primary goal of psychological research should be to develop and test theories, and that generalizability can be addressed through follow-up research and the accumulation of evidence across multiple studies.
175
True or False: Mook believes that in psychological research it is always reasonable to sacrifice external validity for internal validity.
False. Mook questions the idea that it is always reasonable to sacrifice external validity for internal validity and argues for a trade-off between the two.
176
What is external validity?
External validity refers to the extent to which the results of a study can be generalized to other settings, populations, and conditions beyond the specific context of the study.
177
true or false: If one finds a significant improvement in the ex-perimental subjects as compared with the controls, one predicts that implementation of a similar ma-nipulation, in a similar setting with similar subjects, will be o benefit on a larger scale.
TRUE
178
Brian Nosek and his team have been able to effectively reproduce the same findings for approximately ___% of studies they’ve replicated.
47%
179
What are some limitations of relying on laboratory studies to draw conclusions about behaviour?
Relying solely on laboratory studies to draw conclusions about behaviour can limit the generalizability of research findings and reduce their practical relevance. Laboratory studies may not accurately reflect how people behave in the real world, and may miss important factors such as cultural, social, and environmental factors.
180
Can internal validity be established independently of external validity in research, and what is the relationship between the two?
Internal validity cannot be established independently of external validity, as the two are interconnected and equally important for advancing scientific knowledge and developing practical applications.
181
True or False - Brian Nosek argues that one of the key benefits of replication is that it helps identify whether a scientific finding is generalizable across different populations and contexts.
TRUE
182
true or false, Internal validity refers to the extent to which a study accurately measures what it intends to measure within the study itself.
TRUE
183
Fill in the blank. “The greatest weakness of laboratory experiments lies in their ………… Social processes observed to occur within a laboratory setting might not necessarily occur within more natural settings” - Babbie, 1975.
Artificiality.
184
According to the article by Mook, one major problem with the accusation of external invalidity or failure to generalize a research finding to the real-world situation is…
generalizations are often not intended by researchers
185
Mook argues that there is an assumption that the purpose of collecting data in the lab is to 'predict real-life behaviour in the real world'. List other possible reasons.
Examples: - To predict whether something can happen - To specify something that would happen in the lab - To demonstrate the power of a phenomenon - To produce conditions that cannot happen in real life
186
Describe four uses of an experiment other than to predict real-life behaviour according to Mook.
looking at whether something can happen rather than whether it typically does happen, confirming the direction of a prediction, demonstrating the power of phenomenon even under unnatural conditions, and producing conditions that have no counterparts in real life.
187
fill in the blank: Surely one of the experiments that has had permanent impact on our thinking is the study of_____ in rhesus monkeys, elegantly con-ducted b y Harlow
"mother love"
188
True or false: The article by Mook (1983) argues that external validity and generalizability refer to the same concept.
False. Studies can have high external validity, but still limited generalizability.
189
True or False? An assumption put forward in the Mook paper was that the purpose of data collection in the laboratory was to predict real-life behaviour in the real world?
True. However, there are situations where lab findings are not meant to replicate the real world so generalisation would provide no meaning for determining validity. It depends on what kind of conclusions one wants to draw from the study.
190
True or false? A highly artificial laboratory experiment cannot tell us anything we can generalise to the real world
False – As demonstrated in dark adaptation studies, highly artificial laboratory experiments can still tell us about processes that take place in the real world (in that case, that dark adaptation takes place in two phases as function of the cones and then the rods in our eyes)
191
According to Mook, how can researchers improve external validity in their studies?
Mook suggests several strategies for improving external validity, including conducting studies in naturalistic settings, using more diverse samples that reflect the population of interest, and taking steps to ensure that the variables and conditions in the study are similar to those found in the real world.
192
What is External Validity?
It asks the question of how generalisable the effect of populations, settings, treatments, & measurement variables are.
193
True or false: the article by Mook (1983) argued that “generalisations” are often not the intention when somebody is conducting a research
TRUE
194
Fill in the blank: Open science is an approach to conducting research that emphasises __________, collaboration, and sharing of data and findings.
Transparency
195
True or False: Internal validity refers to the extent to which a study can establish causal relationships between variables.
TRUE
196
What is an example where the effects of an independent variable studied in isolation may be exaggerated?
The example provided was wearing glasses increasing IQ. Another may be having tattoos influencing competence in academic work. When it is the only information, it may have a large negative influencing effect on judgement, but when other variables are considered, it is likely to have a much smaller effect on judgements of competence.
197
Replication
Repeating a study’s procedure and observing whether the prior finding recurs
198
According to Mook 1983, what are the two differences between lab and life?
the absence of punishment and the possibility of discounting the danger of obedience
199
True of False: When we make generalizations of our findings to the real world, they may have added force because of artificiality of sample or setting.
TRUE
200
Why should we consider external validity with less fervour?
Generalising findings from an experimental study may not always be the primary intent of the study. As such, making sure that all studies have adequate external validity might be going overboard when evaluating whether a study and its findings are sound or not.
201
According to Mook, what is the "pragmatic fallacy"?
The "pragmatic fallacy" is the erroneous belief that because a study's findings are useful or valuable in some way, the study must also have high external validity.
202
What is the point that the paper by Mook, is making?
The author argues that even in artificial laboratory settings, experiments can provide valuable insights into real-world phenomena and that the focus should be on understanding the processes being studied rather than trying to replicate real-life situations.
203
True or False: Mook (1983) believes that external validity is more important for applied research than for basic research.
Answer: True Explanation: Mook distinguishes between applied research, which aims to solve practical problems, and basic research, which seeks to advance theoretical understanding. He argues that external validity is more relevant to applied research, as it focuses on the generalizability of findings to real-world situations. In contrast, basic research prioritizes internal validity, as it is concerned with establishing causal relationships and developing new theories.
204
What is external validity, and why is it important to psychological research?
External validity refers to the degree to which research findings can be generalized beyond the specific conditions, participants, and measures used in the study. It is important because it determines the extent to which the results can be applied to real-world settings and populations.
205
According to Mook, what is an "Agricultural"?
The assumption that an experiment's results are intended to be generalized.
206
True or False: A misplaced preoccupation with external validity can lead us to dismiss good research for which generalization to real life is not intended or meaningful.
True
207
Unrepresentative samples are of no value at all T/F?
False. Unrepresentative samples can provide valuable insight, e.g., exceptional/different cases can demonstrate what CAN occur
