Seminar one Flashcards
Is it enough that a study replicable?
No. The results need to have some real-world importance. Although this is subjective.
What is importance hacking?
Even if a statistically significant result is obtained in a sound, valid, reliable way, it may be that the result is actually not that useful/important. Researchers can “importance hack” by making out that this effect is more important/useful than it is to make it seem that their work is, well, important.
What are the four validities we discussed?
Construct validity.
External validity.
Internal validity.
Statistical conclusion validity.
What is construct validity?
Questions to ask to ensure construct validity:
Are theoretical constructs well defined?
Are measures and manipulations valid?
To go about ensuring construct validity, we can ask the following question:
What is the construct and how do we measure it?
When there is not construct validity, then we may have sound research practices, but generate results that do not reflect what we are actually interested in.
See the example of the “Crowdsourcing Hypothesis Tests” article discussed in the seminar.
This article showed, that depending on how constructs were defined and subsequently measured had a big influence on whether the results obtained and conclusions drawn, even though the research question was the same.
Do you have to be entirely sure of your research’s construct validity before doing the research?
No.
Our understanding of constructs and how we operationalise them is constantly changing, especially in psychology, where we can rarely directly measure a state of mind, for example.
What is external validity?
External validity refers to whether the results obtained from an experiment/s would be replicated in the population of interest. In other words, are we seeing results reflective of our sample, or reflective of the population the sample is meant to represent?
A question to consider when trying to determine external validity:
Will conclusions generalize across the population of ingerest?
Are the majority of psychology studies done in WEIRD populations?
Yes.
What is the Mueller-Lyer illusion, and how does this effect vary across cultures?
How is this an example of how psychology research that looks mainly at WEIRD populations does not always have very good external validity?
The Mueller-Lyer illusion is where a line looks longer if the ends of the line have two lines pointing on a 45 degree angle away from the line. In contrast, the line looks smaller if these lines at the end of the line are pointing toward the line on a 45 degree angle.
Or so it appears…
The research done on this effect was done in WEIRD populations. When done in other cultures, it appears that the effect is far less pronounced.
Psychology is notorious for using WEIRD populations in their experiments. Why? Psychology, as it is understood in our society and culture, is well….a WEIRD interpretation and understanding of the mind and what it is to be human.
This means that results from psychology research done in WEIRD populations may actually not be externally valid when we consider other cultures.
What is Spencer Greenberg’s definition of Importance Hacking?
Presenting a result in a way that makes it seem more interesting, important, and valuable than it really is.
What is Greenberg’s definition of Hacking Novelty?
What is Hacking Usefulness?
What is Hacking beauty?
Novelty hacking: referring to something in a way that makes it seem more novel than it actually is.
Usefulness hacking: making a result seem relevant to some important outcome when it’s actually useless.
Beauty hacking: making a result seem clean and beautiful when it’s actually messy or hard to interet.
