Session 1 & 2 - Why we need research Flashcards
Between-groups design
“[W]e can manipulate variables in experiments in two ways. The first is to manipulate the independent variable using different entities. This is the method we’ve been discussing − we allocate different bands, or entities, to two different groups − and it’s known as a between-groups, between-subjects, or independent design.’”
Synonym of: between-subjects design; independent design
Essentially, instead of testing the same group of people twice under different circumstances (e.g. the same band giving away bracelets and then not), you test different groups of people (e.g. two different bands, one giving away bracelets and one not).
Between-subjects design
“[W]e can manipulate variables in experiments in two ways. The first is to manipulate the independent variable using different entities. This is the method we’ve been discussing − we allocate different bands, or entities, to two different groups − and it’s known as a between-groups, between-subjects, or independent design.’”
Synonym of: between-groups design; independent design
Essentially, instead of testing the same group of people twice under different circumstances (e.g. the same band giving away bracelets and then not), you test different groups of people (e.g. two different bands, one giving away bracelets and one not).
Boredom effect
“A second source of systematic variation is boredom effects, that is, when participants take part in several experimental conditions they are likely to become
fatigued. Imagine we asked people to take statistics tests while pretending to be themselves, a student good at statistics, a statistics professor, someone who had never done statistics, and as a watermelon (as a control). They would have to complete five statistics tests. By the fifth test they’d be quite bored …’”
Confounding variables
“[T]here is still the problem that other variables that you haven’t measured, called confounding variables, might be influencing both variables. For example, perhaps personality affects both how physically attractive a person is perceived to be and also their popularity. People with a nice personality have more friends (because they are nice) but they are also perceived to be more attractive.”
What this means is that if you have the hypothesis that being attractive makes you more popular, and you find that attractive people do tend to be more popular, you can incorrectly assume that there is a causation between these two concepts (variables in your research). Whereas in reality, it is possible that if you are kind people will perceive you to be more attractive, and will want to be your friend (thus popularity). Thus attractiveness and popularity co-occur, but there is no causation between the two.
Correlational research
“Alice nodded. ‘When you observe what naturally happens in the world without directly interfering with it, it is known as correlational research. There are different ways to do this: we could take a snapshot of many variables at a single point in time (a cross-sectional study), or measure variables repeatedly at different time points (a longitudinal study).”
Compare with: Experimental research
Counterbalancing
“‘Yeah, if you were using a repeated-measures design, you might expect fewer T-shirts to be sold at the second gig because if people had bought a shirt at the first gig and then also come to the next one, they won’t buy another shirt.’
‘That’s true, and in reality you would counterbalance the order in which the samples complete each condition. For our example that means that half of the bands would give away a free gift at the first concert and not at the second, but the others would give away the free gift at the second concert but not the first. Counterbalancing is a technique used to eliminate sources of systematic variation.”
With counterbalancing, once you identify some aspect that would sway your results, you can adjust your research design in a way where that aspect is accounted for, such as in the example above.
Cross-sectional study
“Alice nodded. ‘When you observe what naturally happens in the world without directly interfering with it, it is known as correlational research. There are different ways to do this: we could take a snapshot of many variables at a single point in time (a cross-sectional study), or measure variables repeatedly at different time points (a longitudinal study).”
For example, in a cross-sectional study you would take a group of people, measure how often they use their phones, and then examine them for brain cancer. This can be done in a relatively short amount of time, and measurements are not repeated.
Dependent variable
“Imagine we randomly select some bands, and half of them we asked to give away free gifts with every T-shirt, and the other half gave away nothing. The thing that we have manipulated is the incentive to buy a shirt (the free gift or no gift). This is known as an independent variable because it is not affected by the other variables in the experiment. More generally, it is known as a predictor variable, because it can be used to predict scores of another variable (i.e., we predict T-shirt sales based on whether or not a gift was offered). In this situation it is said to have two levels, because it has been manipulated in two ways (i.e., free gift or no free gift). The outcome in which we are interested is T-shirt sales. This variable is called the dependent variable because we assume that its value will depend upon whether or not a free gift was offered (the independent variable).”
Synonym: outcome variable
If it helps, think of it this way: We assume T-shirt sales will grow if there is a free gift. We measure the number of t-shirts sold, and we “measure” whether or not gifts were offered. The first fact, the number of T-shirts sold will [hypothetically] DEPEND on whether or not there were free gifts. The free gifts are a given, and they are there or not based on how the experiment is set up, INDEPENDENTLY of the other measured variables.
Descriptive statistics
“It boils down to two things that you might want to do with data. The first is to describe what happened in the sample that you collected. You might draw a graph of the data, or calculate some summary information such as the average T-shirt sales. This is known as descriptive statistics. However, because
scientists usually want to generalize their findings beyond the data they collected to the entire population, they use the sample data to estimate what the likely values are in the population. This is known as inferential statistics. Inferential statistics help us to make generalizations about what is going on in the real world, based on a sample of data that we have collected.’”
Very simply, with descriptive statistics you DESCRIBE your data, and with inferential statistics you INFER something out of your data.
Ecological validity
Correlational research gives us a very natural view of the question we’re researching because we’re not influencing what happens and the measures of the variables should not be biased by the researcher being there. This makes it more likely that the study will have ecological validity, which means that the results of the study can be applied to real-life situations. There is a price to pay, which is that correlational research tells us nothing about whether one variable causes another.
Ecological validity means that the way your research is set closely resembles real-life scenarios. Be careful here: just because your study is correlational (and not experimental) does not mean that it automatically has ecological validity.
Experiment (research)
“Comparing two conditions in a controlled way is at the heart of experimental methods: they provide a comparison of situations (usually called treatments or conditions) in which the proposed cause is present or absent, while controlling for all other variables that might influence the effect in which we’re interested. This scenario is an experiment. The T-shirt sales example is a good one. Imagine we randomly select some bands, and half of them we asked to give away free gifts with every T-shirt, and the other half gave away nothing. The thing that we have manipulated is the incentive to buy a shirt (the free
gift or no gift)”
Compare with: correlational research methods
Experimental methods
“Comparing two conditions in a controlled way is at the heart of experimental methods: they provide a comparison of situations (usually called treatments or conditions) in which the proposed cause is present or absent, while controlling for all other variables that might influence the effect in which we’re interested. This scenario is an experiment. The T-shirt sales example is a good one. Imagine we randomly select some bands, and half of them we asked to give away free gifts with every T-shirt, and the other half gave away nothing. The thing that we have manipulated is the incentive to buy a shirt (the free
gift or no gift)”
Compare with: correlational research methods
Hypothesis (hypotheses)
“‘You can use existing theory, to generate a hypothesis, which is a proposed explanation of the specific observation that interests you. Based on consumer theory you might hypothesize “T-shirt sales increase because a free gift improves the value for money”.”
Hypotheses is the plural form of hypothesis
According to the APA: a hypothesis an empirically testable proposition about some fact, behavior, relationship, or the like, usually based on theory, that states an expected outcome resulting from specific conditions or assumptions.
The Oxford Learner’s Dictionary provides a helpful definition: a hypothesis is an idea or explanation of something that is based on a few known facts but that has not yet been proved to be true or correct
Independent design
“[W]e can manipulate variables in experiments in two ways. The first is to manipulate the independent variable using different entities. This is the method we’ve been discussing − we allocate different bands, or entities, to two different groups − and it’s known as a between-groups, between-subjects, or independent design.’”
Synonym of: between-subjects design; between-groups design
Essentially, instead of testing the same group of people twice under different circumstances (e.g. the same band giving away bracelets and then not), you test different groups of people (e.g. two different bands, one giving away bracelets and one not).
Independent variable
“Imagine we randomly select some bands, and half of them we asked to give away free gifts with every T-shirt, and the other half gave away nothing. The thing that we have manipulated is the incentive to buy a shirt (the free gift or no gift). This is known as an independent variable because it is not affected by the other variables in the experiment. More generally, it is known as a predictor variable, because it can be used to predict scores of another variable (i.e., we predict T-shirt sales based on whether or not a gift was offered). In this situation it is said to have two levels, because it has been manipulated in two ways (i.e., free gift or no free gift). The outcome in which we are interested is T-shirt sales. This variable is called the dependent variable because we assume that its value will depend upon whether or not a free gift was offered (the independent variable).”
Synonym: predictor variable
If it helps, think of it this way: We assume T-shirt sales will grow if there is a free gift. We measure the number of t-shirts sold, and we “measure” whether or not gifts were offered. The first fact, the number of T-shirts sold will [hypothetically] DEPEND on whether or not there were free gifts. The free gifts are a given, and they are there or not based on how the experiment is set up, INDEPENDENTLY of the other measured variables.
Inferential Statistics
“It boils down to two things that you might want to do with data. The first is to describe what happened in the sample that you collected. You might draw a graph of the data, or calculate some summary information such as the average T-shirt sales. This is known as descriptive statistics. However, because
scientists usually want to generalize their findings beyond the data they collected to the entire population, they use the sample data to estimate what the likely values are in the population. This is known as inferential statistics. Inferential statistics help us to make generalizations about what is going on in the real world, based on a sample of data that we have collected.’”
Very simply, with descriptive statistics you DESCRIBE your data, and with inferential statistics you INFER something out of your data.
Interval estimate
“‘You mean that the value in the sample won’t always be the same as the value in the population?’
‘Yes, exactly, so we can instead compute an interval estimate, which is a range of values between which we think the population value is likely to fall, based on the amount of sampling error we expect for that sample.”
Compare: point estimate
Think of it this way: You want to study whether being a Milestone student increases motivation to learn new foreign languages. But you can’t survey every Milestone student - it would be impossible to track them all down. So, you take a sample from the population, say, 60 students.
From this sample, you calculate a sample statistic; for example, the average motivation score (which will be a point estimate for the population). An interval estimate, such as a confidence interval, then gives you a range of values within which the true population parameter (like the actual average motivation of all Milestone students) is likely to fall, with a certain level of confidence (commonly 95%).
Latin square design
“Sometimes people use a Latin square counterbalancing method. […] ‘Imagine we just had three conditions to our experiment: we asked 30 people to complete the statistics test as a statistics professor (A), as themselves (B), and as an arts professor (C). In a Latin square design with three conditions we’d split the 30 people into three equal groups. The first group would complete the tasks in order A, B, C (i.e., as the statistics professor, as themselves, then as the arts professor). However, the second group would complete it in order C, A, B (i.e., as the arts professor, as the statistics professor, then as themselves). The final group would complete the tasks in order B, C, A (i.e., as themselves, then as the arts professor, and finally as the statistics professor). The important thing is that across the participants each task or condition appears equally as the first task, the second task and the last task. So, a third of the participants do the task as themselves first, a third of them take that task second, and for the final third it is the last task. Therefore, the order of tasks is balanced. You can do the same type of arrangement with more tasks. With 4 tasks you’d need 4 groups who complete the tasks in 4 different orders, and with 5 tasks you’d need 5 groups who complete the tasks in one of 5 different orders. In all cases, though, across all of the groups a particular task is done at every different position in the order of tasks.’
Check also: counterbalancing
This quote here talks about counterbalancing the boredom effect. So if you ask the same person to fill in a questionnaire multiple times (acting as different people), how do you counterbalance the boredom effect (ensure that it does not sway your data)? This is what the quote above explains in detail.
Longitudinal study
“Alice nodded. ‘When you observe what naturally happens in the world without directly interfering with it, it is known as correlational research. There are different ways to do this: we could take a snapshot of many variables at a single point in time (a cross-sectional study), or measure variables repeatedly at different time points (a longitudinal study).”
For example in a longitudinal study you could take a group of people (without brain cancer) and give them phones, and then measure their phone use and consistently check them to see if they are developing brain cancer over, say, a period of five years. This includes repeating the same measurement over a relatively long time period.
Meta-analysis
“Yes, and pulling together the results of lots of studies on the same question is known as a metaanalysis. It helps us to get more conclusive answers to questions from a range of studies on the same topic.’”
In a meta-analysis, researchers systematically review prior studies on a given topic, and report their findings. Here is one example: https://doi.org/10.14746/ssllt.2018.8.4.2
Outcome variable
“This variable is called the dependent variable because we assume that its value will depend upon whether or not a free gift was offered (the independent variable). More generally, we can refer to it as an outcome variable, because it is the variable that we’re trying to predict the values of (i.e., we want to know how many T-shirt sales there are).
Synonym: dependent variable
If it helps, think of it this way: We assume T-shirt sales will grow if there is a free gift. We measure the number of t-shirts sold, and we “measure” whether or not gifts were offered. The first fact, the number of T-shirts sold will [hypothetically] DEPEND on whether or not there were free gifts [> this will give us the OUTCOME of our study]. The free gifts are a given, and they are there or not based on how the experiment is set up, INDEPENDENTLY of the other measured variables.
Parameter
We can use the data in the sample to compute statistics, which are values that describe the sample. So, the average number of T-shirts sold in our sample is a statistic. However, we an use this value to estimate what the value would have been if we had collected data from the entire population. The value in the population is known as a parameter.’
As the textbook clarifies, in out t-shirt study, the average number of t-shirts bought by the participants in our sample will be our statistic, but we can estimate what this average would have been for the whole population, this is the parameter.
Point estimate
“‘Zach, here are 23 studies from the pre-revolution that looked at whether mobile phone use is related to brain tumours.13 There are no studies that have looked at the Proteus, so this is all we have to go on. I’ve listed the studies down the side of the picture. For each study, the scientists computed a statistic in the sample that represents the size of the effect that phone use had on brain tumours. This statistic is represented by the dots. Remember that we’re interested in the effect of phone use on tumours in the whole population, not just in that particular sample. We could use this statistic as the estimate of the effect in the population. If we do this we are using a point estimate because we’re using a single value, or point, to estimate the effect in the population. However, we know that there will be sampling error.’
Compare: interval estimate
Imagine you measure the height of a sample of 50 Milestone students. The mean height you calculate from this sample is your point estimate for the average height of all Milestone students. You then calculate a confidence interval (don’t worry about the how for now). This interval gives a range within which you’re fairly confident the true average height of all Milestone students falls. It’s not that every individual student’s height is within this range, but that the overall average height of all Milestone students would fall within this interval if you measured everyone. There is another explanation on the interval estimate card, about motivation to learn foreign languages.
Population
“Although you might care only what happens with your band, The Reality Enigma, normally scientists are interested in theories that apply very generally − they want their theories to apply to an entire group of entities or situations. An entire set of entities is known as a population. A population can be quite diverse (for example, you might want to draw conclusions about the T-shirt sales of every band on the planet) but can also be more specific (you might be interested in drawing conclusions only about bands who play a certain style of music, like heavy metal). Different types of scientists might focus on different populations. I work in genetics, so I want my theories to generalize to the population of humans, and this population would also be interesting to psychologists, and epidemiologists too. However, an economist might be interested in the population of “small businesses” or “workers” or “managers”, and biologists might be interested in the population of “cells”.
Think if you wanted to study the motivation of Milestone students. Do you think you could ask every single one of them? Unlikely. There would be at least a few who would not want to participate, or simply you would not be able to track them down. In this case, you take a SAMPLE of your POPULATION, say, 60 students. In this case, your population is “Milestone students” but you work with your smaller sample only.
