PL1010: Research Design and Methods Flashcards
Your Deck Mentor for the Research Design and Methods deck is Bogna. You can email bogna.frykowska@forward-college.eu with any questions/suggestions about the flashcards in this deck.
1
Q
Categorical variable
A
Variable with scores that are not on a numeric scale
2
Q
Descriptive statistics –
A
Summarise samples – giving someone the main points in a simple form To describe data, we will use graphical and numerical (statistical) techniques
3
Q
Inferential statistics –
A
Examine patterns in the data and consider how much data we have You can then draw conclusions about a population based on the analysis of a sample. -> conceptual replication
4
Q
Summarising
A
collecting and summarising data
5
Q
Standardisation
A
creating a system in which data can be assigned values (high vs low vs average)
6
Q
Statistical inference
A
the ability to draw general conclusions from samples
7
Q
How many times does a particular score occur?
A
Percentages/Averages Scores for a particular variable (Frequency statement)
8
Q
Do scores for one variable correlate with scores for the other variable?
A
Statement about association
9
Q
How strong is the correlation or association between two variables?
A
Statement about association
10
Q
Do I trust that there is a “genuine” association (relationship)?
A
Statement about relationship between two variables
11
Q
Frequency Distribution?
A
show scores in order and their frequency of appearance in the sample
12
Q
Negatively skewed
A
13
Q
Positively Skewed
A
14
Q
When not to describe the skew of data?
A
When we cannot put our scores in order , from lowest
to highest so when we are describing a categorical
variable with unordered categories
15
Q
Unimodal?
A
One major peak
16
Q
Bimodal
A
Two major peaks
17
Q
A
Approximately symmetrical
18
Q
The Sample Variance formula
A
the n-1 leads to an unbiased estimate
19
Q
How do outliers and the mean relate to each other?
A
Outliers are extreme values that differ from most values in the data set. Because all values are used in the calculation of the mean, an outlier can have a dramatic effect on the mean by pulling the mean away from the majority of the values.
20
Q
What happens to the mean, median and mode in a skewed distribution
A
in normal distributions, they all take on the same number
21
Q
frequency distribution
A
pattern of frequencies of a variable
22
Q
Why are histograms good?
A
effective visual summary of a variable’s central tendency and variability
23
Q
Why is the IQR a good measure of variability?
A
Because it’s not affected by outliers like the range is as it only includes the middle 50% of data?
24
Q
Difference between central tendency and variability?
A
Central Tendency just tells you where the most important point lie while variability sums up how far apart and thus generalisable they are.
25
standardised scores (z-scores)
26
Z-Score Formula
27
Falsifiability
capacity for some proposition, statement, theory or hypothesis to be proven wrong (through systematic empiricism) a basis provided by the null hypothesis
28
null hypothesis
states the contrary of the experimental or alternative hypothesis
29
falsifiable hypothesis
can be logically contradicted by an empirical test that can potentially be executed with existing technologies .
30
A linear correlation describes
Two variables that are either proportionate or anti proportionate
31
Correlation coefficient
A number also represented by "r" describes positive (r=1), negative (r=-1) or no correlation (0=r)
32
Face Validity
basic form of validity demonstrated when a measurement procedure superficially appears to measure what it claims to measure
33
Concurrent Validity
demonstrated when scores obtained from a new measure are directly related to scores obtained from an established measure of the same variable.
34
Predictive Validity
demonstrated when scores obtained from a measure accurately predict behaviour according to a theory.
35
Construct Validity
requires that the scores obtained from a measurement procedure behave exactly the same as the variable itself.
36
What needs to be taken into consideration for construct validity
based on many research studies that use the same measurement procedure and grows gradually as each new study contributes more evidence.
37
Convergent Validity
demonstrated strong relationship between the scores obtained from two or more different methods
38
Divergent Validity
demonstrated by showing little or no relationship between the measurements and two constructs
39
Test-retest reliability
established by comparing the scores obtained from two successive measurements of the same individuals and calculating a correlation between the two sets of scores.
40
Inter-rater reliability
degree of agreement between two observers who simultaneously record
measurements of the behaviours
41
Split-half reliability
obtained by splitting the items on a questionnaire or test in half, computing a
separate score for each half, and then calculating the degree of consistency between the two scores for a
group of participants.
42
What are the two types of measures of reliability
Successive and simultaneous measurements
43
Construct
hypothetical attributes or mechanisms that help explain and predict behaviour in a
theory
44
Operational Definition
procedure for indirectly measuring and defining a variable that cannot
be observed or measured directly. An operational definition specifies a measurement procedure (a set of operations) for measuring an external, observable behaviour and uses the resulting measurements as a
definition and a measurement of the hypothetical construct.
45
What is naturalistic observation
A form of non-participant observation where a researcher is in a natural setting in which behaviur usually occurs without interupting
46
What is In participant observation
researcher engages in the same activities as the people being observed
in order to observe and record their behaviour.
47
What are naturalistic observation usually used for
used to describe non-human behaviour or children
48
What are the benefits and disadvantages of naturalistic observation (5)
external validity: real world setting behaviour is not manipulated overcoming ethical barriers --> instigating spanking vs. observing spanking time-consuming: having to wait till behaviour occurs research is prone to interruptions
49
When is participant observation needed
When simple observation is not possible. e.g. studying cults or gangs because their presence would alter the behaviour
50
What are advantages and disadvantages of participant observations (5)
access to information and observation unavailable to mere outside observation high external validity because of naturalistic setting time consuming observation potentially dangerous for researcher (sensitive nature) observers presence might alter participants' experience --> objectivity?
51
What is Structured observation?
or contrived observation is the observation of behaviour in settings arranged
specifically to facilitate the occurrence of specific behaviours so they don't have to wait for them to happen
52
Advantages and Disadvantages of structured observations
can be held in laboratory or other controlled settings to percipitate the behaviour that they want to observe --> good for developmental psych can be held in what is perceived a sa natural environment (field setting) less time consuming how natural is the behaviour?
53
During a study using observational methods, it is common to have two observers record behaviour
simultaneously. What is the purpose for this procedure?
objectivity of the measurements
54
In an observational study of children diagnosed with autism spectrum disorder, you record how
much time each child spends playing alone during a 30-minute observation period. Which method
of quantifying behaviour is being used?
duration
55
When researchers use behavioural observation techniques to measure behaviours in movies, what
is the measurement process called?
content analysis
56
What are behavioural tasks
usually computer-controlled, structured tasks measured across multiple repeated trials that researchers use to collect behavioural measures such as
response times and task accuracy measures.
57
What are many behavioural tasks structured around?
human information processing like cognitive tasks assessing attention, memory, language and decision making
58
What can also be measured by behavioural tasks
attitudes, preferences aside from cognitions
59
What is the main measure of interest in behavioural tasks
not the usually correct answer but speed of the response (response or reaction time)
60
What is a task paradigm
task originally constructed to investigate a particular hypothesis is used and adapted to examine others subsequently providing a standard model for line of research
61
What are a few physiological measures commonly used
monitoring
heart rate or blood pressure, measuring
galvanic skin response, imaging
techniques positron emission tomography
(PET) scanning magnetic resonance imaging
(MRI) electroencephalogram (EEG) magnetoencephalography (MEG).
62
What are two prerequisites of behavioural observations
behaviour is not disturbed observations are based in subjective judgments and intepretations which pose a threat to reliability so need for more than one observer
63
Archival research
involves looking at pre-existing records (archives) to measure behaviours or events
that occurred in the past
64
Content analysis
measuring the occurrence of specific events, actions or statements in written
text (e.g., literature, press reports, transcripts) or film/video recordings (e.g., movies, television programmes) or similar media
65
How is the issue of interpretation in observational designs addressed (3)
well-defined categories of behaviour well-trained observers multiple observers or coders to assess inter-rater reliability
66
What are behaviour categories
well/defined sets of behaviour that is to be observed which helps isolating relevant behaviours
67
How are observations quantified (3)
frequency: how many times does something occur in the given time-frame duration: for how long does a behaviour occur interval: does a behaviour occur in a given interval
68
When are the three quantification methods most appropriate
first two techniques are often well suited for specific behaviours but can lead to distorted
measurements in some situations. For example, a bird that sings continuously for the entire 30-minute
observation period would get a frequency score of only 1. Another bird that sings 25 times with
each song lasting two seconds would get a duration score of only 50 seconds. In such situations, the interval method provides a way to balance frequency and duration to obtain a more representative
measurement
69
How do observers overcome issues of complex situations that cannot be watched multiple times
creating a recorded sample or taking a general sample
70
How is a sample taken
first step in the
process of sampling observations is to divide the observation period into a series of time intervals.
71
What are the three forms of sampling
Time sampling: sequence of observe–record–observe–record is continued through the
series of intervals Event sampling: identifying one specific event or behaviour to be observed and recorded
during the first interval, then shifting attention to a different event or behaviour during the second
interval, and so on, for the full series of intervals. individual sampling: identifying one participant to be observed during the first interval,
then shifting attention to a different individual for the second interval, and so on
72
How is reliability and objectivity of observations made from content analysis archival research ensured?
behavioural categories and preparing a list of specific examples to define exactly
which events are included in each category being measured quantification methods for each behavioural category multiple observers and coders
73
What does reliability often refer to
the relationship between two measures as shown by its correlation
74
When is assessing split/half reliability common
single variable measured within a test containing multiple items so that the internal consistency can be evaluated
75
What is the issue of split-half reliability
scores obtained are only from half of the test items which is less reliable because it underestimates the true reliability of the full test
76
What is the Spearman-Brown formula, and what does it do?
adjusts the correlation between the halves of split-half reliability tests, the effect is to increase the size of the correlation to produce a better estimate for the full test
77
What problem of split-half reliability does the Kuder-Richardson Formula 20 solve
The idea that tests can be split in different ways which potentially skews the results
78
What is the Kuder-Richardson Formula 20
a formula to estimate the average of all possible split-half correlations obtainable but limited to tests with dichotomic answers
79
What do all of the components of the K-R20 mean
n / number of items SD p / the proportion of the participants whose response is coded 0 q / proportion of the participants whose response is coded 1
80
How is the K-R20 limited
It can only be used for test that have dichotonomical answer systems and Cronbach's alpha is a modification to this
81
What are the components of cronbach's alpha
the extension is that it includes the sum of the variance produces values between 0 and 1.00
82
What is Cohen's Kappa formula, and what is it used for?
calculating inter-rater reliability not using a simplistic formula and relying on data prone to circumstances and chance
83
What do the elements of Cohen's kappa mean
PA: observed per cent agreement PC; per cent agreement expected from chance
84
When is Cronbach’s alpha used?
when we have a scale that combine responses from several rating-scale items.
85
What’s the scientific method
Way of acquiring knowledge that includes the genesis of a hypothesis and then its systematic investigation
86
What are the three principles of the scientific method
Empirical: systematic/structured observation (with attempts to isolate the relationship between variables) Public: Data available for evaluation/verification to be replicated Objective: low bias
87
What are the steps of the research process?
Research Idea (Field Review) Hypothesis Defining Variables & Measure Participant Selection (Criteria, Ethics) Research Strategy (Design/Ethics) Research Design Selection Evaluate Data & Report Results Refine research idea
88
What are the four main functions of statistics?
Summarising (descriptive statistics) Providing estimates for populations taken from samples (inferential statistics) Data simplification/reduction when there is a lot of data New statistical techniques are needed for mining big data
89
What are the seven reasons research ethics must be considered at every stage of the process?
Dictate measuremets Participant selection Research stategies x population Research design x behavior/populus How the study is conducted Data analysis Reporting results
90
What is a random sample?
By drawing randomly from a population all events should have an equal chance of being included
91
What is meant by inference?
Estimation based on population
92
What are the three decisions of statistical procedures?
Type of data Differences vs. Relationships Number of groups vs. variables
93
What are research ethics?
Principles that concerns the responsibilty of researchers to be honest and respectful towards participants
94
What are the 5 research strategies?
Descriptive (examining individual variables) Correlational (two variables for each individual) → numerical Experimental (cause-effect) Quasi-experimental (less control, assignment) Non-experimental
95
What is a research strategy
General approach to the research shaped by the research question (what do I want)
96
What is meant by the research design
General framework to implement research strategy (how do i achieve what I want)
97
What are the three pillars of research design?
Group vs. individual Same individuals vs. different Number of included variables
98
How can a study’s aspects threaten external validity? (4)
General q: how can the results obtained with this procedure be replicated to other procedures? Multiple treatment interference (fatigue/practise) Novelty effect (anxiety/excitement) Experimenters’ influence
99
How can participants/ subjects threaten external validity?
volunteer/selection bias/ uni students WEIRD characteristics Cross-species comparison
100
How can the measurements threaten external validity?
Assessment sensitisation (awareness) Pre-test sensitisation Results of an operationalised concept can be moderated by the measure → generality across measures Time of measurement
101
What is internal validity?
Continuity that the observed results can account for the propose cause-effect relationship
102
How can environmental variables threaten internal validity?
The room size, the colour of the walls, time of the day, the gender of the experiementer NO SYSTEMATIC DIFFERENCES IN THE ENVIRONMENTS
103
How can individual differences threaten internal validity?
IQ, age, gender, health conditions WEIRD vs non WEIRD
104
How can time-related variables threaten internal validity?
Individual differences that accumulate over time Comparing scores and time influences
105
What is the biggest threat to internal validity?
The effect of extraneous variables that confound the results
106
What are artefacts?
External factors can become a confounding variable and distort both internal and external validity
107
Which artefacts concern the participants?
Demand characteristics: participants react to cues that reveal the purpose/ hypothesis Reactivity: induce behaviour (subject roles)
108
What do participant-related artefacts primarily threaten?
Internal validity: reactivity explains phenomena; not generalisable
109
What are non-participant-related artefacts?
Experimenter bias (single-blind; double-blind) Exaggerated variables
110
What are the four categories of formal measurement theory and who came up with them?
SS Stevens Nominal: just naming Ordinal: small, smaller, smallest Interval: 0-1 = 7-8 Ratio: zero = nothing
111
What is a discrete, continuous, independent and dependent variable?
Discrete: variable that is limited (age, gender) Continuous: exists on a continuum basically infinite between highest/lowest IV: variable manipulated/changed to see whether it has an effect on the DV that might change because of the manipulation DV: variable that, though measured, is not being controlled
112
What is the role of measurement scales?
The numbers don’t necessarily say anything concrete about the objects measured ex.: if I scored high on a test, but someone else scored lower, it’s not necessary because they remembered less even though the data might suggest it → we assume that they mean I remembered more
113
What is the purpose of a frequency distribution?
Organising data into a meaningful order of how many times
114
Which variable do I usually find on the X- and Y-Axis in histograms vs. line graphs?
histogram: dv-iv Line/Bar graph: iv-dv
115
What is the mode, median, mean (+formulas)?
Mode: the highest point in the graph Median: 50th percentileMean: Sum of N/ N
116
If the mean is slightly larger what does it probably say about or distribution?
Positively skewed
117
When will the mean and the median be equal?
Symmetric distribution
118
The benefit of the mode is?
* Representing categorical data * More informative *But not very reflective of the remaining data set
119
The benefit of the median is?
Not affected by outliers Not stable in comparison and not useful to calculation
120
What is the trimmed mean and when do we use it?
Trimming the interval and taking the mean when the data is very skewed
121
What does central tendency refer to?
The scores tendency to distribute in a certain way?
122
What is meant by dispersion/variability around the mean?
Determining how the scores relate towards the mean score
123
What are measures of variability?
Range Interquartile Range Standard deviation Sample variance if sd=0 so we square scores and then take the sum so negative scores become positive Absolute Mean Deviation SD but not squared which could= 0
124
What is the advantage of a bar chart?
* Comparing categories * Mirrors other visualisation techniques were the spread is along the X-axis and the frequency or percentage is along the Y-axis already hints at modality and skewness
125
Why is a normal distribution the most important distribution ?
We assume normal distribution, we can use most statistical techniques Most dependent variables are thought to be normally distribution We can make inferences If we draw a theoretical representation of all possible sample means the sampling distribution would also be normally distributed
126
What are the mean and standard deviation in the standard normal distribution?
mean=0 and sd=1 → z-score standard score specifying the amount of distance of the sd
127
What is an alternate name for the y-axis/x-axis?
ordinate/abscissa
128
What is the mean and the standard deviation of t-scores?
Mean = 50 SD= 100
129
What is meant by sampling error?
Chance difference = the way some statistics naturally varies from sample to sample = in that it wll always deviate from the parameter it is the random variability = standard deviation of the sampling distribution
130
What is sampling distrubtion?
If we calculated a statistic from different samples from one population and visualised it we would see how the statistic is distributed
131
What is the standard error of the mean?
The standard deviation / variability from the estimated parameter mean of the distribution
132
What is the purposes of the standard error?
It gives us an indication of just how much the sample statistic might differ across the samples. It’s like a z-score for all the potential differences we could observe but would not go against the finding
133
What are the logical steps of hypothesis testing?
Set up a research Hypothesis H1 Set up a null hypothesis H0 Get a sample and sample distribution of sample statistics (eg mean) und the H0 Calculate probability value of of sample statistic at least as large as the one obtained Reject or Fail to Reject H0
134
What’s the philosophical hypothesis of the null hypothesis?
M1-m2 =0 has been proposed by Fisher With the logic that we can always show that something is false
135
How do you calculate the IQR?
Order the scores Find the median location (N/2) Find the median of the upper and lower quartile N low /2; N high /2
136
What is the difference between sample and test statistics?
Stats describing samples vs. statistical results of specific proceedures with their individual sampling distributions
137
What is the formula for Cohen’s d effect size?
d = (M1 – M2) / Spooled
138
What do d= 1 and d= .5 indicate?
That the effect the difference is either twice or half as large as the standard deviation
139
How do I interepret the effect size using cohen’s d?
.2 is small because the mean difference is around .2 standard deviation → .5 (medium), → .8 (large)
140
What is the defining difference between in-between subjects and within-subjects design?
Create equivalent groups and compare them in different trearment conditions vs. use the same group of participants compared in all different trials
141
How can the trials be administered?
Subsequentially or intermixed
142
What’s instrumentation?
Changes in the measurement or measuring instruments (observations are heavily dependent on the observing researcher)
143
What is a research strategy?
A method of data collection
144
What is a non-experimental and quasi-experimental research strategy?
No manipulation and controlling for extraneous variables vs. limitation of confounding variables without controlling the environment
145
What aspects threaten internal validity?
Time order effects the fact that counterbalancing cannot be applied
146
What are the absolute deviations from the mean?
X- Mean that's why if we average and take the root of them we get the standard sample deviation
147
Suppose you sell ice cream with three different flavours: chocolate, strawberry and yogurt. The ice cream flavours are measured on a ____________ level. You sell ice cream to children, adults and elderly people. These age groups are measured on a ____________ level.
nominal; ordinal
148
operational definition
defining a variable in terms of the set of steps or procedures that the researcher goes through in order to manipulate or measure the variable
149
right skewed
positively skewed
150
What does a negatively skewed distribution reveal?
A lot of people got close to the maximum score
151
What does central tendency mean?
average score
152
how much is removed in the 5% trimmed mean'
10% in total 5 from the top and the bottom
153
‘Alex has an accuracy score of 15, while Samir has an accuracy score of 5; therefore, Alex was three times as accurate as Samir.’ For this statement to be appropriate and precise, which scale(s) of measurement could accuracy have been measured on?
Question 7
a. Interval
b. Ratio
c. Ordinal
d. Nominal
This statement is only meaningful if a score of ‘0’ represents ‘no accuracy’. Therefore, the scores should be from a ratio scale if this statement is to be made. b
154
Age in months is an example of a variable with a ratio scale of measurement. Select one:
True
False
T
155
Sampling variability
differences in or across samples due to random things happening
156
Confidence
accuracy across 100 treatments that we've found the likely range limits
157
Type I error
158
Type II error
159
What are two ways to visually represent to measurement data variables?
1. scatter plots 2. contingency tables/crosstabulation
160
What is a way to visually represent a mix of categorical and measurement data?
compound histogram
161
What is a way to visually represent categorical data pairs?
crosstabulation
162
How should the strength of a correlation be interpreted?
* Perfect:(-) 1/1 * Strong:(-) 09-07 * Moderate:(-).6-.4 * Weak: (-). 3-.1 * None:0
163
What do frequency distributions for a categorical variable not include?
cumulative percentages
164
What are the groupings of scores in histograms called?
bins
165
Do these images show the same data?
Yes
166
Which visual representation should you choose if you want to show that variables vary simuntaneously?
scatter-plots
167
What does a boxplot do?
summarises the data while showing the range, interquartile range, as well as the min, max and the median
168
When is the mean most useful?
best for interval/ratio measurement data (categorical data can hardly be split into 2), needs equal spacing between adjacent values
169
What is the mode most useful for?
all but notably for nominal/ordinal categorical data because popular choice
170
When you collect data from a sample, the sample variance is used to ?
make estimates or inferences about the population variance and comparing the variance of samples helps you assess group differences
171
How is the sample mean related to variance and standard deviation?
it is expanded on in the formulas for variance and standard deviation
172
Which five steps need to be taken to calculate the sample variance?
1. The mean (∑ 𝑋 /N) 2. The Deviation from the mean X- (∑ 𝑋 /N) 3. Squared deviation from the mean (X- (∑ 𝑋 /N))^2 4. Find the sum for all scores and devide by N-1 5. Take the root to find the standard deviation or z-score
173
What are strengths naturalistic observation?
1. behaviour observed in the real world 2. useful for non-manipulated behaviours 3. actual behaviours observed and recorded
174
What are strengths of participant observation?
1. when natural observation is not possible 2. get information not easily accessible 3. participation gives unique perspective
175
What is a strength of structured observation?
do not have to wait for behaviours to occur
176
What is a weakness of structured observation?
less natural
177
What are weaknesses of participant observation?
1. time-intensive 2. potential loss of objectivity 3. increased chance of observer influence
178
What are weaknesses of naturalistic observation?
1. time-intensive 2. potential subjective interpretation 3. potential for observer influence
179
Variables are
properties of objects that vary in the values that they take on
180
A score is
an individual value for a variable
181
Measurement data describes
scores on a numerical scale
182
Categorical data describes
scores not on a numerical scale
183
A Population describes
a complete set of scores that might be of interest
184
A Sample is
a sub-set of scores from a population which were obtained
185
A parameter is
a number that summarises the entire set of scores in a population
186
A statistic is
a number that summarises the scores in a sample
187
Descriptive Statistics...
summarise samples by presenting the main points in a simplified way
188
Inferential statistics...
examine patterns in the data and consider the amount of data
189
Ethnicity or political ideology are examples
nominal variables
190
The lower quartile signifies
all scores one-quater into the distribution
191
The upper quartile signifies
all scores three-quaters into the distribution
192
The range signifies?
the difference between the minimum and the maximum score
193
The interquartile range signifies?
the difference between the upper and lower quartiles
194
The standard deviation is more informative about the variability than the variance.
False
195
The standard deviation is expressed in larger units than the variance.
False because the root is taken
196
What does the standard deviation tell me?
how far, on average, a value lies from the mean which is why it is derived from the variance (square root)
197
Which graph describes a correct null hypothesis?
right
198
The p-value can be defined as
the probability of obtaining a significant result when the null hypothesis is true
199
What are the five characteristics of normal distributions?
