Section 2: Scientific Processes Flashcards
What distinguishes experimental methods from the other methods?
All experiments have: IV, DV, cause and effect relationship. The aim is to establish the cause and effect relationships between the IV and the DV.
Independent Variable
The variable in an experiment which is manipulated by the researcher. This is done to see if it has any effect on the DV.
Dependant variable
The variable in an experiment which is observed/measured by the researcher. This is done to see if it has been affected by the changes made to the IV
Extraneous variable
Any variable other than the IV that might have an effect on the DV if it is not controlled. As part of designing any study, researchers should try to identify all possible EVs and attempt to control them.
4 types of extraneous variables
Participant variables
Investigator effects
Environmental variables
Demand characteristics
Participant variables
any individual characteristics or traits of the participants (other than the IV) that might affect the DV and unfairly influence the results.
Examples of participant variables
age, gender, personality, mood, intelligence
Investigator effects
Any cues or behaviour from the investigator (other than the IV) that may encourage certain behaviours in the participant, and which may allow the researcher’s expectations to unfairly influence the results, causing inaccurate results.
examples of investigator effects
words used/instructions given
tone of voice
body language
individual/physical characteristics (age/gender)
environment variables
any aspect of the research environment or situation (other than the IV) that might unfairly influence the results.
examples of environmental variables
noise, lighting, temperature, weather, time of day
demand characteristics
any cues that reveal the aims of the study to the participants. if participants become aware that they are being studied, these cues may help them to work out the aim of the study and cause them to change their behaviour, giving inaccurate results
negative effects of demand characteristic
‘please you effect’ : may try to please the researcher by giving them the result they want
‘screw you effect’ : may try to ‘ruin’ the experiment and the results
may be more self conscious if they know they are being tested/observed
- results wont reflect natural behaviour: low internal validity
examples of demand characteristics
participants working out/guessing the aim of the study due to cues given
leaving questions that reveal the aim of the experiment
2 key ways of controlling variables to ensure accurate results
standardisation
randomisation
standardisation
standardising research means putting in place controls to ensure every aspect of the research is the same for all participants and meets a consistent standard.
includes a standardised environment, standardised procedures, and standardised instructions.
how to standardise environment
using a laboratory to ensure every aspect of the environment is controlled
how to standardise procedures
using the same research methods
using the same timings
using the same materials
how to standardise instructions
read from a script or prerecord the instructions (same for all)
use the same researcher to give instructions
randomisation
ensuring all research choices are randomly selected by chance (eg using a random number generator) rather than being determined by the researcher. This is done to avoid researcher bias.
examples of randomisation
using a random name/number generator
random allocation of participants to different conditions
putting task sequences together randomly (eg a list of words) instead of the researcher putting them in a chosen order
how to control participant variables
- randomisation: random allocation of participants to different conditions
- experimental design: change to a repeated measures or matched pairs design if appropriate
how to control investigator effects
standardisation: give the same instructions to all participants (scripted/prerecorded), use the same researcher for all participants
randomisation: random allocation of participants to conditions to avoid researcher bias
single or double blind trial to avoid researcher bias
how to control environmental variables
standardisation: standardised environment (eg using a laboratory to control all environmental variables)
confounding variable
any extraneous variable that is not controlled and therefore may have ‘spoiled’ (confounded) the results.
This is because it is no longer clear if any change in the DV was caused by the IV or the confounding variable.
If a study has confounding variables then it’s results are invalid (inaccurate)
validity
the accuracy of the research and it’s findings
the extent to which an observed effect is genuine
two main types of validity
internal and external validity
internal validity
whether or not the research measured what it intended to measure
how successful the study has been in controlling extraneous and confounding variables
external validity
whether or not the research findings can be generalised to the outside world
whether the research is representative of people, places, and times in the real-world.
three types of external validity
ecological validity
population validity
temporal validity
ecological validity
a form of external validity that refers to whether or not research findings can be generalised to real life settings or situations
population validity
a form of external validity that refers to whether or not research findings can be generalised to other people in the target population
temporal validity
a form of external validity that refers to whether or not research findings can be generalised to other historical times and eras (such as from a long time ago to the current day)
high internal validity
EVs and CVs were successfully controlled and the IV did cause the change in the DV (measured what it intended to measure)
low internal validity
EVs and CVs were not successfully controlled and therefore may have affected the findings. any change in the DV may have been caused by something other than the IV (bad because it didn’t measure what it intended to measure)
high external validity
external validity will increase the more you conduct the study in different settings, with different people, and at different times to produce the same findings
low external validity
external validity will decrease the less you conduct the study in different settings, with different people, and at different times if the same findings are not produced
reliability
consistency of the measurement/findings
the results of a research are reliable if, when the same measurement or study is replicated, similar/the same results are produced
how to test reliability of a study
test-retest method
test-retest method
running the study once and then doing exactly the same study again using exactly the same participants and conditions. if the same/similar results, the research is reliable.
pilot study
a small-scale trial run of a research study that takes place before the full-scale research project begins.
They usually use a smaller number of participants who can be questioned afterwards about their experience.
aims of pilot study (piloting)
to check that the research/study works as it is intended to, and that there are no extraneous variables that may affect the results, and that it is practical. Any problems can then be rectified before running the full scale study
how are pilot studies useful
answering:
do participants understand the instructions?
are the materials and timings appropriate?
how was the participants’ experience (demand characteristics)?
have variables been operationalised successfully?
any problems can be rectified before the full scale study begins
how can pilot studies help to see if participants understand the instructions
participants can be asked if the instructions were clear ( or in a question are if the questions were clear)
if they understood what they were required to do
if the questions/instructions meant the same thing to all participants
how can a pilot study help to see if materials and timings were appropriate?
eg in a memory recall research:
- can check whether words are clear for the age group/ability of the participants
- whether the list contains the right amount of words,numbers,images etc
- whether pictures used are clear
- whether time shown for is appropriate
eg in an interview or question are research:
- helpful to try out questions in advance and remove any that were confusing or ambiguous
ensure participants have the right amount of time to do complete what they are supposed to
how can a pilot study identify demand characteristics
asking participants about their experience of taking part: whether they guessed/figured out the aim of the study and therefore changed their behaviour
how can a pilot study help to check if variables were sufficiently operationalised?
may help to highlight more behaviours not previously outlined
(eg studying children’s aggression: how many times the hit/kicked, the PS may include biting etc)
allow researcher to identify and operationalise all variables in order to maximise internal validity
aim
a general statement about what the researcher intends to study: the aim states the purpose of the study.
eg “Aim: to investigate the effects of noise on memory”
hypothesis
a precise and testable statement that states the expected relationship between variables.
‘Operationalisation’ is a key part of making this statement precise and testable.
If the findings support the hypothesis then it can be accepted, but if not it must be rejected.
difference between aims and hypotheses
an aim is a general statement about what the researcher intends to study that doesn’t precisely define variables
whereas
a hypothesis is a more precise and testable statement that clearly defines variables and the expected relationship between them
three types of hypotheses
directional
non-directional
null
directional hypothesis
a hypothesis that not only started that there will be an effect but also states which way the effect will go.
(eg: saying it will be higher/lower than another condition)
justification of directional hypotheses
psychologists use directional hypotheses when past research suggests that the findings will go in a particular direction
non-directional hypothesis
a hypothesis that simply states that there will be an effect but doesn’t state which way the effect will go.
(eg: saying that there will be a difference between conditions but not saying which group will score higher/lower)
justification for non-directional hypotheses
psychologists use it when past research is absent (no previous research in the area) or mixed/unclear
null hypothesis
states that there will be no significant difference: the IV has no effect on the DV
template for directional hypothesis
participants that (IV1) will (DV) (more/less/greater/smaller/quicker/slower) than (IV2)
template for non directional hypotheses
there will be a significant difference between participants that (IV1) compared to participants that (IV2) in terms of (DV)
template for null hypotheses
there will be no significant difference between participants that (IV1) compared to participants that (IV2) in terms of (DV)
operationalisation
the process of clearly defining variables
to make them testable ands measurable
this is how variables are ‘put into operation’
eg, rather than stating intelligence state IQ score
social desirability
participants behaving or responding dishonestly to present themselves in a more favourable light
often happens in self-report research such as interviews when asked personal questions
population
the population refers to the large group of individuals who share specific characteristics that a researcher is interested in studying
in psychology the population does not refer to the general population and therefore it is referred to as the target population, as it refers to a particular subset of the general population
eg, year 12 students at a particular college
eg, teenage boys in britain
why is it necessary to select a sample out of a population
populations are usually too large to study as a whole (too expensive, time consuming, inefficient) so we select a sample that is representative of everyone in it
difference between population and sample
a population is a large group of individuals who share specific characteristics that the researcher is interested in studying whereas a sample is participants who are selected from the target population, that is representative of a smaller group of a target population
sample
a group of the target population that is selected for a study.
ideally, the sample drawn will be representative of the target population so that generalisation of findings becomes possible, giving the research high population validity
generalisation
the extent to which findings from a specific sample can be applied to the population.
this is made possible if the sample is representative of the population.
general issue with sampling
finding a sample that is representative of the population so that generalisation of findings is possible and it has high population validity is often difficult to achieve in practice, and most sampling techniques involve some degree of bias
bias (sampling)
in terms of sampling, this refers to under-representing or over-representing certain groups within the sample.
for example, too many younger people or too few males
types of sampling techniques
- opportunity sampling
- volunteer (self-selected) sampling
- systematic sampling
- random sampling
- stratified sampling
opportunity sampling
opportunity sampling involves selecting anyone who is willing and available to take part at the time.
for example, approaching people in the street.
this is the most common sampling technique used in psychological research and often results in students being used since most research takes place within universities.
opportunity sampling: advantage
P: quick, convenient and economical
E: means it does not require the level of planning and preparation that many other sampling methods require
E: eg, a study into stress levels during shopping can simply involve a researcher approaching shoppers at a shopping centre rather than having to pre-identify participants
L: leads to less delays in the research process and less money spent
opportunity sampling: disadvantage
P: can be biased and unrepresentative
E: means the ppts that happen to be available at the time of the study may not represent everyone in the target population
E: eg, if a study is conducted in the middle of the working day, the sample may only include people who work reduced hours or the unemployed and not full-time workers. also, researchers may avoid people they don’t like the look of (researcher bias)
L: means the sample may be biased and cannot be generalised to everyone, lowering population validity
volunteer (self-selected) sampling
volunteer sampling involved selecting individuals who have put themselves forward to take part in research
as ppts put themselves forward it is sometimes also referred to as a ‘self-selected’ sample
researchers may place adverts in newspapers or posters or on a university notice board
volunteer sampling: advantage
P: quick, convenient, and economical
E: means it doesn’t require the level of planning and preparation that many other sampling methods require to identify ppts
E: eg, a researcher studying memory can advertise for ppts and the ppts should present themselves
L: there are less delays in the research process and less money spent
volunteer sampling: disadvantage
P: can be biased and unrepresentative
E: volunteers tend to be a certain type of person
E: eg, they tend to be more confident and motivated than most
L: the sample may be biased (volunteer bias) and the findings cannot be generalised to everyone, lowering population validity
systematic sampling
systematic sampling involves selecting every nth member of the target population
this involves obtaining a list of names of everyone in the target population (eg. the school register or database of members) organised in some way (eg alphabetical order) and choosing, for example, every 5th name m
systematic sampling: advantage
P: avoids researcher bias
E: researcher has no influence over who is chosen as it is simply who happens to be in certain positions in a list that are selected
E: eg, picking whoever is in every 5th position on an alphabetical list prevents them from only choosing people they think will help support their hypothesis
L: the research is less biased, more objective and less open to abuse or researcher influence
systematic sampling: disadvantage
P: not guaranteed to be representative
E: every nth name on a list could, by chance, lead to only a certain type of person being selected
E: eg, every nth name could be made even though there are just as many females on the list. furthermore, there is still an element of bias involved as not everyone has an equal chance of being selected as people with names at the start of the register, eg, are unlikely to be selected
L: findings cannot be generalised to everyone, lowering population validity
random sampling
random sampling is where everyone in the target population has an equal chance of being selected
firstly, you must obtain a list of everyone in the target population (eg the school register or a database of members)
secondly, all names are assigned a number
third, the sample is generated through some lottery method (eg equal pieces of paper in a hat or bowl drawn at random or a computer-based randomiser)
random sampling: advantage
P: avoids researcher bias
E: means the researcher has no influence over who is being selected
E: eg, picking names from a hat prevents them from only choosing people they think will help support their hypothesis
L: random samples are less biased, more objective, and less open to abuse or researcher influence
random sampling: disadvantage
P: not guaranteed to be representative
E: means that drawing names randomly from a hat could still, by chance, lead to only a certain type of person being selected
E: eg, every name drawn could be male even though there are just as many females names in the hat
L: the findings cannot be generalised to everyone, lowering population validity
stratified sampling
stratified sampling invilved selecting a sample that reflects the proportions of people in different subgroups according to their frequency within the population
for example, if 40% of the target population are male, then 40% of the sample should be male
- obtain a list of all names in the target population
- researcher identifies the different subgroups that make up the population
- researcher works out the proportions needed to make the sample representative (eg 40% of the population are male)
- ppts from each subgroup are chosen randomly (eg putting the names of all the males in one hand and the females in another and selecting the right proportions from each hat)
stratified sampling: advantage
P: highly representative
E: not only avoids researcher bias but also ensures all subgroups are proportionally represented in the sample
E: picking names from hats prevents the researcher from only choosing people they think will help support their hypothesis but also focus on subgroups means the process isn’t left entirely to chance to represent all types of people
L: it is likely to be more representative than other sampling techniques
stratified sampling: disadvantage
P: time consuming and inconvenient
E: takes a great deal of planning to identify relevant subgroups and count frequencies within each subgroup prior to starting the random selection process
E: eg, none of this level of planning is necessary with opportunity of volunteer samples
L: more delays in the research process and more money spent
experimental design
experimental design refers to the way in which ppts are allocated to the different conditions in an experiment (experimental condition and control condition)
types of experimental design
- repeated measures
- independent groups
- matched pairs
repeated measures design
all ppts take part in all conditions: one group of pots completes one condition and then the same group completes the next condition.
for example, if you were testing the effect of noise on memory, the group of ppts could be tested in noisy conditions (condition A) in the morning and the same group tested in silent conditions (condition B) in the afternoon
repeated measures design: advantages
P: no participant variables
E: no individual differences between the ppts in each condition
E: eg, ppts in condition A will not differ in any way from ppts in condition B bc they are the same ppts
L: increases internal validity of the research
P: requires half the number of ppts as other designs
E: less pots are needed compared to independent groups or matched pairs designs to achieve the same amount of data
E: eg, to achieve 10 ppts in each condition, only 10 ppts are needed in total compared to 20 ppts being needed for other designs
L: stufies using this design do not have to recruit as many ppts which is cheaper and less time-consuming
repeated measures design: disadvantages
P: suffers from order effects
E: the order ppts complete the diff conditions may affect their performance
E: eg, ppts may do better in the second test bc they have alr had practice at the task (practice effect) or may do worse bc they are bored w doing the same task again (boredom effect)
L: lowers internal validity
P: high demand characteristics
E: ppts have more of a chance of guessing the aim of the experiment which may affect their behaviour
E: eg, completing a memory task in noisy conditions then silence, ppts may work out the aim of the experiment and change their behaviour accordingly
L: reduces the internal validity
independant groups design
ppts are places into separate (independent) groups
each group completes one condition of the experiment only
we then compare the performance (dependant variable) of each group
eg, if you were testing the effects of noise on memory you would l have one set of ppts being studied in noisy conditions (condition A) and the other in silent conditions (condition B)
independant groups design: advantages
P: doesn’t suffer from order effects
E: order pots complete the different conditions doenst affect performance bc ppts only complete one condition
E: eg, ppts will not do better (practice effect) or worse (boredom effect) in the second task bc they only complete one task
L: increases internal validity
P: low demand characteristics
E: ppts have less of a chance of guessing the aim of the experiment
E: eg, as ppts only take part in one condition they are not necessarily aware of the research expectations so their behaviour is less likely to change to fit these expectations
L: increases the study’s internal validity
independant groups design: disadvantages
P: low degree of control over participant variables
E: might be individual differences between the ppts in each condition that haven’t been controlled and could unfairly influence the results
E: eg, ppts in CA may be, on average, much more intelligent than those in CB
L: decreases the internal validity and may confound the results
P: requires twice as many ppts as a repeated measure design
E: more ppts are needed w this design to achieve the same amount of data
E: eg, to achieve 10 ppts in each condition, 20 ppts are needed in total compared to only 10 being needed for a repeated measures design
L: studies using this design have to recruit more ppts which is more expensive and more time-consuming
matched pairs design
matched pairs design involves different ppts being used in each condition
however, the ppts are matched into pairs based on important characteristics that may influence the results (eg age, gender)
one of each pair then takes part in CA and one in CB
for example, for every young male in CA there is a paired young male in CB
matched pairs design: advantages
P: doesn’t suffer from order effects
E: order pots complete the different conditions doenst affect performance bc ppts only complete one condition
E: eg, ppts will not do better (practice effect) or worse (boredom effect) in the second task bc they only complete one task
L: increases internal validity
P: low demand characteristics
E: ppts have less of a chance of guessing the aim of the experiment
E: eg, as ppts only take part in one condition they are not necessarily aware of the research expectations so their behaviour is less likely to change to fit these expectations
L: increases the study’s internal validity
matched pairs design: disadvantages
P: less degree of control over participant variables than repeated measures
E: although ppts have been matches on important criteria to control ppt variables more than an independent groups design, there is still more of a chance of individual differences between ppts in each condition than a repeated measures design, which could unfairly influence the results
E: eg, ppts in CA may be, on average, much more intelligent than those in CB
L: decreases the internal validity and may confound the results
P: requires twice as many ppts as a repeated measure design
E: more ppts are needed w this design to achieve the same amount of data
E: eg, to achieve 10 ppts in each condition, 20 ppts are needed in total compared to only 10 being needed for a repeated measures design
L: studies using this design have to recruit more ppts which is more expensive and more time-consuming
ways to overcome issues with experimental design
- random allocation (independent groups)
- counterbalancing (repeated measures)
random allocation
one way of controlling researcher bias in an independent groups design is random allocation of ppts to conditions
this involves assigning all of the pots a number and randomly selecting numbers (random number generator or drawing out of a hat) so that, for example, every other number goes in CA
this should help to reduce issues of bias and help control participant variables
counterbalancing
one way of controlling order effects in a repeated measures design is counterbalancing
this involves half of the ppts starting in CA followed by CB, and the other half starting in CB followed by CA (referred to as ‘ABBA’)
this balances out the problem of order effects as it means it is not always the same condition performing better due to practice or worse due to boredom
what is an ethical issue
ethical issues refer to the conflicts about what is acceptable in research
researchers have a responsibility to protect the rights and best interests of the participants they study by conducting their research in an ethical manner
any research which does not protect the ppts rights in this way may be said to raise ethical issues
types of ethical issue
- deception
- lack of informed consent
- lack of protection from harm
- lack of right to withdraw
- lack of confidentiality
deception
deception is an ethical issue that occurs when ppts are deliberately misled about the true nature of a study
eg, telling them it is a study of one thing when it is actually a study of something else
this is an issue bc informed consent can’t be gained since ppts don’t know what they are truly consenting to
it may cause the ppt to become distrustful of psychology studies in the future
how to deal w deception
debriefing the ppts
this involves telling the ppts the true nature of the research study after it has taken place
this ensures they don’t leave in any confusion and can be given the chance to withdraw their data (retrospective withdrawal)
lack of informed consent
informed consent involves revealing the true aims and purpose of the research before giving them the choice of agreeing (consenting) or refusing to participate in the research
there are many situations where giving informed consent is impossible, for example, when:
- deception is being used
- ppts are unaware that they are part of research such as some field experiments or covert observations
- ppts may be unable to fully understand the nature of the study and make a balanced choice (eg if they are children or have a severe psychological condition)
failure to gain true informed consent may cause ppts to be distrustful of psychology studies in the future
how to deal with lack of informed consent
using ‘presumptive consent’
this is where you tell ppl from the same target population what the study involves and ask if they would have been happy to take part even without informed consent, before the study
if they agree, we can ‘presume’ that the real ppts would too as they should be a representative group
lack of protection from harm
protection from harm refers to a researcher’s responsibility to ensure ppts come to no more harm than they would in every day life
harm can refer to physical and psychological harm
psychological harm might include anxiety, distress or embarrassment
the researchers should aim to ensure that the ppts leave the experiment in the and state as they arrived
how to deal with lack of protection from harm
using an ‘ethical committee’: a group of experts in the area who can decide whether or not the harm in the planned study is too great
if research goes ahead and harm only then becomes clear, ppts have the right to withdraw or the researcher can abandon the study to prevent further harm
lack of right to withdraw
right to withdraw means ppts must be made aware at all points in an investigation that it is their right to leave the study at any time they choose
this may be before the study has started, during the study, or after the study (if they choose to have all of their data removed afterwards as a part of ‘reterospective withdrawal’)
how to deal with lack of right to withdraw
researcher should remind ppts before, during and after the study that they have the right to withdraw from the research by either physically leaving or having their data withdrawn (retrospective withdrawal)
lack of confidentiality
confidentiality means that ppts have the right for their data to remain anonymous
personal details (eg their name) should not be given away during the investigation or afterwards, on publication of the research report
this includes keeping the names of ppts confidential as well as any information which might allow others to identify that individual (eg details of their career, age, or photographs)
how to deal with lack of confidentiality
use pseudonyms (fake names) such as ‘participant A’ when referring to them in the research
forms of giving instructions to participants
- consent forms
- debriefs
- standardised instructions
how to write a consent form
- thank the pot for their involvement
- inform them of the true aims of the study (or false aims of being deceived)
- exactly what they will be doing
- how long it might take
- outline any ethical considerations (eg right to withdraw and right to confidentiality)
- ask them to sign and date the consent form to confirm agreement to participate
how to write a debrief
- thank ppt for participating
- explain they have made a valuable contribution to this area of research
- explain the true aims of the study and if appropriate, an explanation of why it was necessary to mislead them initially
- if there was more than one condition, give details of what ppts in the other condition did
- outline any ethical considerations (eg right to withdraw their data, right to confidentiality, gaining informed consent if they were initially misled)
- ask if they have any questions
how to write standardised instructions
- clear and full details on the task they need to complete
- how long it will take
- outline ethical considerations (eg right to withdraw and right to confidentiality)
- ask if they have any questions
who deals with ethical issues
British Psychological Society
BPS ethical guidelines
BPS code of ethics are the rules or guidelines that all researchers in Britain must follow when designing and conducting psychological studies
- Deception: ppts should be fully debriefed after the study including giving the chance to discuss concerns, ask questions and the right to withdraw their data (retrospective consent)
- Informed consent: where consent is not possible, an alternative should be used (presumptive consent)
- Protection from harm: avoid risk greater than everyday life, allow pots to withdraw if harm becomes too much and researchers should stop the study if the researcher suspects harm
- Right to withdraw: ppts should be informed before, during and after the study of their right to leave or have their data withdrawn (reterospective withdrawal)
- Confidentiality: researchers should not reveal aspects of ppts identity when discussing and reporting in their research
limitations of using deception/debriefing
- after the debrief ppts may choose to have their data withdrawn which could skew the results
- debrief may be too late if there has already been deception and psychological harm (eg feeling embarrassed, etc)
limitations of using presumptive consent
- individual differences: may not generalise
- informed consent cannot always be offered so deception may be needed
limitations of guidelines on protection from harm
- harm caused may be permanent/long-term
- psychological harm may not always be obvious compared to physical harm
limitations of ethical guidelines on right to withdraw
- researcher may avoid reminding them in order to continue w their study
- ppts may feel obliged and pressured to take part as though they have not choice but to participate
- if ppts do withdraw it could skew the results
limitations of ethical guidelines on right to withdraw confidentiality
- even by using pseudonyms, findings can still make the ppts identifiable
- some personal details may be relevant to the study (eg age) and need to be disclosed
failure to comply with the BPS code of ethics
failure to comply with the BPS code of ethics, whilst not resulting in prosecution, could restrict future research and result in the removal of the name of the ‘offending’ psychologist from the professional register of chartered psychologists
observational sampling
selecting which behaviours to observe and record
types of observational sampling
event sampling
time sampling
event sampling
continuously watching a certain behaviour (or event) and counting the number of times that event occurs in a target individual or group
ensures no behaviours are kissers and generates a great deal is data for analysis
eg, event sampling of socialisation in children may involve counting the number of times they talk to another child during playtime
event sampling: strengths
P: doesn’t overlook important behaviour
E: this is bc behaviour is watched and recorded during the entirety of event so no behaviours are missed
E: eg, event sampling of socialisation in children may involve counting the number of times they talk to another child during playtime
L: is more accurate, detailed and generates a great deal of data for analysis
event sampling: limitations
P: may suffer from observer bias
E: observers expectations may influence what the researcher sees, hears or the data that they record
E: eg, eg, expecting boys to be more aggressive than girls may lead a researcher to spend more time looking for aggression in boys and be more likely to interpret their behaviour as aggressive even if that isn’t the case
L: this makes the observation less objective and reduces the internal validity of the research findings
time sampling
watching and recording behaviour at specific time intervals (as opposed to continuously recording)
for instance, recording what the target individual or group are doing every 30 seconds or every minute
eg, observing the behaviour of students in a library may involve making a ‘sweep’ of the library every 30 minutes to record specific behaviours (eg how many are using a computer or books)
time sampling: strengths
P: reduces observer bias
E: observers expectations are less likely to influence what the researcher sees, hears or the data that they record because they are only observing for limited intervals
E: eg, even if an observer expects boys to be more aggressive than girls they are less able to spend more time looking for aggression in boys and less likely to interpret their behaviour as aggressive even if that isn’t the case
L: this makes the observation more objective and increase the internal validity of the research findings
time sampling: limitations
P: may overlook important behaviour
E: this is bc behaviour is only watched and recorded for specific time intervals so some behaviours may be missed
E: eg, time sampling of socialisation in children may involve counting the number of times they are talking to another child every 10 minutes, rather than for the entirety of playtime
L: less accurate and detailed and generates less data for analysis compared to event sampling
when might event sampling be useful compared to when time sampling may be useful
event sampling is useful when a target behaviour or event is quite infrequent and could be missed by time sampling whereas time sampling may be useful when a target behaviour id event is quite complex and data needs reducing
time sampling may also be better used in participant observations to reduce the observer bias
how are behaviours recorded in observations
behavioural categories (behavioural checklist)
behavioural categories
dividing a target behaviour (such as stress, aggression or affection) ingo a subset of specific, observable behaviours.
eg, ‘aggression’ may be broken down into ‘punching’ ‘kicking’ ‘shouting’
these categories must be clearly operationalised (clearly defined and made measurable) and must cover all the possible ways the behaviour may occur
a pilot study may be used to ensure all behaviours are covered and clearly defined
A word on ecological validity
the concept of ecological validity is often misunderstood because students think it is about the ‘naturalness’ of the study where a more natural setting automatically means the study must have high ecological validity. however, this isn’t quite true.
the key feature to consider is the dependant variable (DV) and how real-to-life or artificial this is.
for example, Rutter et al’s study of Romanian orphans often seems quite high in ecological validity because the IV (whether the children were adopted before or after 6 months of age) was not controlled by the researchers and was real life. however, one DV in this study was intellectual development which was assessed using IQ tests. such tests are conducted in a controlled environment and may be quite ‘artificial’ measures of intelligence.
equally, Godden and Baddeley’s study of context-dependent forgetting using deep-sea divers who had to learn a list of words either in land or underwater might seem high in ecological validity as it was not conducted in a laboratory and was instead conducted ‘outside’ in the ‘real-world’. however, as the DV involved learning and recalling word lists, this was an artificial way to test memory and therefore may not represent real-life memory.
A word on temporal validity
temporal validity is the issue of whether findings from a particular study, or concepts weighing a particular theory, hold true over time.
for example, critics have suggested that high rates of conformity within the original Asch experiments were a product of a particularly conformist era in American history (as demonstrated when Perrin and Spencer failed to replicate these effects in Britain 30 years later)
ways of assessing validity
- face validity
- concurrent validity
explain face validity
a basic form of validity which refers to whether a test or measurement looks like it is measuring what the researcher intended to measure
in other words, assessing whether test items appear ‘on the face of it’ to measure what the test claims to measure
this can be done by simply ‘eyeballing’ the measuring instrument or by passing it to an expert to check
for example, you could assess a stress questionnaire for face validity by checking that all the questions look like they are obviously related to stress
explain concurrent validity
the extent to which a test or measurement produces similar findings to a recognised and more established test of the same topic
in other words, assessing whether the current measurement produces results that are similar (or equivalent) to a previously validated measurement of the same variable(s)
this can be done by giving ppts both measures (eg. questionnaires) at the same time and then comparing their scores
for example, you could assess a new IQ test for concurrent validity by comparing the results from that threat with the results the same ppts got on an older, more established IQ test to see if the two scores correlate strongly (i.e. are similar)
improving validity: poor face validity
if any questions on a test or questionnaire are judged as having poor face validity (i.e. they don’t look like they are measuring what they are intending to measure) then the questions should be removed, revisited, and/or rewritten so they relate more clearly and obviously to the topic being measured
improving validity: low concurrent validity
if any questions on a test or questionnaire have low concurrent validity (i.e. scores or responses do not correlate with those on the more established test of the same topic) then the questions should be removed, revisited, and/or rewritten so they can be tested or checked for concurrent validity again
improving validity: low internal/external validity
improvements should come from changing the methods, techniques and designs used to conduct the research
improving validity: participant variables
- use a stratified sample to represent the target population proportionally -> high population validity
- use a matched pairs design
improving validity: a field experiment affected by environmental variables
- change to a laboratory experiment/setting -> control environmental variables -> reduce confounding variables
improving validity: investigator effects
- standardised questions and instructions, can use a script or pre-recording so it’s the same for all ppts
- use a single or double blind trial
improving validity: demand characteristics
- use lack of informed consent so that ppts don’t know they are being studied
- use deception (give a false aim)
improving validity: order effects
use counterbalancing (ABBA)
improving validity: a laboratory experiment that lacks ecological validity
change to a field experiment in a natural/real-life setting
improving validity: sample lacks population validity
make the sample more varied (random or stratified sampling)
improving validity: low temporal validity
replicate the research to see if the results are still the same and update the theory
ways of assessing reliability
- test-retest reliability
- inter-observer reliability
- inter-eater reliability
explain test-retest reliability
the extent to which the same test or measurement, given to the same ppts on two separate occasions produces the same results
in other words, the same ppts are tested and then retested a second time in the same way to see if their scores correlate strongly
usually there is a short interval between tests, such as a week or two, so that ppts don’t remember their answers but also this interval should not be so long that their opinions or abilities may have changed
for example, giving a set of ppts an IQ test and then repeating the same IQ test with the same ppts two weeks later before comparing the scores from the two occasions to see if they are correlates
as well as tests and questionnaires, this assessment can also be used to check the reliability of interviews
explain inter-observer reliability
the extent to which there is agreement between two (or more) observers involved in observations of a behaviour
in other words, it refers to whether different observers watching the same event independently collect scores that correlate strongly
for example, two observers might independently count the number of aggressive acts shown by a group of schoolchildren in a playground using the same behavioural categories before comparing results to see if they are consistent (i.e. whether they both counted 4 punches and 2 kicks)
when the scores from both observers are correlated, a correlation co-efficient can be calculated
a result of 0.8 or more suggests high inter-observer reliability
explain inter-rater reliability
The level of agreement in scores on a measure between different raters or observers rating the same target.
Two researchers can both do the same content analysis using the same categories before comparing results
High interrater reliability suggests the ratings are objective and not overly influenced by rater subjectivity or bias.
improving reliability: low inter-observer reliability
(1) it may be that the behavioural categories were not operationalised clearly enough.
for example, if one observer interpreted an action as a ‘punch’ whilst one interpreted the same behaviour as a ‘push’
these categories can be removed, revised, and/or rewritten to ensure greater precision and clarity and less overlap between categories before running the test again
(2) it may be that some it may be that some observers just need more practice using the behavioural categories so they can respond more quickly
improving reliability: low test-retest reliability
(1) it may be that test items or questions were ambiguous (not clear enough) resulting in ppts giving different answers each time they respond
for example, the question ‘what are your thoughts about dieting?’ night leas some ppts to interpret this as asking for factual information, and provide the facts they know about dieting, whereas others might think the question was more about emotions and respond with their own feelings about their experiences with dieting
these test items or questions need to be removed, revised, and/or rewritten to ensure greater precision and clarity before running the test again
(2) it may be that the test items or questions were overly complex or too broad, covering too many issues
to improve reliability, questions may be revised and rewritten to simplify them
furthermore, open questions (where there is more room for (mis)interpretation) can be replaced by closed, fixed choice alternatives
(3) it may be that the test is conducted slightly differently each time
for example, if it is conducted by different researchers or in different conditions
to improve this, researchers must work hard to ensure all aspects of a study are standardised (eg. using the same researcher and ensuring all aspects of the environment and procedure are kept the same each time a test is run)
using correlations to assess validity and reliability:
strong positive correlation
the data represents that the co-variables have been accurately tested and the findings are consistent
we can accurately and consistently state that as one co-variable increases, so to does the other co-variable
using correlations to assess validity and reliability:
strong negative correlation
the data represents that the co-variables have been accurately tested and the findings are consistent
we can accurately and consistently state that as one co-variable increases, the other co-variable decreases
using correlations to assess validity and reliability:
zero correlation
the findings were not consistent or accurate as there is no relationship between co-variables
OR
we can accurately and consistently state that there is no relationship between co-variables
