Research Methods part 2 - Key terms Flashcards
Reliability
The extent to which the findings or research are consistent and replicable.
Test re-test reliability
The extent to which the same results will be achieved everytime the test is carried out (Applies to all research methods).
Experiments - if they have standardised instructions, procedures & controls we can replicate them and should get the same results each time.
Questionnaires – if we gave a questionnaire out again, we would expect a ppt to give the same answers again so, we would consider it reliable.
Inter-rater reliability
The extent to which the way data is interpreted in a consistent way. This applies when data has been collected by self-report and requires interpretation.
eg, a researcher who is using a questionnaire or interview with open questions may find that the same answers could be interpreted in different ways, producing low reliability. If these differences arose between different researchers, this would be an inter-rater reliability problem.
How to achieve inter-rater reliability
Best ways to increase inter-rater reliability:
* Clear and objective methods of collecting data
* Making rating scales as objective as possible to ensure that all researchers are using the same criteria to assess the behaviour and applying the scales in the same way.
Intra-rater reliability
whether one researcher is consistent overtime.
Inter-observer reliabilty
The extent to which the way data is interpreted in a consistent way.
* Applies when data has been collected by observation and requires interpretation.
eg. in an observation, researchers gave different interpretations of the same actions, this would be **low inter-observer reliability. **
How to achieve inter-observer reliability
- Have more 2 or more observeres
- Train them to carry out the observation in the same way to ensure that behavioural categories are clearly operationalised.
- Observe the same behaviour
- Compare results
- If there is 0.8 correlation between observers results (high similarity) then it’s considered a reliable observation.
Validity
The extent to which the findings of a study are ‘true’ and ‘accurate’.
Internal validity
Are we measuring what we set out the measure?
* If there are EVs, then validity is lowered - because we are no longer testing the effect of the IV on the DV.
External validity
Can we accurately generalise? Are the results true beyond the study?
Face validity
Is a subjective assessment of whether or not a test appears to measure the behaviour it claims to.
eg. a study measuring aggression amongst teenagers, measures the number of times they swear at each other (swearing is not always linked to aggression, especially in teenagers)
Content validity
Is an objective assessment of the items in a test to establish whether or not they all relate to and measure the behaviour in question. Experts assess whether the test is measuring what it set our to do.
eg. A driving test that assesses both theoretical knowledge of traffic rules and practical driving skills: This ensures the test covers all necessary components to be a safe driver.
Concurrent validity
Is a comparison between two tests of a particular behaviour. One test has already been established as a valid measure of the behaviour, and the other test is the new one. If the results from both significantly correlate, then the new test is valid.
eg. Comparing a new IQ test to a well established IQ test. The ppt should score similar on both to show that the new one is a valid measure.
Predictive validity
Refers to how well a test predicts future behaviour. eg. do your mock exams predict your actual results? Are they vaild?
External validity
Population validity
How well can we generalise from the sample to the population we want it to represent. Is the behaviour true for others? Is our sample representative?
External validity
Ecological validity
How well can we generalise from the study to how people behave in everyday life. Is the behaviour true in other settings? Does the study environment or task represent real life?
External validity
Temporal validity
Would the findings of an older study still be true today, is the ppts behaviour in the study still generalisable to today’s society?
Extraneous variable
A variable which could affect the DV and therefore should be identified and controlled before you begin the research.
Cofounding Variable
A variable that has affected the DV and so would be mentioned after the study has been conducted.
Control variables
- This is where psychologists keep everything the same (except the IV) across the different conditions and for all ppts.
- Ensures that the IV has affected the DV and nothing else.
- Ensures that the research can be consistently replicated in the same way for all ppts.
- Increases validity and reliability of the research
Extraneous Variables
Participant variables
Demand characteristics: ppt guesses the aim of the study, changes their behaviour to please the researcher
“Screw you” effect: ppt guesses the aim of the study, changes their behaviour to sabotage the research
Individual differences: Differences between ppts, such as personality, IQ, mental health etc which could affect the DV.
Extraneous Variables
Situational variables
Factors in the environment that can unintentionally affect the results of a study.
Eg.
Noise
Temperature
Distractions
Extraneous variables
Experimenter variables
Experimenter effects: elements of the experimenter’s appearance or behaviour affects the behaviour of ppts
Experimenter bias: the experimenter interprets behaviour or data in a way that is biased to support a particular outcome.
* Or the experimenter selects ppts who are more likely to behave in a way that will support a certain outcome.
Types of Experimenter bias
**Non-verbal communication: **Researcher can communicate their feelings about what they are observing. Can also encourage ppts behaviour to support their own hypothesis, sometimes without even realizing that they are doing this.
Physical characteristics: The appearance of the researcher will influence the behavioral response of the participants. eg. gender, outfit
Bias in collection/interpretation: collecting and interpreting the data in a biased way.
Bias in selection of ppts: could select ppts who they know would be more likely to behave in a specific way or who may be easier to work with.
Extraneous variables
Task variables
Aspects of the instructions, materials and the design are not kept constant across the different conditions.
* Or the way the tasks have been set up affect the results eg. order of tasks
Techniques to control extraneous variables
Standardised instructions/procedure: Carrying out the experiment/study in the same way with the same instructions for all ppts = reduces situational & task variables
Counterbalancing: Varying the order that the ppts do the tasks or the conditions of the experiment (ABBA) = reduces order & practice effects
Experimental design: Repeated measures design can reduce ppt variables
Independent measures & matched pairs can reduce task variables
Single blind: Ppt doesn’t know what condition they are in > reduces demand characteristics.
Double blind: Ppt and researcher does not know what condition they are in > reduces demand characteristics and experimenter bias
Pilot Studies: A study on a small scale performed prior to the main study (pre-study). > This reduces task variables
Mundane Realism
The extent to which the task represents a real-world situation. eg. Would we behave this way in everyday life.
Similar to ecological validity. If the experiment lacks mundane realism, then it will be low in ecological validity.
Low mundane realism = task is artificial
Ecological validity = can’t generalise from the findings.
Participant Observation
The observer participates in the behaviour they are observing (but can be overt or covert)
Non-participant observation
The observer does not take part in the behaviour they are observing (but can be overt or covert)
Structured observation
Uses behavioural categories and a coding frame to collect data (event sampling/time sampling)
Unstructured observation
Collects all data that is relevant with no pre-planned behavioural categories.
Controlled observation
Participants are given a task and then observed doing the task
Naturalistic observation
participants are observed behaving naturally with no manipulation from researchers
Covert observation
Under-cover > participants don’t know that they are being observed
Overt observation
Participants know they are being observed.
Data collection techniques for controlled/structured observations
Main sampling methods
Event sampling - observer decides in advance what types of events is of interest and records all occurrences. All other types of behaviour are ignored.
Time sampling - observer decides in advance that observation will take place only during specific time periods and records the occurrence of the specified behaviour that period only. eg. every hour, 1 hour per day
Observations
Evaluation of event sampling
Strengths - Behaviour won’t be missed as every time specific behaviours occur, they are recorded.
Weaknesses - If too many observations happen at once, it may be difficult to record everything.
Observations
Evaluation of time sampling
Strengths - The observer has time to record what they have seen as they aren’t observing for the whole time they are present.
Weaknesses - some behaviours will be missed if they are outside the designated time intervals. Therefore, observations may not be representative.
Correlations
Shows the relationship between two or more variables. It cannot show a cause-effect relationship. There may be an intervening variable.
Difference between correlations and experiments
- In an experiment we are looking for a difference in behaviour across the different conditions – so it is a test of difference.
- Whereas in a correlation we are looking for relationships between variables.
- There will be no IV and DV (just co-variables: variable 1 and variable 2)
- There will be no experimental design in correlations
- The hypothesis for correlations will not state that the IV will/will not affect the DV – just that there will be a relationship between the two variables
- Data will be shown on a** scatter graph** rather than a bar chart.
- **Quantitative data **is required for a correlation. Participants will need a score on both the variables you are investigating the relationship between.
Positive correlation
As the value of variable 1 increases so does variable 2 (although not necessarily at the same rate) Both variables go in the same direction. So they can also both be decreasing. E.g. As ice cream sales increase, so do burglaries.
Negative correlation
As the value of variable 1 increases the value of variable 2 decrease. eg. The more school you skip the worse your grades will be.
How to write Correlation Hypothesis
- One-tailed (directional) > “there is a positive relationship/correlation between age and beauty”
- Two-tailed (non-directional) > “there is a relationship/correlation between age and beauty”
- Null hypothesis > “there is no relationship/correlation between age and beauty”
Strengths of correlations
- Can test relationships between naturally occurring variables that would be difficult or unethical to measure experimentally.
- Good way of indicating trends which may lead to further research being carried out
- Can establish a cause-effect relationship
- Useful to establish the reliability of a questionnaire (split-halves method), an observation (inter-rater reliability) or an experiment (test-retest reliability)
Weaknesses of correlations
- Because the variables in a correlation are naturally occurring, the psychologist may find it difficult to control extraneous variables.
- Correlation studies cannot produce cause-effect relationships – which means we may be unsure which variable was the cause and which was the effect or if the effect was caused by an intervening variable.
