Research Methods (cont) Flashcards
content analysis, assessing and improving reliability and validity, probability and significance, features of science, descriptive statistics and display of quantitative data
content analysis
way of analysing and transforming qualitative data into quantitative data
secondary source content e.g. adverts, films, diaries, transcribed verbal communication
ways of categorising data
(content analysis)
top-down
- pre-defined categories before research
bottom-up
- allow categories to emerge from content
- read/watch first, then again
- provides more detail, won’t miss important themes
quantitative analysis
(content analysis)
create coding system and tally each time a behavioural category occurs
should be pre-defined and clearly operationalised
- less subjective, limits misinterpretation
statistical analysis can then be carried out
makes more scientific, reliable, valid
quantitative analysis process
(content analysis)
data collected
examines data to familiarise (bottom-up)
identify coding units
data analysed by applying coding units
tally of each tome coding unit appears
qualitative (thematic) analysis
(content analysis)
familiarise with data
generate initial codes
search for initial emerging themes (lots of different codes to sort in to themes)
review themes, collapse into each other, cross over
define and name
write up
content analysis strengths
reliable
easily replicated, standardisation of coding units and pre-existing secondary data
coded more than once (intra-rater reliability) or by different researchers (inter-rater reliability)
can check for consistency
but, subjectivity may affect findings, define codes differently, decreasing consistency
content analysis strengths
ethical
already in public domain (no privacy issues)
does not involve direct use of participants
less ethical guidelines that may restrict research
but, still some ethical considerations
researcher needs to ensure they have consent of stakeholders to analyse confidential material e.g. medical records
can be difficult - make some topics difficult to investigate
content analysis weaknesses
prone to subjective analysis
involves interpreting qualitative data from secondary sources alongside a coding system
affected by gender or cultural background of researcher
prone to researcher bias
mean
interval / ratio data
adding all scores and dividing by number of scores
if fairly even distribution around centre
mean strengths
accurate and sensitive measure
takes all numbers into consideration
highly representative
is numerical centre point of actual values
used to calculate standard deviation
mean weaknesses
can be skewed by anomalies
rogue scores can significantly increase or decrease mean score
not representative
not always an actual score (e.g. 2.5 children)
not accurate reflection of data set
median
ordinal data (or higher)
middle score when data in ordered list, middle scores averaged
when extreme high or low scores
median weaknesses
may not be an actual score
not representative
not appropriate in small data sets or when there are large differences
median strengths
unaffected by extreme scores
only concerned with middle
more accurate and representative
quick and easy to calculate
mode
nominal data
most common score
can be bi-modal or multi-modal if multiple common scores
least meaningful, especially when multiple modes
mode strengths
unaffected by extreme scores
more representative
always an actual score
accurate representation
mode weaknesses
sometimes doesn’t have a mode or has many
limited usefulness
doesn’t use all data
accuracy questioned
measures of central tendency
how close scores are to average
mean
median
mode
measures of dispersion
how spread out scores are
provide fuller picture
range
standard deviation
analyse how far away scores are from average responses e.g. spread or variability
normally large dispersion is due to individual differences or poor experimental control
range
ordinal data
difference between highest and lowest score
subtract lowest from highest score
range strengths
easy and simple to calculate
takes into account extreme values
range weaknesses
ignores most of data
doesn’t reflect true distribution
easily distorted by extreme values
standard deviation strengths
precise as all values accounted for
accurate representative of distribution
detailed conclusions made
allows for interpretation of individual scores in terms of how it falls from the mean (130 IQ = 2 SD away from the mean)
standard deviation
interval / ratio data
indicates average distances of scores around the mean
takes every score into account
larger SD, more spread out relative to the mean
measures collectively how much individual scores deviate from the mean, presenting this as a single number –> how much data is dispersed
standard deviation weaknesses
complex to calculate
not quick or easy
less meaningful if not normally distributed
standard deviation comments
should comment on the spread
large spread suggests inconsistencies in data, highlighting individual differences
larger SD, more spread out, more variability
smaller SD, more similar the scores
positive, negative and zero correlations
correlations are co-efficients between -1 and 1
closer the figure to 0, the weaker the correlation
closer to 1 or -1, very closely related variables
positive = both co-variables increase together
negative = one increases while the other decreases
zero = no apparent relationship
moderate (0.5), strong (0.7), weak (0.3)
probability
likelihood of an event occurring
expressed as decimal or percentage
inferential statistics necessary to determine whether results are significant or simply due to chance
- show which hypothesis to accept or reject
- use probability of p <= 0.05
likelihood of the data (in terms of difference or relationship found) being due to random chance is less than or equal to 5%.
less than or equal to 5% chance of the null hypothesis being true
type I error
false positive / error of optimism
claims there is a significant difference when there isn’t
claims support for research hypothesis with significant result when caused by random variables and not really significant
level of significance not cautious enough e.g. p<= 0.10
type II error
false negative / error of pessimism
accepts null hypothesis, claiming there is no significance when there is an effect beyond chance
level of significant too stringent e.g. p<=0.01
- used in medical or safety critical situation
scientific process
Psychology follows the scientific method – states a theory and uses observations and experiments to test the hypothesis through laboratory or field experiments. The understanding from research is applied to create evidence-based strategies that solve problems and improve lives.
- Observation / question
- Research
- Hypothesis
- Experiment
- Collect data
- Analysis
- Conclusion
objectivity
- When study is bias free of experimenter and there are operationalised definitions of behaviour being used. Also refers to the validity of a measure.
- Concepts psychologists measure are not always easily measurable e.g. cognitive processes and attachment, obedience, anxiety can only be interpreted.
experimenter bias
Bias in recruitment or allocation of participants
Investigator effects – cues that influence how participants behave
Confirmation bias – selectively attending to factors confirming hypothesis.
Interpretation of results subject to bias – especially with qualitative data.
improving objectivity
Gather objective, quantitative data
Double blind procedure
Operationalise definitions and use well-defined and clear behavioural categories
Standardised procedures and instructions with consistent measures (e.g. videos)
Controls
Representative sample randomly selected
Interrater reliability checks
empiricism
- When information is gained through direct observation rather than argument or belief. We should be able to operationalise IV and DV and directly observe them.
- Increases reliability and validity of data, increasing levels of certainty and confidence in conclusions.
empiricism in the humanistic approach strengths
- Lower levels of Maslow’s hierarchy, as physiological needs are likely to be empirical, as we can see and measure them quite easily.
It becomes harder to measure the higher up the hierarchy we go. - Success of therapy using card sort.
Way of quantifying and operationalising concepts like self-image and ideal self.
Participants place cards with characteristics written on them on a nine-point continuum.
On the initial sort, they had to place them according to how they are at that moment, to measure their true self.
Cards then redistributed according to their ideal self.
Allows for the gap between ideal self and true self to be measured.
empiricism in the humanistic approach weaknesses
- Congruence and self-actualisation are abstract concepts.
Cannot see or measure it.
Difficult to operationalise. - Success of therapy
Subjective experience opposed to empirical research evidence.
empiricism in the biological approach strengths
- Localisation of function and brain lateralisation
Can observe images of the brain and physically see structures and activity levels without having to make inferences.
Processes can be seen and measured quantitatively. - Role of hormones
Can be measured in an objective and quantifiable way.
empiricism in the biological approach weaknesses
- Neurotransmitter levels in the synapse
Measurement not precise enough
empiricism in the learning approach strengths
- Pavlov measured drops of salivation and Skinner measured behaviour of pressing the lever
Quantifiable and can be easily measured and observed
Behavioural responses can be seen - Bandura measured the number of aggressive or non-aggressive imitative or non-imitative acts.
Behaviour is observable
No inferences need to be made in terms of learning of aggressive responses
empiricism in the learning approach weaknesses
- Mediational processes cannot be seen
Must be inferred
Inner mental processes cannot be studied empirically
empiricism in the cognitive approach strengths
- Localisation of function, emergence of cognitive neuroscience
Mental processes now able to be observed via imaging
E.g. PET scans show brain activity / where cognition is occurring.
fMRIs measure blood flow so show indirect neural activity / processing.
Produces images and waves that can be quantified.
empiricism in the cognitive approach weaknesses
- To investigate mental processes, some inference is involved
As mental processes cannot be observed
Unscientific and not empirical
empiricism in the psychodynamic approach strengths
- Could argue that consequences of fixation are observable and measurable
empiricism in the psychodynamic approach weaknesses
Although impossible to establish whether fixation during psychosexual stages is what causes this
* Id, ego and superego, psychosexual stages and the unconscious cannot be observed.
Have to make inferences.
replicability
- Opportunity to repeat an investigation under the same conditions in order to check and verify scientific information by finding consistent results.
- To draw conclusions, the procedures and findings should be repeatable. Unrepeatable results may imply flaws or lack of control within the method used and are of limited use in theory construction.
- Replication increases our confidence in results, suggesting that data is more likely to be valid if findings are consistent.
Reduces the impact of anomalous results. - Strengthens credibility of a theory through repeated attempts to refute and falsify.
- Reduces the likelihood of fraud.
Standardised procedures allow other investigators to repeat the experiment to eliminate investigator bias and confirmation bias, resulting in more valid and trustworthy data obtained. - Can also improve generalisability (experimented with different groups of people) and temporal validity (experimented in different eras) of findings.
replicability of research methods (high to low)
Lab studies
Technical equipment like a brain scan
Questionnaires
Observations
Field studies
Unstructured interviews
Case studies
falsifiability
- Popper argued that scientific theories must risk being proved wrong.
- What separates a science from a non-science is not the ability to verify it but the ability to falsify the theory (to be able to prove it wrong).
- For psychology to be scientific, we have to construct testable hypotheses (deductive method) which have the potential to be falsified.
- The inductive method does not yield certainty – no matter how many observations are made that confirm a theory, there is always a possibility that a future observation could refute it.
- Increases confidence in results – increases validity. Theory strengthened through repeated attempts at falsification. Reliability of findings supported. Generalisability increased.
falsifiability in the humanistic approach
- Hypothetical concepts (self-actualisation and congruence) are not directly observable. Inference means falsifiability is not possible.
falsifiability in the biological approach
- Can observe localisation of function / hemispheric lateralisation
Damage can be seen on a scan.
Behavioural changes can be observed. - Can directly manipulate and observe testosterone levels in rats
falsifiability in the learning approach
- Could disprove classical conditioning if Little Albert did not become scared of the rat when it was paired with the loud noises.
Fear response is observable.
Pavlov measured salivation in a quantifiable and measurable way - Operant conditioning can be disproved if Skinner’s rat did not change behaviour after changing stimulus
falsifiability in the cognitive approach
- Experimentation and hypothesis testing in lab based research can be falsified.
- Role of brain areas can be falsified due to use of brain scans.
- Case study evidence that cannot be replicated.
- Mental processes need to be inferred so cannot be falsified.
hypothetico-deductive model
- Should follow the scientific method and formulate a hypothesis, operationalise variables and devise a procedure to test this hypothesis.
- Must be possible to prove the theory incorrect (falsifiability).
- If study yields data that confirms the hypothesis, would confirm the initial idea or existing theory can be developed.
- If hypothesis not met, would amend of reject the theory and look for alternative explanations.
Loftus’ research into leading questions (had theory that memory was unreliable and designed an experiment to test it)
Freud’s case study of Little Hans (case study where observations helped develop a theory)
hypothesis testing
- Allows replication to test for reliability.
Variables operationalised, controlled conditions
If consistent findings produced, establishes validity - Ensures research is falsifiable
Possible to test hypothesis and support the null or alternative hypothesis and confirm or deny theory - Allows for laws and scientific principles to be generated through testing of assumptions and new information.
deductive method
theory, hypothesis, observation, confirmation
inductive method
observation, pattern, hypothesis, theory
Scientific method established through inductive method
Theory tested in experimental and falsifiable wat
Start with observations to formulate general hypotheses and test through experimentation.
Ainsworth’s theory of attachment
Work in Uganda provided observation where she identified different general patterns of attachment, then generated a hypothesis and experiment, then concluded a theory.
Strange situation follows deductive method as she had a theory and used a hypothesis and experiment to measure this, and observations helped reinforce her theory.
Freud’s theory of psychosexual development
Based on case study evidence and not empirical research.
Observations helped him create a theory with no hypothesis testing or operationalised variables.
Loftus and Palmer’s research into reconstructive memory
Loftus suggested that eyewitness testimony is unreliable.
Created experiments to support her theory.
Used lab experiment with standardised procedure, questionnaire to limit investigator effects, video clips
role of serotonin
Deductive method – after falsified, hypotheses tested again to develop new drugs
paradigms
- Kuhn suggests that to be a science, there needed to be a shared set of assumptions that everyone within that discipline agrees on.
- Scientific research and thought defined by paradigms that consist of formal theories, experiments and trusted methods. Scientists typically accept a prevailing paradigm and extend its scope by refining theories, explaining data and establishing more precise measures.
- Psychology has many approaches with own sets of assumptions and do not agree on one way of thinking (unconscious mind, learning through environment, genetics)
paradigm shifts
- When a scientific community moves from one established way of explaining / studying human behaviour to another
- Must be an improvement and require total change in way subject is viewed
- Introspection (Wundt) studying behaviour (Skinner)
Used empirical methods that are more scientific
Large-scale and highly controlled studies - Cognitive revolution and current emphasis on cognitive neuroscience
Changes how behaviour and internal mental processes are studied
New technology like scanning methods
development of a science
- Pre-science (no generally accepted paradigm)
Stage psychology is at due to lack of shared paradigms and universal laws of human behaviour.
Can be changed due to existence of new technology. - Normal science (all scientists working from the same paradigm)
- Revolutionary science (new model introduced that causes a paradynamic shift)
internal reliability
consistency of the measurement
external reliability
consistency of results over time and extent to which an measure varies from one use to another
assessing internal reliability: split half method
- Internal consistency of a test e.g. psychometric tests and questionnaires.
- Extent to which all parts of the test contribute equally to what is being measured.
Split a test into two halves e.g. one half may be even-numbered questions and the other odd-numbered questions.
Administer each half to the same individuals.
Repeat for a large group of individuals.
Plot of a graph and find correlation between scores of both halves.
Higher the correlation between two halves, the higher the internal consistency of test or survey. It indicates that all parts of the test are contributing equally to what is being measured.
Strong positive correlation of 0.8 indicates good internal validity. - Technique works best when there are equal number of questions within a ‘test’ but also when the questions on the test measure the same construct or knowledge area.
assessing external reliability: test re-test
- Method assesses external consistency of a test.
- Measures stability of a test over time.
Evaluates temporal validity and whether results are still relevant today.
Results may have been affected by current cultural, social, and political norms and not relevant today. - Typical assessment would involve giving participants the same test on two separate occasions. If the same or similar results are obtained, then external reliability is established. Results from both occasions would be compared and correlated.
assessing external reliability: inter-rater / inter-rater observer
- Degree to which different raters give consistent estimates of the same behaviour.
- Single event is measured simultaneously and independently by two or more trained individuals. If data is similar, then it has external reliability.
If two researchers are observing aggressive behaviour of children, they would both have their subjective opinion regarding what aggression comprises. It would be unlikely that they would record behaviour the same and the data would be unreliable.
If they were to operationalise the behaviour category of aggression this would be more objective and make it easier to identify when a specific behaviour occurs.
‘Pushing’ is objective and operationalised, so researchers could count how many times children push each other during a certain duration and then compare results.
improving reliability
- If agreement not found in terms of internal and external reliability, researcher will seek to improve the reliability of their test.
improving reliability of procedure
Use scripts / recorded instructions / standardised procedures detailing what researchers should say to limit investigator effects.
Environmental control – specify and carry out in the same environment using the same stimulus materials and control extraneous variables.
More ethical experiments lead to greater credibility.
improving reliability of observational research
Use the same researcher so investigator effects are equalised over all replications – likely to be carried out in the same way.
Clearly operationalise behavioural categories so it is objective what researchers are looking for
Use the same equipment
improving reliability of questionnaires
Makes all questions weightings the same (same difficulty and importance)
Use quantitative closed questions to limit subjectivity of interpretation.
improving reliability of interviews
Structured interviews allow for less interpretation, leaving more freedom for the research may cause information and cues in each interview to differ – may not get the full picture in some interviews, so comparison is difficult.
Use same equipment so cues given off are the same
Use same interviewer, so mood, demeanour of researcher does not differ, so responses not influenced differently.
validity
accuracy of results and whether they accurately reflect natural behaviour
external validity
- Refers to the extent to which you can generalise the findings of a study to other situations, people, settings and measures.
ecological validity
the extent to which the findings of a study can be generalised to alternative / real-life environments.
Could carry out research in a natural environment that uses a tasks that accurately reflects real life behaviour.
E.g. instead of testing participants using artificial word lists in a laboratory environment, they could use a more natural environment such as a classroom, where it is not out of the ordinary to learn and recall word lists. This would help produce more natural behaviour.
population validity
the extent to which the sample used in the study is representative of the target population.
Should carry out research using a sample that is representative of the target population.
temporal validity
– the extent to which the findings of a study can be generalised to other time periods. Questioning whether findings would be the same if researchers conducted the study in modern society.
Could repeat the experiment today to test if results remain consistent.
internal validity
- Measure of whether results obtained are solely affected by changes in the variable being manipulated (i.e. by the independent variable) in a cause-and-effect relationship.
- Is the study measuring what it claims to be measuring?
- Affected by individual differences, investigator effects, demand characteristics, the Hawthorne effect, extraneous variables in the environment.
Uncontrolled extraneous variables make cause and effect relationships less likely to be established.
assessing internal validity: concurrent validity
- Type of criterion validity (looking at how a test relates to other measures of the same concept) which measures how well a new test compares to a well-established test.
- Demonstrated when a test correlates well with a measure that has previously been validated.
When measuring Authoritarian personality, carry out a pilot study with participants who fill in both the f scale and a new questionnaire.
assessing internal validity: face validity
- Whether a test appears to measure what it’s supposed to measure.
Concerned with whether a measure seems relevant and appropriate for what its assessing.
Assesses the degree to which a procedure or test (questionnaire) appears effective in terms of its stated aims.
E.g. does a questionnaire looking at depression effectively measure depressive symptoms in order to provide a trustworthy measure of depression. - Best carried out by someone who is an expert in the field who views the test and makes a judgement as to whether it seems appropriate.
improving ecological validity
use real-life settings participants are likely to encounter in the real world.
improving mundane realism
real-life tasks participants are likely to encounter in the real world.
improving population validity
stratified sample – more representative of different sub-groups in a target population and prevents bias favouring a particular group.
improving temporal validity
repeat in different time periods after temporal shifts
improving investigator effects
- Use of leading questions – use of open-ended questions.
- Bias in allocation of participants to groups – use random sampling to prevent the researcher having any control over allocation.
- Measurement criteria when assessing concepts such as ‘aggression’ or ‘obedience’ – create behavioural categories to clearly define specific behaviours they are looking for to reduce subjectivity.
- Lack of control over procedure (e.g. amount of time groups get to spend to tasks) – use standardised procedure in controlled environment.
- Selective criteria on which data to include.
- Interviewer dynamics – use same interviewer so investigator effects are equalised over all participants.
- Bias in interpretation of behaviours – use double blind procedure.
improving participant effects
- When a participant picks up cues from the study and thus changes their behaviour reducing the internal validity of the study.
- Hawthorne effect
Reduced with a covert observations – do not change behaviour because do not know they are being observed. - Social desirability
Reduced with anonymity – answers cannot be traced back so more likely to be more truthful
No fear of appearing undesirable - Demand characteristics
Mild deception and making aims less clear
Do not know full extent of the aims of the study so do no affect behaviour to suit the aims
Independent measures or matched pairs – only take part in one condition so are less likely to guess the aims of the study
improving participant variables
PARTICIPANT VARIABLES
* Differing individual characteristics of participants in an experiment, sometimes referred to as individual differences.
* Type of extraneous variable that can impact the internal validity of research.
* E.g. age, gender, education levels
* Can be reduced using repeated measures and matched pairs as individual differences do not affect results as either matched on significant characteristics or they take part in both conditions.
* Random allocation can avoid researcher bias favouring certain participant groups.
improving confounding variables
- Is what an extraneous variable becomes when it has an impact on the dependent variable.
- Include situational variables, participant variables and investigator effects.
Situational variables reduced by use of lab experiment
improving validity of unstructured interview
- Ensure no leading questions – affect response due to phrasing
- Using open questions so no limit to amount and quality of data gathered
- Pilot study to test whether questions are useful measurements and whether they provoke an emotional response
improving validity of observation
- Use event sampling rather than time sampling so that some behaviour are not missed.
improving validity of lab research
- Use more natural environment and task with mundane realism to more accurately represent behaviour in the real world.
nominal data
data in categories or groups
ordinal data
ranked data or on a made up scale
interval/ration data
official standardised scale
bar chart
bars shouldn’t touch
comparing multiple groups
scatter graph
correlations
histograms
bars touch
interval data
equal sized intervals of single categories
continuous data
pie charts
nominal results
sections of scientific report
abstract
intro
method
results
discussion
referencing
abstract
brief summary of report (aims, method, results, conclusions)
to determine relevance
written last
200 words
introduction
brief summary of previous research, hypotheses, purpose and context
background introduction, previous studies
method
how study was carried out
design, ppts, materials, variables, controls, sampling, consent
for replicability, reliability
consent forms
purpose, explanation of what required to do, duration, now to withdraw, confidentiality, participant details, space to gather consent and date
debrief forms
purpose, how and when results available, confidentiality, right to withdraw, psych harm, contact for further info
results
detailed summary of results, descriptive analysis of results
graphs, central tendency, dispersion, qualitative data
stats
discussion
conclusions, limitations
placed in context of existing research, highlight direction of future research
references
Name(s) of author(s) (Surname, followed by initials) - Publication Date -. Article Title - Journal title - Issue - Page Range.
Name(s) of author(s) (Surname, followed by initials) - Publication date - Book title - Place of publication - Publisher
random allocation
avoids bias and controls individual differences
equal chance of being allocated to each condition
drawing names from hat or computer programme selecting names at random
counterbalancing
order effects - practice, fatigue
order of conditions mixed up, repeated measures
order effects equalised
randomisation
making groups of items random
presentation of trials to avoid systematic errors
reduces bias as researcher has no control
e.g. difficulty of word lists
standardisation
procedures kept the same
IV affects DV
more successfully replicated
also standardise materials
questionnaire construction
avoid leading questions (encouraging particular answer)
avoid vagueness / ambiguity
avoid double-barrelled questions
avoid overly complex phrases or technical jargon
organise in order to avoid extraneous variables e.g. order effects or demand characteristics
assess validity and reliability
designing interviews
consider qual/quan data
open/closed questions
structured or unstructured
depends on need of interview
consider social desirability, bias and ethical issues
whether recorded, make notes
