Research Methods Flashcards
Reliability
Measure of consistency within set scores & items and also across time.
Therefore, is it possible to obtain the same results on subsequent occasions when the same ….(method, test, measuring implement) is used
Empirical Measures
The ensuring that a statement is true through direct observation and collection of facts, rather than through reasoned argument
Evidence rather than logic
Objectivity
The research must not be affected by the expectations or wishes of the researcher.
To achieve objectivity, systematic data collection and controlled conditions are preferable (therefore a lab experiment is best) using an impartial experimenter who will not bias the outcome of the study
Theory construction
A collection of general principles which can explain facts and predict natural phenomenon.
Theories are modified through hypothesis testing
falsification
The process of proving a theory to be correct by trying to find ways of disproving it. If we repeatedly fail we can be reasonably sure of the theory’s validity. Theories must be able to be disproved (evolutionary theories often cannot)
Hypothesis testing
The predictions generated by the theory form a hypothesis which is tested in order to ascertain the validity of the theory and modify it if necessary (falsification is used)
Induction
reasoning from the particular to the general. Repeatedly observing (like Newton) -> drawing conclusions from hypothesis testing -> generalising to general law (theory)
Deduction
From the general to the specific. Proposing a theory then generating hypothesis to test in specific situations in order to find instances than support or challenge the theory (eg. Darwin)
Problems with Psychology as a science
Science is nomothetic - it makes generalisations
Some psychology is ideographic - it studies individuals and specifics. Individual people do not always conform to tidy patterns or fit with general principles. They won’t always act the same each time.
Psychology tests things that cannot be measured. Eg. Clark and Mills test ‘love’ in relationships. ‘de-individuation’, a ‘phonological loop’, all concepts not solid, testable facts
There are problems with participant effects and conformation bias etc. which compromise validity. While these also exist in regular science they can be more of an issue when testing people
Science is reductionist, people and their situations are holistic. Science is also determinist (looking for causes) this has implications for applying to people.
What are the positives about psychology as a science?
Empirical evidence is desirable. It means more people will accept the findings and statements of psychologists if there is valid ‘empirical’ (factual) support to back up their claims
Most psychologists do conduct well controlled experiments on their models and theories which enable the models to be falsified. They aim to control EVs and reduce subjectivity and bias, just like regular scientists.
Why is peer review useful?
Allows rankings of research and universities. This allows the government to allocate research funding to specific areas and institutions.
It allows psychologists to share and learn what current research is being conducted in their field, keeping them up to date with developments.
It prevents faulty research from entering the public domain and being relied on by the gullible public.
What are the problems with Peer review?
Anonymity- while it’s aim is to increase objectivity, it can result in researchers settling old scores or burying rival pieces of research
Publication bias: interesting or positive research is more likely to get read and published. Boring research which may accept the null hyp or confirm previous research can get pushed aside. On the opposite side of the spectrum, research which seriously disrupts the status quo (disproving existing theories or accepted ideas) will be viewed with suspicion. Change may therefore be slowed down.
It can be difficult to find an expert in obscure areas, poor research may be passed because it was not understood
Once published, poor research continues to be use, even if it is debunked.
What are the major features of science according to the spec?
replicability objectivity theory construction hypothesis testing the use of empirical methods
What are the different types of sampling techniques?
Opportunity sampling Volunteer sampling random sampling stratified and quota sampling (snowball sampling)
Stratified sampling
Before sampling, the target population is divided into subgroups based on characteristics important for the research. If 35% of the pop are left handed then left handers will be randomly selected to fill 35% of the sample.
quota sampling
The researcher decides how many left handed people he wants in his sample eg. 50% and then goes and finds that amount. This is different from stratified sampling because 50% of people are not left handed therefore the sample is not representative of the target population. Nevertheless, it may be necessary to have an unrepresentative sample if you are testing a particular, perhaps minority trait.
Opportunity Sample
Those who are available. Eg. you may go to Asda on Wed morning and hand out food shopping surveys to all those you find.
This may be biased as no working people or children would be at Asda, people who shop at Waitrose would not be at Asda… etc.
Easy, quick, finds people who would be relevant for the for the study, eg, you could find a load of schizophrenics at schizophrenia support group, they may not be representative of all schizophrenics but this may not be wildly important
Volunteer Sample
Can acquire a large sample easily (especially if advertising in a national newspaper)
will inevitably be biased (only motivated, un-busy and interested participants)
Random Sampling
Members of the target population are allocated a number and then numbers are randomly generated in order to select participants.
This often results in more representative samples as everyone has an equal chance of being picked, however this is not guaranteed and people may refuse to take part
Snowball sampling
If it is difficult to find people with a certain trait then snowball sampling could be used. This is where one participant can introduce another few potential participants to the study, because they may know them and you do not. This allows you to conduct research on groups of people who are difficult to identify (eg. get interviews with members of the persecuted church) although inevitably leads to a biased sample as only a select few who know others will be contacted.
What ethical issues need to be considered in research?
informed consent (children and vulnerable adults need special consideration, pressure or monetary incentives should be avoided)
right to withdraw (must be made clear, can be retrospective if poss)
protection from harm
deception (deliberately misleading people should be avoided if poss)
anonymity
confidentiality (their information should not be identifiable as theirs)
The ethics of using animals in research
How to overcome: a failure to protect ppts from harm
right to withdraw
informed consent
debrief and continuing support if necessary
How to deal with: lack of informed consent in research
Presumptive consent (ask others, would you do this? do you think x will mind?)
Right to withhold data (retrospective withdrawal from the study)
debrief
Ways to deal with: lack of right to withdraw
Informed consent
right to withhold data
Ways to deal with a lack of confidentiality
informed consent
right to withdraw
right to withhold data
Animal research, pluses and minuses
Can conduct research which is highly necessary which we cannot do on humans, can minimise discomfort through painkillers, using humane techniques, small samples
Animals may suffer harm, may not be justified due to extrapolation issues anyway, is favouring humans ‘speciesism’?
Significance level
The level of probability (p) at which it has been agreed that the null hypothesis should be rejected. Often the significance level is:
P
Type 1 error
We incorrectly accept the alternative hypothesis when we should have accepted the null
because we used a significance level that was too lenient.
Type 2 error
We incorrectly accept the null hypothesis and reject the alternative hypothesis because we were
too stringent and used a significance level of p
One tailed test
For a directional hypothesis
Two tailed Test
for a non-directional hypothesis
Null hypothesis
an assumption that, within the target population, there is:
no relationship/difference/ association with respect to the variables being studied.
The null suggests the results are not significant, they are due to chance
alternative hypothesis
A testable statement about the relationship between two variables. In an alternative hypothesis there will always be a relationship or difference or association between the variables
This is called an experimental hypothesis if it is an experiment
The alternative hypothesis states that the results are significant and are not due to chance
directional hypothesis
States the nature of the effect of the IV on the DV (
the direction of the relationship (positive, as x increases y will increase)
the direction of difference (higher than, more, less than)
non-directional hypothesis
Predicts that there will be an effect or difference or relationship but does not state the direction this will take
‘there will be a difference between…’
‘there is a relationship between…’
What information do you need to find whether something is significant using an inferential test
(N) - number of ppts in the study, This is also sometimes called degree of freedom or (df)
Is this a one tailed or two tailed test
What significance level are you using (usually this is p
nominal data
…and the necessary inferential test
…and the necessary graph
names (duh!) so: red, blue, Hans Solo, Sheffield…
things that cant be plotted like numbers
Chi-Square
bar chart
Ordinal data?
Suitable graph…
Suitable inferential test…
Data that can be ordered (unlike nominal data like names)
so ages, scores, heights etc…
bar charts and scatter graphs (for correlations) are both good
depending on other variables you can use Man-Whitney U, Wilcoxen T, Spearman’s rho (rank)
When do you use a Chi Square
Nominal data
When do you use Mann-Whitney U?
ordinal data
independent measures design
Test of difference (not a correlation)
When do you use Spearmans rho?
Correlation
ordinal data
When do you use Wilcoxon T?
repeated measures or matched pairs design (not independent groups)
ordinal data
test of difference (not correlation)
What is the observed value?
The thing that you compare to the critical value which you find in the inferential test
it is the ‘test statistic’ - the result of your research study and some unknown calculations.
It is called the ‘rho’ or ‘U’ or ‘T’ retrospectively
What is the ‘critical value’?
The number which the observed value must be less than or more than in order to accept or reject the null hypothesis.
It is found in the giant statistical test tables if you know how
Interval data
data at equal intervals such as scores
Frequency polygon
A frequency polygon is the graph that you get when you take the scores given on a histogram for each of the group, which are called classes (for example there were 6 bananas in the group aged ‘16-20’) you then plot this score on the y axis. On the x-axis you plot the point in the middle of the values for that groups (so your co-ordinate would be 18,6 for the example given above because 18 is (hopefully) the middle of the group 16-20)
Histogram
like a bar-chart but for data which is grouped into ranges
eg. 1-
Triangulation
comparing results on a particular topic from a number of different studies which use different methodologies.
Thematic analysis
pinpointing, recording and analysis themes and patterns which emerge from the data.
familiarise yourself with the data
break it up into meaningful codes (so it is shorter and easier to handle)
assign a label to each code, eg ‘sadness shown’ ‘playing with toy’
search for themes amongst the codes, begin to categorise
review, define and name specific themes
produce the final report illustrating the emerging themes using quotes
draw/ state conclusions from them)
Grounded theory
When analysing qualitative data, explanations and theories slowly emerge throughout the course of the investigation based on the data which is being examined (also called emergent theory?)
discourse analysis
generic term for all approaches which analyse language (written or verbal) Discourse analysis aims to examine the socio-psychological characteristics of people rather than examining the text stucture like a linguist.
Interpretative phenomenological anaylsis
aims to discover how a person ‘interprets a given phenomenon’ based on their context. Eg. driving may be viewed as a right of passage
Quantitive data
numbers or quantities
good because: easy to analyse, produces neat conclusions
bad because: it over simplifies the reality of the human experience
qualitative data
not numerical, what people say, thoughts, names etc.
Is good because: often represents complexities of human behaviour, captures rich detail and gains access to thoughts and feelings which quantitive data can’t)
Bad because: more difficult to detect patterns and draw conclusions, subjective analysis could take place (although possible in both)
Ethnography
provides rich, detailed qualitative description and analysis of an area of culture
behavioural catagories
dividing target behaviour into subsets eg. aggression into punching, swearing, glaring.
content analysis
summarising qualitative data is inductive and painstakingly slow.
It can be done via thematic analysis, by which themes gradually emerge and statements are categorised into various themes which have arisen from the text.
it is good in that can extract findings from qualitative data, gaining insights into previously overlooked things (which can then be included in closed questions later)
long, difficult and hugely prone to the whims of the researcher who chooses the themes and categories, summarises and groups. researcher bias (& confirmation bias) should be done by more than one
Lab experiments
Very well controlled environment, standardised procedure.
repeatable (due to standardised procedure) therefore can improve reliability
control of EVs and IV means cause and effect can be established
artificial task, Lacks mundane realism, low ecological validity.
Demand characteristics and investigator effects may lower validity
Field experiments
IV controlled by researcher but in a natural environment
Ppts more likely to behave in a way that represents real life - often higher ecological validity
if unaware - less demand characteristics
Cannot establish cause and effect due to EVs,
potential ethical issues if ppts are unaware of experiment
difficult to repeat to improve reliability
Natural experiment
IV occurs naturally and its impact of the DV is observed by the experimenter.
can research events that cannot be manipulated either due to practical or ethical reasons
V. high ecological validity and mundane realism, real people really responding to actually events (no probs with demand chars or experimenter effect unless he is super intrusive
Hard to replicate
situation often unique (therefore may not be applicable)
Cannot control EVs - cannot establish cause and effect
Naturalistic observation
can be highly representative of real life, highly objective and highly reliable
research can be performed where manipulation would be impossible (unethical or impractical)
inter-observer reliability can be low due to bias and perception
when confronted with a continuous string of behaviour it may be difficult to categorise into a separate observable indicators and emotional states
controlled observation
EVs can be controlled, greater internal validity
Task may be artificial, mundane realism?? demand characteristics if overt observation, observer bias (inter-rater reliability)
covert observation
ppts do not know they are being observed: higher validity (less demand characteristics and experimenter effects) can observe places like gangs which would not reveal themselves if you asked
can be unethical and practically difficult, observer bias
Overt observation
demand characteristics, social desirability bias, observer bias (confirmation bias?)
often easier to pull off and ask Qs (depending on situation)
behavioural catagories
dividing a target behaviour (like aggression) into specific behaviours which can be recorded. This can be done using a behavioural checklist (list of behaviours to be recorded) or a coding system (observers told to systematically record behaviours giving a code to each for ease of recording)
time sampling
every 30 seconds or 2 days or whatever a note is made of what behaviour is occuring
event sampling
every time a ppt performs a behaviour the event is written down in the behavioural checklist or coding system
Interviews- structured
predetermined set of questions (a pilot study may be useful to set these)
faster to analyse and collect
can be easily repeated to generate more data or clarify findings
can see facial expressions and hear tone of voice
Interview bias and exp. effects may affect phrasing of questions and interpretation of answers given. Ppts may conceal information due to social desirability bias or forget or just not know some things (only their opinion) although on some things some ppt may not lie to your face but might in a questionnaire
Interviews - unstructured
no (few) set questions - topics raised as relevant
more freedom to follow relevant topics and discover rich insights
harder to analyse and repeat
need trained, skilled interviewer
Interviewer bias may result in certain phrasing or things being emphasised incorrectly
social desirability bias, and people not knowing or forgetting answers (although people may be less likely to lie to your face, depends on person and issue)
Questionnaires
flexible, can be fast, easy and cost effective way to target a very large sample, they can also be easily repeated and often are easily to analyse (esp. if closed questions)
people may be more truthful about awkward matters
Only some will return a questionnaire (sample bias)
leading questions, social desirability bias and response bias (such as always wanting to answer ‘yes’
people may not take it seriously and so may make stuff up to get it done
Open questions
Unrestrained ‘ tell us about your worries starting uni…’ ‘what did enjoy this week?’ ‘why…?’
produce qualitative data
richer, more representative
harder and more time consuming to analyse and represent
closed questions
answers set -can produce quantitive data which is easier to analyse and present
results relevant
forced choice - may not reflect real feelings of ppt. (a pilot may be useful to establish answer selection)
Case study
observe individual entity (person, event or institution) in detail using a range of methods to produce an indepth portfolio from which conclusions can be drawn
can examine rare and impossible to manipulate situations
due to rich detail, can demostrate how areas of life interlink and allow opportunties for new ideas, unusual insights and illustrations of theories
cannot be repeated
may not be representative of anything/one else (not generalisable)
privacy must be respected
very prone to researcher bias (certain element of their life are discarded, others focused on)
time consuming
demand characteristics
features of an experiment which cue ppts to behave in a certain way
social desirability bias
ppts lie or change behaviour in order to try and present themselves in a good light
investigator (experimenter) effects
unconscious cues which the investigator may give off or things they may do which affect ppt behaviour, often by alerting them to the aims of the study or the expectations of how they will act
order effects
Extraneous variables which arise based on the order that ppts perform the conditions within a repeated measures design. May be boredom, practice, tiredness etc…
can be avoided by counterbalancing: in which ppts are divided up so that some do condition 1 first and others condition 2 first. Thus both/all conditions are tested 1st or 2nd in equal amounts
Repeated measures design
every ppt performs in every condition
(so they do condition 1 and then condition 2 and the 3 etc)
Prone to order effects although avoids ppt effects. Order effects can be avoided by counterbalancing the conditions
operationalised
providing variables in a form that can be easily tested and understood, so the vague concept of ‘intelligence’ may be operationalised to become ‘score out of 100 on standardised IQ test’ or something. ‘Happiness’ could be ‘no. of times smiled in 5 minutes’
correlational analysis
allows us to determine the extent of a relationship between two variables
can test things that cannot practically or ethically manipulate (like smoking causing lung cancer)
from this we can make predictions
It is also useful preliminary research for quickly establishing that there is a relationship between two things and the direction, the specifics of which can then be tested
can be done on large data sets and easily replicated
Cannot establish cause and effect, must be careful about making predictions
can lack validity (depending on sample and variables used and operationalised)
matched pairs
ppts are coupled based on a similar score in an area which the investigator deemed necessary to control in order to conduct a valid exp. so if you were testing effect of room temp on maths test score, you would need to match people on maths ability so that all the smart ones did not end up in the cold room and cause an invalid finding.
Time consuming and difficult to match (can end up losing some ppts if they have no partner)
what do you match on? pilot needed.
avoids certain ppt variables that you matched for whilst still allowing independent groups design and therefore not creating order effects
independent groups
different ppts go to different conditions.
avoids order effects
can suffer from ppt effects and differences in the environment, for example is one condition unintentionally noisier or colder than another (these should be minimized)
Reliability
consistancy
internal - assessed using split half method
external - assessed using test-retest
inter-rater reliability - assesed by comparing scores of 2 researchers
Improved by:
inter- rater - pilot, meeting to ensure consistency, clear behavioural categories - standardisation of criteria, trained raters/experimenters
operationalised variables
repeating the measure/ test / experiment
conducting a pilot study to ensure apparatus work consistantly
Validity
truthfullness
internal - assessed by face validity, concordant validity, predictive validity
external - ecological, population, historical
to Improve external: more mundane realism, representative sample
to improve internal: erradicate EVs by using a single or double blind or counter balancing
run a pilot study to iron out probs
measures of central tendency
mean- +++ / total no. this can be unrepresentative of data as a whole as it can be skewed by extreme values, not for nominal data
median- middle value, not all valued reflected, less skewed by extreme values, not for nominal data
mode - most common value, can be used for nominal data, can be multiple modes, not always representative as only one (or only the ‘mosts’) represented.
Abstract
a summery of the study which allows the person who reads it to know what the report says so they know whether to read the rest of it. contains:
aims, hypothesis, procedure (brief), results, conclusions, implications
introduction of a report
review of previous research reasons for this research hypothesis aims predictions the intro should be a funnel, starting broad and moving to the specific
Method (in report)
detailed description of what was done so the study can be replicated. Includes: sampling (ppts) design procedure apparatus ethics
results (in a report)
What was found. Include:
descriptive statistics: tables and graphs, measures of central tendency and measures of dispersion
Inferential statistics: observed value, significance level and whether to accept of reject the hypothesis.
Qualitative research would include: categories used, themes used with examples and quotes from the data
discussion
summery of results
evaluation of research - criticisms of method and suggestions for improvement
relationship to previous research
suggestions for future research
Implications for psychological theory and for the real world
References and appendices
references: as usual, full details of journal articles or research cited
appendices: relevant documents mentioned in the report (or necessary to replicate the experiment) that would have disrupted the flow of the main text, eg. consent forms and questionnaires used.
