Research Methods ★ Flashcards
1
Q
Define and Describe:
A
What happened?
2
Q
Explain:
A
Why did that happen?
3
Q
Predict:
A
Under what conditions is it likely to happen again?
4
Q
Control
A
How can I we apply our
principle to make this
happen again? To make
sure it never happens again?
5
Q
What is Research?
A
Research is a careful and detailed study into a specific problem,
concern, or issue using the scientific method
6
Q
What is the scientific (research) method?
A
The scientific methods rests upon empirical observation and reason.
Scientific evidence requires the testing of statements that potentially can
be shown to be false (disconfirmed).
7
Q
What are the 8 research steps?
A
- A question
- Background literature
3.Hypotheses
4.Method. Participants
5.(sometimes) Conduct a pilot study
6.Collect data - Analyze Data
8.Draw Conclusions
8
Q
Aims…
A
Investigate something- (A general statement about the purpose of an investigation).
9
Q
Hypotheses…
A
a precise, testable statement of what the
researchers predict will be the outcome of the study.
10
Q
Null Hypotheses
A
there is no differences between the
two variables
11
Q
Alternative Hypotheses
A
there is a relationship between the
two variables
12
Q
Non-directional hypothesis=two
tailed hypothesis:
A
the independent variable will have an effect on the dependent variable, but we don`t know the direction.
13
Q
Directional hypothesis=one tailed:
A
the independent variable will have an
effect on the dependent variable and
we now the direction
14
Q
In an experiment we…
A
Investigate a cause-and-effect relationship. The
researcher investigates the way one variable effect the other.
15
Q
IV
A
In an experiment the researcher will
always alter one variable. This is the independent variable.
This variable manipulates the experiment, standing alone, not depending on
anything.
16
Q
DV
A
The dependent variable is the thing the researcher is measuring. The dependent variable depends on the independent variable (the thing that is being manipulated/changed), in order to be valid.
17
Q
Extraneous variables
A
variables that are likely to effect the results of an
investigation. They might confuse the results.
Either acts randomly,
affecting DV in all levels of IV or systematically.
18
Q
Confounding variables
A
extraneous factors that affect the performance of
the participants
19
Q
situational variable
A
a confounding variable caused by the environment
20
Q
Participant variable
A
each participant varies from the other and can effect results
21
Q
Experimenter Variable
A
unconsciously conveys to participants how they
should behave
22
Q
To be confident that the IV has caused the DV, the researcher must control
all other aspects of the experiment. This is called…
A
VALIDITY.
(To what extent the results are valid.)
23
Q
Operationalisation
A
Clearly define what is the IV and the DV = so you (or anyone else) can replicate the study.-measurable
24
Q
Standardisation
A
Standardisation is when different aspects of an experiment are
controlled so that everyone goes through the same experience.
25
Standardised procedure
Each participant must be treated in exactly the same way, doing the same tasks, with the same materials,
in exactly the same order.
This reduces the
extraneous variables in the
procedure.
26
Standardised instructions
Each participant must be given exactly the same instructions, by the same person, and in the same way. Written instructions are best for this.
27
Independent measures design/independent groups:
Experimental group and the control group.
Each condition of the experiment includes a
different group of participants.
-usig diff participants for each condition of exp.
28
Repeated measures Design
The same participants take part in each condition of the independent
variable.
29
Matched pairs design
The study will use different participants in each group but they are matched in pairs on the basis of variables which are related to the study. (age, gender, intelligence, personality).
30
Independent measures design strengths
no order effects. Participants see only one level of the IV,
reducing the effect of demand characteristics. Random allocation to
levels of the IV reduce the effects of individual differences.
31
Independent measures design weaknesses
more participants needed, participant variables can
distort results if there are individual differences.
32
Order Effects are...
When someone goes through one condition and then behave
differently in the second condition because they are starting to guess
the hypothesis.
Practice effect: familiarity or learning a task
Fatigue effect: boredom or tiredness
33
To overcome order effects we use...
Counterbalancing
34
Counterbalancing is...
when he sample will be split into two- ABBA
35
Repeated measures design strengths
each person acts as their own baseline. Counterbalancing
reduces order effects. Need fewer participants.
36
Repeated measures design weaknesses
order effects could distort the results demand characteristics
37
Demand Charasteristics
Features of the experimental situation which give away the aims.
Participants may change their behaviour because of that, so it reduces
the validity of the study.
38
Matched Pairs design strengths
Reduces participant variables. Avoids order effects.
39
Matched Pairs design Weaknesses.
hard to find matched pairs (time!), hard to match people exactly (unless they are twins), if 1 participant drops out you loose the
other one too.
40
Psychologists decide on the size of their sample by taking account of
factors such as...
the experimental design and the time available.
* Sampling is where you choose who the people taking part in your
study (participants) will be.
41
Random Sampling
Every member of the target population has an equal chance of
being selected
42
Volunteer sampling (self selecting)
Where participants self-select themselves, and choose to take part in
the research.
* E.g. people who return questionnaires, or have responded to an advertisement in the newspaper.
43
Opportunity sampling
Anyone who is available and agrees to take part in the research can become a participant.
* E.g. selecting a sample of students from those .coming out of the library
44
Experiments...
Experiments give wider control over what happens, helping us to test cause and effect, and so make some guesses about why things happen and why people behave the way they do
* Experiments use an independent variable and a dependent variable so
that they can measure cause and effect
* An experiment is an investigation in which a hypothesis is scientifically
tested
45
Lab controlled experiments...
This type of experiment is conducted in a well-controlled environment and therefore accurate measurements are possible.
* The researcher decides where the experiment will take place, at what time, with which participants, in what circumstances and using a standardized procedure.
Participants are randomly allocated to each independent variable group.
46
Lab experiments weaknesses
The artificiality of the setting may produce unnatural behaviour that does not reflect real life, i.e. low ecological validity.
This means it would not be possible to generalize the findings to a real
life setting.
47
Lab experiments strengths
It is easier to replicate a laboratory experiment.
They allow for precise control of extraneous and independent variables. This allows a cause and effect relationship to be established.
48
Field experiments...
Conducted in the everyday (i.e. natural) environment of the participants but the situations are still artificially set up.
* It is conducted in the normal environment (normal situation) for the
participants.
* The experimenter still manipulates the IV, to be responsible for
changes in the DV.
49
Field experiments strengths
more likely to reflect real life, higher ecological validity than a lab experiment, less likelihood of demand characteristics
affecting the results.
50
Field experiments weknesses
Less control over extraneous variables, makes it difficult
for another researcher to replicate the study in exactly the same way.
51
Reliability is
the consistency of the findings or results of a psychology research study. If findings or results remain the same or similar over multiple attempts, a researcher often considers it reliable.
52
Validity is
The validity definition in psychology assumes that the test in question measures precisely what it aims to measure, meaning the data collected is accurate and represents some truth compared to others outside of the study. If it does, then the test is valid.
53
Ecological validity
Contributes to the generalisability of the results.
* The extent to which the findings of research in one situation would
generalise to other situations.
* This is influenced by whether the situation represents the real
world effectively and whether the task is relevant to the real life.
54
Test retest reliability
Assesses the external consistency of a test.
Using a measure once and using again in the
same situation. If the reliability is high the
results should be the same.
55
Inter rater reliability
The extent to which two researchers
interpreting qualitative responses will
produce the same records from the same
data (answers can be interpreted in different
ways=low reliability).
56
Inter-observer reliability
Consistency between two researchers
watching the same event. If a researcher gives
different interpretations of the same
actions=low inter-observer reliability.
57
Laboratory experiments features:
Controls:
* Standardisation- for each participant the procedure can be kept exactly the same.
➢The findings of the experiment is RELIABLE
* Controlling variables improves VALIDITY
* Keeping the situation the same: any differences in the DV due to the differences between levels of the IV (not because of the extraneous variables).
58
Field experiment Features:
Harder to control variables and standardise procedures:
➢Reliability and validity may be lower than in a lab
experiment.
* Validity can be improved: familiar environment
➢Better ecological validity than lab experiment.
59
Non experimental methods involve...
The collection and
analysis of data when a variable is not or cannot be manipulated.
60
Open question features:
* More in-depth answers
* No pre-set answer options and allow the respondents to put
down exactly what they like in their own words.
61
Closed question features:
Structure the answer by allowing only answers which fit into categories that have been decided
in advanced by the researcher.
* Fixed set of possible responses.
* The options can be restricted to two or include quite complex lists of
alternatives from which the respondent can choose.
* Closed questions can also provide ordinal data with a rating scale.
62
Closed question strengths:
Economical: large amounts of research data for relatively low costs.
* Large sample size can be obtained =representative of the population
=researcher can then
generalize from.
* Questions are standardised=
questionnaire can be
replicated easily to check for reliability.
63
Closed question weaknesses:
lack of detail
64
Open question strengths:
Rich qualitative data
65
Open question weaknesses:
Time consuming to collect the data.
* Time consuming to analyse the data.
* Coding
* Not suitable for less educated respondents.
66
Questionnaires Strengths:
Relatively cheap, quick and
efficient way of obtaining large amounts of information from a large sample of people.
* Quantitative and Qualitative Data can be collected relatively quickly.
* Closed questions: easier to analyse than interviews:
* Effective
* Measuring the behaviour
* Attitudes
* Preferences
* opinions and intentions
67
Questionnaires Weaknesses:
Respondents may lie due to social desirability.
* Answers to open questions have to
be interpreted= lack of reliability.
* More than one researcher: lack of inter-rater reliability.
* Return rate can be low
* Language of a questionnaire should
be appropriate to the vocabulary of the group of people being studied:
---: social background of respondents' age/ educational level / social class /ethnicity .
68
Structured interviews:
Formal interview:
* Questions are the same for every participant.
* Order is fixed
* (Instructions for the interviewer about how to sit, dress etc).
69
Unstructured interview:
Questions asked dependent on what the participant says.
* Flexible
70
Semi structured interview:
Compromise between the unstructured and structured.
* Some fixed questions
* But possible to ask some questions that are specific to individuals.
71
Structured interview strengths:
Easy to replicate
* Easy to test for reliability
* Fairly quick to conduct
72
Structured interview weaknesses:
Not flexible
73
Unstructured interview strengths:
More flexible as questions can be adapted and changed depending on the respondents’ answers.
* Allows the respondent to talk in some depth, choosing their own words.
74
Unstructured interview weaknesses:
Time consuming to conduct an one and analyse the qualitative
data.
75
Other Features of structured Interviews
Formal interview
* Questions are the same for every
participant
* Order is fixed
* (Instructions for the interviewer
about how to sit, dress etc)
76
Other Features of Unstructured interviews
Questions asked dependent on what the participant says
* Flexible
77
Semi Structured Interview General
Compromise between the unstructured and structured
* Some fixed questions
* But possible to ask some questions that are specific to individuals
78
Subjectivity VS objectivity
Objective means verifiable information based on facts and evidence.
Subjective means information or perspectives based on feelings, opinions, or emotions.
79
Evaluating Interviews
Researchers can be subjective
* The aim would be to be objective
* How?
* Ask researchers who are experienced but do not know the aim of the study
* Participants may lie
- Social desirability bias
- They think they know the aim of the study
* Time consuming
- Participants who volunteer?
80
Case Studies
In-depth investigations of a single person, group, event or community.
* Data can be collected from a variety of sources like observations, interviews, questionnaires.
81
When can case studies be useful?
Rare cases
* Developmental changes (a progress of a child)
* A person with a disorder
82
Case Study strengths
Provides detailed information.
* Highly valid
* Provides insight for further research.
* Permitting investigation of otherwise impractical (or unethical) situations.
* An important way of illustrating theories and can help show how
different aspects of a person's life are related to each other.
83
Case Study weaknesses
Can’t generalize the results to the wider population.
* Researchers' own subjective feeling may influence the case
study (researcher bias).
* Difficult to replicate.
* Time consuming.
* Not reliable
84
Observations
watching what people do
85
Naturalistic observations
In the participant`s normal
environment.
* Without interfering in either the social or physical environment
86
Controlled observation
In a situation which has been manipulated by the researcher.
* Can be in a normal environment of the participants or in an
artificial situation
87
Naturalistic observation strengths
observe the flow of behavior in its own setting studies: greater
ecological validity
* gives the researcher the
opportunity to study the total situation
88
Naturalistic observation weaknesses
often conducted on a micro scale and may not be representative
* less reliable as other variables cannot be controlled
* the researcher needs to be trained to be able to recognise aspects of a
situation that are psychologically significant
* cause and effect relationships
cannot be established
89
Controlled observation strengths
can be easily replicated: easy to test for reliability
90
Controlled observation weaknesses
Lack of validity
* Demand characteristics
91
Unstructured observations
Without focus
* Records the whole range of possible behaviors.
* It is difficult to record all
activities accurately- some
activities can be irrelevant
92
Structured Observations
With focus
* Records only a set of behaviours which is important for the study.
* The specific activities that are recorded called behavioural categories-helps to improve inter-
observer reliability.
* Code behaviour according to a previously agreed scale using a
behaviour schedule.
93
Different roles- participant observer.
When the observer is part of the participants.
* Participants` behaviour are more likely to be natural
* More valid
94
Different roles- Non participant observer
Do not involve in the situation
* How?
* One-way glass
* But it can affect validity
95
Covert VS Overt Observers
Covert observer: the role of the observer is not obvious for the participants.
* Overt observer: the role of the observer is obvious for the participants.
96
Association
A measure of the extent to which two variables are related.
97
Correlations
A change in one variable is related to a change in the other.
We do not know whether the change in one variable is responsible
for the change in the other.
* Two factors we have measured vary together.
98
When are correlations useful?
When you can measure the variables but can not
manipulate.
99
Techniques to collect data for observations
Self-report methods
* Observations
* Tests
100
The nature of the relationship between two variables can described in
terms of direction...
Positive and Negative
101
Variables
Measured variables
* Co-variables
* You don’t call these variablesindependent/
dependent variables.
* To ensure validity you need to define the variables
* Reliability depends on the measures of both variables being consistent
102
No correlation
No relationship between two variables this is
known as a zero
correlation.
102
Positive Correlation
Increase in one
variable tends to be
associated with an
increase in the other
103
Negative correlation
Increase in one
variable tends to be
associated with a
decrease in the other.
103
Correlation Strengths
Allows the researcher to
investigate naturally occurring variables that maybe unethical or
impractical to test
experimentally.
104
Correlation Weaknesses
Not useful to check cause and effect
105
Difference between experiment and correlation
Experiment: isolates and manipulates the independent variable to
observe its effect on the dependent variable, and controls the environment in order that extraneous variables may be eliminated.
* A correlation identifies variables and looks for a relationship between
them.
* The experiment can predict cause and effect (causation) but a correlation
can only predict a relationship.
106
Longitudinal studies
Research which follows up the same pps at intervals over time to track their
development.
* Longitudinal studies recruit a group of people at the same point in time - the COHORT.
107
Cross sectional studies
a way to investigate
developmental changes, by comparing separate groups of pps of different ages
108
Data is taken about the cohort at the start. This is the...
Baseline Data
109
Test Points
There are usually several TEST POINTS over the period of the study
when the same data is collected again.
* At the end of the period of study, there is a FINAL TEST POINT.
110
Strengths of Longitudinal Studies
Same group of participants is followed throughout entire study so so participant variables don't affect data collected.
These studies are best way of spotting developmental trends as they repeat tests at regular intervals and compare the findings.
111
Longitudinal studies weaknesses
Certain participants may wish to move away, disrupting study.
Withdrawal also means that if remaining participants share a characteristic, finding is biased.
Number of practical difficulties: can be costly, very time consuming, and data collection and analysis vary in its strength if the researchers change over time.
--Ethical issues? Reliability?Sample Attrition: the sample size is likely to fall....
112
Raw Data..
when you have numerical results from your investigation
113
Quantitative Data
The quantity of a psychological measure.
* Numerical form which can be put into
* categories, or
* in rank order, or
* measured in units of measurement.
* This type of data can be used to construct graphs and tables of raw data.
* Either numbers or frequencies of occurrence.
* Can be analyzed using statistical techniques.
* This data is associated with experiments and with correlations.
114
Quantitave data strengths
Highly objective
* Easy to statistically analyse, making it less time consuming.
* Scales and questions are often reliable.
115
Weaknesses
Less detail can be obtained-
lowering validity
116
Qualitative Data
Indicates the quality of a psychological characteristic
* It is more in-depth
* The aim of qualitative research
* to understand the social reality of individuals, groups and cultures as nearly as possible as its participants feel it or live it.
* Observations
* Responses to open questions
* Interviews
* Case studies
117
Qualitative data Strengths
In depth insight into peoples thoughts and feelings, or detailed
information about behaviour.
* Focus is on causes rather than simply looking at effect.
* Data is valid because pps can express themselves.
118
Qualitative Data weaknesses
Subjective- findings can be
invalid.
* Transcribing the data can take a long time.
* Interpretations may be incorrect, biased by the researcher`s opinions/feelings
* Can not generalize to the majority
119
Experimental design
is the way you allocate participants in the experiement
120
Cross sectional data- VS longitudinal Data
In a cross-sectional study you collect data from a population at a specific point in time; in a longitudinal study you repeatedly collect data from the same sample over an extended period of time.
121
An interview is...
a face to face self report method/ interaction
122
a cohort is
A cohort is a group of people that share similar characteristics, such as their birth place or year.
123
Random sampling
every member of a pop has an equal chance of being selected
124
Random sampling strengths
represantative, eliminates sample bias
125
Random sampling weaknesses
very hard to achieve (time, money)
126
Volunteer sampling
participants self selec themselves to take part in research
127
Volunteer sampling strengths
convinient, ethical with informed consent, not costly or timely
128
Volunteer sampling weaknesses
unrepresantive-bias
129
Opportunity sampling
select available people
130
Opportunity sampling strengths
quick + economical (most common type)
131
opportunity sampling weaknesses
unrepresentative and often biased by researcher who will choose participants.
132
The aim of descriptive statisctics...
is to give an accurate summary of the
data
133
Measure of central tendency
Gives a typical value for the data set
* Tells you where the middle of the data set is
134
Measure of spread
Indicates how the data are spread out
* Tells you what the rest of the data are doing
135
Saying central tendency is essentially saying...
average
136
Three different measures
MODE, MEDIAN, MEAN
137
MEAN
Can be only used with numerical data
* The mean is calculated by adding up all of the scores and dividing by the number of individuals scores
* The most informative measure of central tendency because it takes
every score into account
* When the sample size is large and does not include outliers, the mean score usually provides a better measure of central tendency.
* Use it when the data are normally distributed and there are no outliers
138
MEDIAN
It is the middle score in a set of scores.
* The median is used when the mean is not valid. (i.e outliers that give a msileading av, skewed data, ordinal data, missing or open end values)
* Put all the scores in order (from the smallest to the largest) and find the middle score=that is the median.
* If there are even numbers in the middle, add together and divide by 2.
* Median is a better indicator of the most typical value if a set of scores has an outlier. (An outlier is an extreme value that differs greatly from
other values.)
139
MODE
The number that occurs the most frequently or the most common
observation among a group of scores.
* The mode is the best measure of central tendency for categorical
data.
140
Measures of spread
Indicates how the data are spread out
* Tells you what the rest of the data are doing
* Are they clustered together or widely dispersed?
* The range
* The standard deviation
In A-level psychology, measures of spread describe how data points differ. Key ones are:
Range: Difference between the highest and lowest values.
Interquartile Range (IQR): The range of the middle 50% of data, reducing the effect of outliers.
Variance: The average squared difference from the mean.
Standard Deviation (SD): Shows average deviation from the mean, more interpretable than variance.
These help assess data variability and reliability in research.
141
RANGE
The simplest measure of spread.
* The difference between the biggest and the smallest values in the
data set plus one.
* Why do we need to add plus 1?
* Because the scales we use measure the gaps between point, not the
points themselves.
* Does not accurately reflect outliers (An outlier is an extreme value that
differs greatly from other values)
142
The Standard deviation
A more sophisticated measure of dispersion, because it calculates the
average distance from the mean of all scores.
* More powerful than the range since the value of all scores are taken
into consideration.
* It is usually used when the data itself is precise.
143
Standard deviation continued
The calculation of the average difference between each score in the
data set and the mean.
* S=standard deviation
* X= each score in the data set
* -x=the mean of the data set
* Σ=sigma, `the sum of`, add them
* n=the number of score in the data set
* √= the square root
144
Three types of graphs to present data
Histograms
Bar charts
Scatter graphs
145
Bar charts
Used for data in discrete categories.
* Used for totals of data.
* Shows amounts or the number of times something occurs.
* Each bar will represent a certain category and the bar charts must be
separate.
* The x-axis represents the distinct groups and not the linear scale.
* Can be drawn horizontally or vertically.
146
Histograms
Show the pattern in a whole data set, where there is continuous
data (for example data measured in a scale)
* Show frequency on a continuous scale.
* There is no space between adjacent columns (usually).
* The x-axis has the score value/group of scores.
* The y-axis has the frequency of each category.
147
The main difference between bar charts ad histograms
Bar charts, each column represents a group defined by a categorical variable
* Histograms, each column represents a group defined by a continuous, quantitative variable.
148
Scatter graphs
Correlations
* The results from a correlation study are displayed on a scatter graph.
* Strong correlation: all the points lie close to the line
* Weak correlation: points are more spread out
* Strengths of correlation:+1 to -1
* Values close to +1 are strong positive correlations, close to -1 are negative
correlations. (closer to 0 weaker the correlation)
* No correlation: 0
* You can not draw causal conclusion from a correlation study. Only tells
the relationship!
149
Correlation=
Association, a measure of the extent to which two variables are related.
150
Ethical questions
Focus on the important questions:
* Why are ethics important?
* What are the issues?
* What is and isn’t acceptable?
* How should we make ethical judgements?
151
Ethical issues
Using animals or humans have the potential to cause concerns about the welfare of them.
* The nature of the study can cause potential discomfort or
psychological discomfort.
* Hide the real aim of the study.
* What can psychologist do then?
* Follow the ethical guidelines=ethical codes
* Every country has an organisation which produces a code of conduct.
152
Ethical guidelines relating to human participants
Informed consent: knowing enough about a study to decide whether
you want to agree to participate.
* Presumptive consent: Researcher asks a group of people similar to
those who will become participants whether they find the study acceptable.
* researcher can presume that the actual participants would have agreed
* Debriefing: explain them the aims and potential consequences at the
end of the study
* Protection (physical an psychological)
153
Ethical guidelines relating to human participants continued.
Right to withdraw: they should know that they can remove themselves, their data from the study ANY TIME.
* Privacy (participants` emotions, physical space) has to be respected in
all experiments.
* Confidentiality (participant`s results and personal information) has to be respected in all experiments.
* Deception: participants should not be misinformed, deception should be avoid. (tell them at the end what was the real aim).
154
Ethical guidelines relating to the use of animals
Driscoll and Bateson (1988): animals are convenient models
* The means justify the ends =animal suffering caused by the planned
experiment is outweighed by the benefits.
* Replacement: we should always consider to replace animal experiments
with alternatives (computer simulations)
* Species and strain: what you choose should be the one least likely to suffer
pain or stress
* Number of animals: minimum
* Pain and distress
* Housing: put animals in a cage can cause distress
* Anaesthesia, analgesia, euthanasia
155
Experimental Conditions
The levels of the IV being compared may be two or more experimental
conditions (such as bright and dull artificial lights) or there may be one
or more experimental conditions that are compared to a control
condition (e.g. artificial light compared to daylight).
156
Control Conditions
The control condition is simply the absence of the experimental
variable.
157
Conditions example
For example, in a comparison of the effect of eating chocolate on
paying attention, we might compare either the effect of eating one bar
or two bars (two experimental conditions) or the effect of eating one
bar to no chocolate at all (one experimental and one control condition).
158
Answering a 10marker:
- AIM-
-(HYPOTHESES)
-SAMPLE+ SAMPLE TECHNQIUE
-IV, DV, OPERATIONALISATION
-PROCEDURE( Standardisation, control of extraneous vaiables, location, time, number of participants, type of reseacrh method)
-EXPERIMENTAL DESIGN
-DATA
-ETHICS
159
Snowball sampling
you ask someone who then asks another and then another...
160
Correlation does not imply...
Causation
161
Attrition is ...
participant dropout over time in research studies. It's also called subject mortality, but it doesn't always refer to participants dying!
162
Postal quesionnaires are those...
posted out to the sample.
163
An example of a phsychometric test is an ...
IQ TEST
164
behaviours are observed using a...
behavioural schedule
165
In ethics, we aim to...
minimise harm and maximise benefits
166
Filler questions
Filler questions are used to fill in time or space, often with small talk or specific queries that may not be significant on their own.
-to disguise the aim of the study by hiding the important questions among irrelevant ones so that participants are less likely to change their behaviour by working out the aims.
167
What is a meta-analysis?
A meta-analysis is where researchers combine the findings from multiple studies to draw an overall conclusion.
