Introduction Flashcards
5 levels of evidence strength?
- meta analysis
- experimental studies
- correlational studies (longitudinal and cross sectional)
- qualitative studies
- ad hoc personal observations
2 most important concepts of study design?
validity
reliability
Define validity
Are we really measuring what we say we are
measuring?
Define reliability
Can our measure produce the same results every
time?
• Provided the thing we are measuring is not subject to change.
• For example, personality is supposed to be a stable trait and therefore a measure of personality should get the same (or very similar) results when a person is tested at different times.
• However, some things such as attitudes can change and the very point of our study may be to see if a certain intervention changes attitudes.
• For example, we might want to see if we can change attitudes to engaging in pro-environmental behaviours.
Existing data advantages
- If open source it is easy to access.
- Most often does not require lengthy ethical assessments.
- For certain types of study, it is the only practical source of information (see Lecture 2).
Existing data disadvantages
We must think carefully about its validity – is it really a measure of what we think it is?
• Qualitative data needs skilled manipulations - especially in relation to reliability (see Lectures 3 and 4).
Define coding
the process of classifying observables such as behaviours into specifically defined categories for data analysis
What is a code book/scheme?
descriptive document that explains how data has been defined and classified in order to be converted into numerical (often categorical) data
Na
Na
Observational studies - advantages
Observations allow us to study what people actually do.
Observational studies - disadvantages
Can generate enormous quantities of qualitative data.
• It can be hard to decide what is important.
• Can be very difficult and time consuming to convert into data.
• Requires experience to develop reliable data coding schemes.
• If the observer is present (or known about) it can change the behaviour of the observed.
• If the observer is not known about it there are a lot of ethical issues to be addressed.
• Because the participants have not given consent to be observed.
Questionnaires - advantages
- A relatively large number of people’s responses can be collected.
- The participants may be widely geographically distributed.
- Reliable scales can be developed.
- Scales allow direct comparability between subgroups’ responses.
- Or direct comparability at two or more different time points.
- They are less time consuming than interviews.
Questionnaires - disadvantages
- They require a lot of skill to do well.
- The issues of concern are pre-determined by the researcher.
- There is no opportunity to explore issues in more depth.
- They have poor response rates.
Interviews - advantages
Allow the researcher to get detailed information about what the participants think.
• Are not totally based on the researchers’ preconceptions and allow follow up of unforeseen issues.
Interviews - disadvantages
- All the same problems as other qualitative data collection techniques, i.e. what to do with large quantities of descriptive data.
- Responses may be affected by the individual interviewer.
- Participants may not want to reveal information about sensitive subjects in a one-to-one situation.
- Very time consuming.
- Often difficult to make comparisons between participants.
