Self Report Design Flashcards
why is self-report design important?
questions cannot just be asked, they need to be designed to avoid issues such as leading questions
for example, the way question is written or delivered may bias the response that is given, resulting in the collection of data that does not truly represent what a person actually thinks or feels
what is needed for a good question?
clarity
lack of bias
analysis
clarity
questions must be written so that the respondent understands what is being asked, there should be no ambiguity
double negatives reduce clarity — a double negative is when there are two negative words in one sentence, for example “are you against banning capital punishment?”, it’s not immediately clear what this question is asking
double barrelled questions are also a problem — for example, asking “do you suffer from sickness and headaches?” may cause confusion on how to answer if you only suffer one or the other
lack of bias
any bias in the question might lead the respondent to be more likely to give a particular answer
social desirability bias — respondents prefer to give answers that make them look more attractive, nicer and more generous rather than being totally truthful
analysis
questions need to be written so that answers are easy to analyse, they can either be open or closed
open questions invite respondents to provide their own answers and reasons, more likely to receive different answers from each respondent which makes them harder to compare and analyse — tend to produce qualitative data
closed questions are where a range of possible answers is fixed, such as listing five possible answers to choose from or asking a question with a yes or no answer, such questions are easier to analyse and compare as they are standardised BUT the respondents may be forced to select answers that don’t represent the real thoughts or behaviour — tend to produce quantitative data
what else is needed for a good questionnaire?
filler questions — it may help to include some irrelevant questions to distract the respondent from the main purpose of the survey, this may help to reduce demand characteristics
sequence — it is best to start with the easy questions, saving questions that might make someone feel anxious or defensive until the respondent has relaxed and is used to the process of being asked about themselves
sampling — the respondents need to be representative of the population, questionnaire studies often use stratified sampling
pilot study — the questions can be tested on a small group of people which means they can be refined in response to any difficulties encountered
open question evaluation
larger amount and more detailed information can be collected as respondents are able to expand on their answers and give reasons
can provide unexpected answers, therefore allowing researchers to gain new insights into peoples feelings and attitudes that they had not expected before
respondents that are less literate may find open questions difficult and most respondents may simply avoid giving complex answers, therefore in practice open questions may not actually provide detailed extra information
they produce qualitative data which is more difficult to summarise because there is likely to be such a wide range of responses. in any study, patterns are looked for so conclusions can be drawn about the behaviour being studied but if there are lots of different answers it is more difficult to summarise the data and detect clear patterns
closed question evaluation
have a limited range of answers and produce quantitative data which makes the answers to closed questions easier to analyse using graphs and measures like the mean
do not provide as much detail information
but do you provide more focused information that is more relevant to the behaviour being studied
respondents may be forced to select answers that don’t represent their real thoughts or behaviour, reducing the validity of data collected
types of data
qualitative data is non-numerical data
quantitative data is numerical data
interview design
previous issues of clarity, bias, analysis, filler questions, sequence, etc all apply to interviews as well
design of interviews also involves…
• recording the interview
• effect of interviewer
• questioning skills
recording the interview
interviewer may take notes throughout the interview to document answers
but this is likely to interfere with their listening skills AND it may also make the respondent feel a sense of evaluation because the interviewer may not write everything down and then the respondent feels that what they said was not valuable
alternatively interviews may be audio recorded or video recorded
effect of interviewer
presence of an interviewer may increase the amount of information provided
non-verbal communication — various behaviours such as sitting with arms crossed and frowning communicate disapproval and disinterest whereas head nodding and leaning forward encourage the respondent to speak
listening skills — an interviewer needs to know when and how to speak, they should not interrupt too often and when they do speak they should have a range of encouraging comments to show they are listening and engaged
questioning skills
in an unstructured interview, there are special skills needed in order for the interviewer to know what follow-up questions should be asked
it’s important to be aware of the questions already asked and to avoid repeating them
it is also useful to avoid probing too much or asking “why” too often, it is best to ask more focused questions, both for the interviewee and also for later analysis of the answers
