C9

Question 1

Importance of good design

Answer 1

Effect research and quality data depends on good design; not only should it be effective addressing research objectives (collecting valid & reliable data that addresses problems clearly and unambiguously. Questionnaire plays role in helping interviewer:

gather and record data accurately and effective
Helping respondent provide accurate, complete and reliable data

It must be a workable, user-friendly tool for interviewer, respondent and data analyst. It also plays a role in representing research and the research industry to the wider role.

Question 2

Relevant MRS COC

Answer 2

All written and oral assurances made by member involved in commissioning or conducting projects must be factually correct and honoured by member
Reasonable steps to design research to specification agreed with client
Reasonable steps to design research which meets quality standards agreed with clients
Reasonable steps that data collection process is fit for purpose & clients have been advised accordingly
RS that design and content of questionnaires are appropriate for audience being researched
Respondents able to provide information in a way that reflects the view they want to express, inc DK and PNTS where appropriate
Not led towards particular view
Capable of being interpreted in unambiguous way
Personal data collected are relevant and not excessive

Question 3

Some of the ways that badly designed questionnaires may introduce error into research process:

Answer 3

Unpleasant experience for respondents and p[oor perception of research, research industry; which can lead to unwillingness to take part in future research
Poor introduction or presentation of research can lead to high level of non-response and problems with unrepresentativeness in sample
Poorly conceived Qs that do not measure what they claim to measure mean data collected is not valid
Unsuitable / irrelevant content - questions that lie outside of respondent’s frame of reference, relate to subjects of which they do not have knowledge of or tat rely too heavily on respondents memory to provide accurate answers - inaccurate and unreliable data
Poorly worded Qs (ambiguous vague difficult, unusual or technical language), can be misunderstood, misinterpreted or interpreted differently by different people leading to unreliable and inavlid data
Badly structured qnn (difficult, sensitive or personal Qs appearing before sufficient rapport has been established) can result in refusals to answer or complete qnn
Poor q order may result in order ias or contamination of later responses by earlier questions
Long, boring or repetitive Qs may result in loss of interest in answering or produce inaccurate responses
Long qnn can lead to respondent fatigue, loss of interest and thus poor-quality data - too short and there is n time to build rapport to collect relevant data
Inaccurately or poorly written interviewer / resp instructions can result in response and recording errors
Poor layout can put respondents off starting or completing qnn and errors in recording and data processing

Question 4

Validty and types

Answer 4

Internal validity refers to ability of specific measures or questions used in research to measure what they claim to measure, in qnn design. There are three types of this measurement validity:

Construct validity - about what qn is measuring, and how it was constructed.
Content validity - suitability of qn to measure concepts it claims to measure. More subjective than construct validity
Criterion validity: how well a new measure or question works in comparison to a well-established one, or how well qn works in relation to other qn that are considered meaningful measures of characteristic or attitude being studied

Question 5

Whats reliability

Answer 5

Consistency of research results. Perfect reliability relies upon the same conditions pertaining each time it is repeated. Extent of which questions will produce the same result when repeated under same conditions. In designing qnn, briefing and training interviewers in how to administer it, it is important to bear in mind reliability

Question 6

Methods for assessing reliability of questions:

Answer 6

Test / retest method: test and retest on the same subjects in the same way. Problems with this method are that resembling same sample and same conditions is not always possible (something may have occurred that leads respondent to change their views)and asking same questions on same respondents on more than one occasion may cause respondents to lose interest, with result that their responses differ or may recall answers from original tet and repeat them due to that
Alternative forms methods: two different but equivalent versions of q are administered simultaneously to the same people.responses are examined to determine whether two measures are correlated. High correlation would show that two measures are measuring the same thing. Designing an equivalent question is difficult
Split-half method: does not assess stability of a question or a Q over time but rather assesses internal consistency of research. Splits sample into two matching halves and applies alternative measures to each half. Results from each are checked using a correlation technique.

Question 7

Questionnaire design and the respondent

Answer 7

The questionnaire follows a conversational template where the respondent should be a willing, interested and able participant. Questionnaire should facilitate this process, by thinking about how to begin the conversation (intro), what words to use, how and in what order to present the topics and questions and how to bring it to a close.

Question 8

Importance of intro

Answer 8

Establishes sound footing for interaction by engaging respondent interest and attention right away. Also has an ethical role in establishing ‘ground rules’ for qnn, key ethical and professional code of conduct issues relating to anonymity and confidentiality, voluntary participation and informed consent (including transparency) and no harm to participants.

Question 9

qnn intro MRS COC

Answer 9

Where lists of named individuals are used e.g. client databases
If there is to be any recording, monitoring or observation during interview respondents must be informed at recruitment and beginning of interview
Must not be misled when being asked for co-operation to participate
Right to withdraw from project at any stage must be respected
Respondents must not be unduly pressurised to participate
Ensure that all of following are clearly communicated:
Name of interviewer and interview identity card shown if F2F
Assurances that interview will be carried out according to MRS COC
General subject of interview
Purpose of interview
If asked, likely length of interview
Amy costs likely to be incurred by respondents

Question 10

qnn intro - what should be included?

Answer 10

Purpose and nature of research and the general area or topics under investigations/ exact purpose of research may need to be disguised, and therefore shouldn’t explain precise objectives but honestly explain broad subject matter. Must in no way mislead respondent otherwise breaching MRS COC
Whether the interview is to be recorded, monitored or observed. MRS COC says that respondent must be told this when recruited and at beginning of the interview
How long qnn will take to complete, in order to achieve informed consent of time commitment. A lack of transparency about this has been found to impact both quality of data collected and respondents attitude to research
Why the respondent was chosen and how. If used list of named individuals to generate sample e.g. database, then source must be stated
Name of organisation conducting research and its contact details and if interviewer is involved, name of them
Assurance that information respondent provides will be treated confidentially
That respondents participation is voluntary and that they can refuse to answer any questions or withdraw at any time, and if they wish, all or part of information they give will be destroyed at once
Online surveys must state organisations privacy policy
If prize draw or incentive is offered link to rules must be provided

Question 11

qnn close - use and what to include

Answer 11

As an interview is a conversation of a kind, it is important to bring it to a close properly. It is common in online surveys to include a progress bar which allows respondents to know when they are approaching the end of the survey,. Good practice to end on an open-ended question; allowing opportunity to offer final comments on topics covered / survey itself. If you will need to recontact respondent you must ask permission to do so for the purpose of research.

Where appropriate you may want to inform respondents about what happens next and reiterate how data will be used / stored and necessary contact details. Finally, I should include a note of thanks.

Question 12

Qnn design and perception of research

Answer 12

Questionnaire and interviewer who administers it are ambassadors for the research industry. Interviewers should never be in a position where they have to administer a poorly designed qnn. With declining response rates it is more than ever on the researcher to prepare qnn that is clear and easy to understand and easy to administer or fill in. research experience should serve to bolster the credibility of the research industry and high standards and professionalism it espouses. Effective qnn design can help to ensure that we do not ‘spoil the field’ for future research.

Question 13

Contribution of good design by stage?

Answer 13

Data quality
Delivering valid and reliable data
Minimisjng non-response (encouraging and maintaining participation)
Minimising error - question error, response and recording error and data processing error
Interviewers task
Making task as straightforward as possible
Minimising questioning and recording errors

Respondent experience
Gaining and maintaining interest in and willingness to participate
Making it an enjoyable experience
Making as easy as possible
To analysts task
Making data processing and analysis accurate and efficient
To perception of research
Raising profile of research
Enhancing professionalism and credibility of research
Increasing goodwill of general public towards research

Question 14

Purpose of qns?

Answer 14

Purpose of qnn is to collect valid and reliable data that can be used to address the research problem. First task in designing questionnaires (or DG) is to clarify objectives, information requirements, and agree what exactly qns need to measure.
Effective questionnaires cannot be designed without clarity about what information it has to deliver.

Question 15

Overview of qnn design process

Answer 15

Aim qnn design is to convert research objectives into meaningful questions and assemble these into effective and workable qnn. Stages are:

Clarifying what questions need to measure
Wording questions
Deciding on types of questions and response format
Putting questions into effective logical order
Designing layout
Testing draft version
Revising draft and agreeing final version

Question 16

Overview of considerations to be made before designing process

Answer 16

Qnn should follow on from thorough and rigorous examination of research problem and clear understanding of nature of the vidence needed to address it. Decisions about question context, wording and order are the end result of a process that considers the following:

Research problem - background, definition, objectives, use of data
Evidence needed to address this - exploratory, descriptive or explanatory
Ideas, concept and variables to be measured - definitions and indicators
Appropriateness of data - qual / quant
Suitable method of data collection - observ, interviews (self-completion or interviewer), F2F / phone / postal / online
Where data will be collected - respondent home, shopping centre, place of work
Responses be captured - pen and paper, computer and audio / video recording
Constraints - time and budget
Analysis - computer and manually

Question 17

Standard questions

Answer 17

Questions determining eligibility to take part in surveys and characteristics / circumstances of those who do are necessary. In consumer social surveys these classification Qs may be age, social grade, housing tenure, gender. In B2B these may be type of org, job title, number of employees. Using standard or consistent Qs makes qnn reparation easier and gives standardised format allowing comparison between surveys. It is also essential should you wish to combine or fuse data.

Question 18

Screening and eligibility

Answer 18

Client confidentiality or reasons of certain groups not being representative of the target group may mean you want to exclude them from the sample. These are done through classifying respondent by demographics that represent either group.

Question 19

Clarifying the meaning: concepts, definitions and indicators

Answer 19

Being clear about what is being measured (concept or variable) means agreeing a definition of concept or variable. This should occur before qnn design process begins so that it is clear what questions need to be measured.

Question 20

Concepts and conceptualisation

Answer 20

You will need a nominal or working definition of a fairly abstract concept and specify a set of indicators of it e.g. sexism. The processing of moving from abstract to concrete is known as conceptualisation.

Question 21

Definitions

Answer 21

You may arrive at working definitions by using formal / informal research by asking groups what concept means to them. You may also check definitions others have used via a search of secondary sources. Outcome should be a clear specification of exactly what it is you are going to measure with the question or set of questions. Nominal definition of sexism might be ‘the view that one sex is inherently superior to the other and/or particular roles / tasks are suited to one sex or the other.

Question 22

indicators

Answer 22

Once a clear definition of concept has been agreed the next step is to develop a set of indicators that will be used in designing questions / set of questions to measure concept. In making these decisions you should refer back to research objectives and question why you are interested in measuring relevant concepts. A review of relevant literature and/or exploratory research can be useful.
The next step is to think about how to interpret responses to indicator questions; what patterns of response should be interpreted to measure incidence of concept.

Question 23

The process of conceptualisation summise

Answer 23

You should now be aware of what you want to measure and want to know how best to word questions. Concepts and variables identified should be turned into meaningful, objective questions that measure what we want to measure. These questions should be ones that respondents are willing and able to answer.

Question 24

what to ensure so that Qs provide valid and reliable data?

Answer 24

Questions should be worded to ensure valid and reliable data, this means ensuring:

It measures what it claims to measure
Relevant and meaningful to respondent
Is acceptable to ask respondent
Respondent (and interview) understands it
Is interpreted in the way in which is intended
Interpreted same way be all respondents
Elicits accurate and meaningful response
Meaning of response is clear and unambiguous

Question 25

Effective question writing rules?

Answer 25

Use simple, straightforward grammar - activer rather than passive voice, simple rather than complex sentences
Avoid convoluted questions
Avoid negatively phrased questions (leading Qs)
Avoid double barrelled question
Keep questions as short as possible - 20 words
Use simple, everyday words
Avoid jargon and technical language unless it is suitable and uniformly understood by target population
Use precise, specific rather than general abstract terms. Where appropriate illustrate what you mean with example or provide clear and precise definitions of concepts / terms used in question
Specify reference period - taking account of degree of precision required in relation to research objectives, type / usage or behaviour you’re asking about and what the respondent can be reasonably expected to remember
Give respondents precise quantifiers e.g. never, once a month, several times a week, once a week…
Avoid hypothetical questions about respondents likely future behaviour - provides meaningless, hypothetical data
Include material that allows respondents to think themselves into situation you are asking about to get more meaningful data
Don’t embed assumptions into question wording e.g. how often do you travel to France
Do not lead respondents towards a particular answer e.g. isn’t green the best colour?
Ensure that any definition or description of a topic, concept or term used as well as instructions for interviewer or respondents are full and complete. Neither respondent / interview should have to rely on own interpretation, or in case of interviewer have to rely on their words to explain any part of Qs

Question 26

Questions on sensitive topics

Answer 26

If not clearly worded or in the right place on qnn, question and answer recorded without embarrassment on part of the interviewer, sensitive Qs can lead to refusals to answer / continue with the interview / take part in the first place. What is judged to be intrusive / embarrassing / sensitive varies within certain cultures, including money, voting, religion, sexual activity, criminal behaviour and use of alcohol / drugs. Respondents can be asked to fill in questionnaires themselves on screen or in separate self-completion sheet F2F. alternative cards can be shown in which respondent reads out a code for their responses. Anonymity of telephone interview / online survey often makes these approaches unnecessary.

Question 27

Social desirability bias and prestige define

Answer 27

Here responses do not accurately gather respondents actual behaviour, attitudes or opinions for several reasons. Respondents smay choose an answer that is socially desirable - one that is viewed more favourably by society rather than one that actually applies to them. Over-reporting occurs in Qs on being a good citizen, well-informed / cultured person, fulfilling moral and social responsibilities e.g. using your vote or environmental behaviours. SDB may surface as respondent not wanting to appear uninformed about the topic under question so rather than respond DK they give a response.

Prestige bias affects answers to questions about age, occupation, income, cleanliness and grooming. Under-reporting occurs in relation to issues such as alcohol, drugs and sexual activity.

Question 28

How to mitigate SDB

Answer 28

Questions therefore must be designed to be as easy and painless for respondents to give low prestige answers as it is to give high prestige answers. This can be done by using self-completion, using show cards as questions about sensitive topics, ensuring questions presented in a way that all answers are allowable and equally acceptable or to offer respondents a valid escape route. As with sensitive topics, more anonymous methods of data collection e.g. telephone and self-completion may be better suited to collecting this type of information.

Question 29

how to prevent boring participants

Answer 29

It is important to maintain respondents’ interest throughout whole questions - this means avoiding repetition in the form of banks of similar questions, lists of tick or click box questions can bore respondents leading them to answer automatically without any thought. Boring participants can also be done by asking irrelevant or unnecessary questions. Every question should be relevant to research objectives and relevant to respondents’ situation or experience and on a subject they can be reasonably expected to answer accurately. If a question is irrelevant to a particular subset of respondents then routing instructions should be used to ensure they are not asked the question.

Question 30

Importance of a good translation and how to conduct this

Answer 30

May have to translate questions from original language designed in. it is therefore important to ensure words used mean the same thing in all languages used, thus it is necessary to language but also the wider cultural context around it, and the cultural context around the context of research topics within each country. This understanding should allow you to find words, phrases that give the meaning you want. If possible, have a speaker who is living in or has been recently living in the country and whose first language it is to do the translation. Also important to check conventions on using scales (may be interpreted in different ways in different countries) and asking demogs (social grade varies).

Back-translation - retranslating into original language is advisable, particularly where consistency and comparability across counties is important. Even when words or phrases are back translated they miss the meaning of original; may be that there is no word in the language for something that needs to be translated.

Question 31

what is an open Q? how used?

Answer 31

In open / free response questions the respondent gives response in own words. This can be given verbally e.g. phone or F2F, or self-completion e.g. postal, online. Responses to open Qs in interview-administered surveys can be ‘pre-coded’ and listed on qnn where the interview will record responses that correspond to respondents’ answers. If the answer is not on the list, the interviewer should record it under ‘other’ which should be accompanied by ‘write in’ or ‘specify’ instructions.

Question 32

adv / disadv open Q?

Answer 32

Main advantage of open Qs is that they make respondents feel more at ease and in control - the interviewer or researcher behind the qnn wants to know exactly what they think and is not making them select a pre-formulated response. For this reason it may be useful to include early in qnn or the start of a new topic to build rapport. Also allow you to see a wide range of responses rather than limited ones you might get using prompt response qn, you also then have those responses in words used by respondent. Also offers chance to probe for more detail in personal interviews and are easier to word than closed questions.

Disadvantages

Require more of the respondent, interviewer and data processing provider and so are more time consuming and costly to use
Respondent has to articulate response - which may be difficult for certain topics
Interviewer has to record responses word for word where detailer meaning can get lost in process (respondent not wanting to write or type something in full may shorten sentences or abbreviate words or interviewer may not be able to write or type as fast as respondent talks
Data processing department has to build a code frame which can be difficult and expensive

Question 33

Closed qs - define adv / diadv?

Answer 33

Offers choice of answers - they may be visible, read out or shown on prompt card by interviewer, or asked to tick or click, underline or circle in self-completion qnn.

They are relatively easy to administer, take up less time than open and do not involve interviewer / respondent in recording detailed responses. They also make data processing tasks relatively easy. Main disadvantage is that they can be difficult to formulate well and all-encompassing of opinions that respondents may want to give - this results in poor-quality and potentially non-representative data of opinion. Loss of sensitivity in measurement - what respondents actually said or thought is not recorded and there is no way of analysing ‘real’ response. Too many closed Qs in succession can be boring and repetitive for the interviewer to ask and respond to answers which may have a negative impact on data quality.

Question 34

Forced completion and its impacts

Answer 34

Forced completion can eliminate or reduce missing data, but raises questions about the ethical principle of voluntary participation and respondents right not to answer questions if they choose. Thus it should be avoided - should not be of detriment to survey since forced completion typically has negative impact on survey completion rates.

Question 35

response scales and what to consider?

Answer 35

Response scales are a form of closed questions used to measure attitude, preference, likelihood to buy and satisfaction. Choice of scale and response format will depend on information requirements, level of sensitivity needed in measuring issue under investigation and suitability for the method of data collection. Things to consider:

Where to included a DK option
Common in interviewer administered surveys not to offer options but record if it is given as an answer. If the respondent hesitates over response the interviewer should not force them to choose an option but should record DK. MRS COC states that respondents should be able to provide information in a way that reflects the view they want to express inc DK and PNTS where appropriate. Thus good practise to include DK and PNTS in self-completion surveys
Number of options to include in scale
Desirable length should be between 5 to 8 with a mid-point or neutral options as it slightly increases validity and reliability of scale
Whether to label all items in response scale and what to label them
Numeric / verbal and whether to use unipolar (from 0 to 10) or bipolar (-5 to +5) and whether to label or only some of them. Typically, numeric scales should be unipolar with matching verbal scales at each scale. Should also have all numeric values attached.
Order / direction to place scale options.

Question 36

Probing and prompting ?

Answer 36

Follow up questions that sometimes accompany open Qs. Purpose to obtain a more detailed or fully considered answer, typical probes are ‘what else?’, ‘why do you say that?’ and should be included within the qnn or script so the interview is clear about when and how to apply them. Important for reliability of data that each interviewer applies and asks them in the same way. Prompts are used to elicit responses to closed qns and can be read or shown to respondents.

Question 37

questions to measure attitude and using attitudinal statements

Answer 37

Difficult to design Qs on attitudes, which are not factual, in a way that achieves both validity and reliability. Attitudes are complex and difficult to research. In setting out to design research and craft questions that gather data on attitudes it is important to be as clear as possible what is needed to know to address research objectives. It is important to research attitudes towards specific events and not attitudes to generalities. Also important to research current attitudes. Capturing the essence of an attitude is almost impossible by using one Q or statements; unlikely to capture complexity of attitude so will lack validity, and unlikely that it will deliver consistent results (sensitivity of respondents to wording) so will not deliver reliability. Using a bank of questions or attitude statements combined in an attitude scale can be used to increase V&R. validity by encompassing the complexity of attitude, and reliability because issues of question wording and context may be cancelled out across a range of statements.

So designing questions to gather data on attitude consists of two parts - designing and choosing a list of attitude statements or ‘item pool’ for a particular attitude variable and choosing the response format.

Question 38

Designing, evaluative and descriptive attitude and belief statements how to?

Answer 38

Attitude statement is a single sentence that expresses point of view, belief, preference, judgement or emotional feeling, position for or against something. List of statements included in an attitude scale should be grounded in an in-depth understanding of the subject area. Study of previous research or literature review are good starting points. Depth interviews / group discussions among survey’s target group are invaluable, they allow us to examine nature / complexity of the attitude and help to determine exactly what it is that is needed to be measured, what indicators should be and understand language respondents use to express attitude in question.

Once belief statements are generated you should ask following:

Is each statement clearly worded
Is each statement unambiguous 
Any statements too long?
Each statement contain one issue only
Is list balanced - roughly equal numbers of pos / neg items
In random order

Question 39

Item analysis

Answer 39

Item pool generated from review of relevant literature, qual research and pilot testing should offer a valid measure of attitude in questions. It is needed however to check that this is the case when conducting item analysis. This helps to determine which statements are the most valid measures of attitudes and thus are best ones to use in scale. We have no external, measurable output of the attitude against which o assis each of the attitude statements, so have to examine how well each individual item correlates with the rest of items in the pool based on the assumption that the whole item pool is the best measure of attitude in question.

The ‘item whole’ correlation is calculated - correlation strength of association between each item and rest of items in the pool done via statistical or data analysis package. Items that correlate poorly with rest or pool are excluded from use on scale on basis that they do not measure attitude measured by other items.

Question 40

Likert scale response format?

Answer 40

Main concern for choosing items in likert scale is that all items should measure aspects of the same underlying attitudes - should be unidimensional. Neural items and those at the extremes of the attitude continuum should be avoided. List of statements may have some positively phrased items along with some negatively phrased items - these should be consistent in how they are scored and analysed. In the scoring attitude scale to suit you may have to make changes at the data processing or analysis stage. Once you have decided on attitudinal statements and response format, pilot the Q and examine respondents reactions to each of the statements. Check whether they answer extremes, neutrals or DKs. this information should tell you whether your attitude statements are working or not. You should begin to see patterns, with respondents falling into different groups according to their responses.

Question 41

positive linear scales?

Answer 41

If you are for instance scoring favourable attitude statements from 5 - strongly agree, to 1 - strongly disagree (known as a linear scale). The possible scores on this attitude scale are 50 if there is 10 Qs. respondents score therefore indicates strength of attitude to particular subject or variable.

Question 42

semantic differentials

Answer 42

A semantic differential is a 7 point bi-polar rating scale (some are 10 point scales), with extremes of scale denoted by adjectives that are opposites in meaning e.g. strong weak, masc fem. Respondents will be asked to rate a series of objects e.g. brands using the scale. It is important that elements of scale are carefully chosen - pilot work (review of 2ndry research, qual exploration and quant testing to determine relevant factors) is useful in this regard. It is also important to ensure adjectives used to describe ends of scale are really opposites. Statements should be rotated or randomised in order to avoid order bias. Ratings for each object can be averaged across samples and can be used to compare perceptions held by different types of respondents of a particular object (brand / service / organisation).

Question 43

ranking qs

Answer 43

Can also measure opinion / attitudes to object by asking respondents to rank a set of attitudes or opinions relevant to object. By ranking we get an idea of the way in which a person evaluates an object on a set of criteria. One important thing to bear in mind is that we cannot say anything about distance or intervals between rankings - it is an ordinal scale. In constructing a ranking question we must take care to ensure that instructions are clear and unambiguous, so respondent is clear about the basis on which to compile ranking, and list of items to be ranked should be limited at 10 - any more makes it difficult to manage for respondent and researcher. Criteria on which we ask respondents to rate an object must be meaningful. For completeness it is important to include a DK and Option. Average rank the scores across sample and how many first place rankings a particular criterion received, how many got second place and so on.

Question 44

paired compariosn

Answer 44

Form of ranking in which respondents are presented with two objects and asked to choose between them. This is used in product testing often. To get a rank order measurement from a series of objects, we must present each pair combinations to the respondent. In designing any rating scale the guidelines that pertain to question wording should be followed. Particular attention should be paid to wording of instructure, to ensure that they are clear and easy to follow. Relevant information should be given as to the context of the required rating. The rating criteria or attitude statements, element of the scale, should be relevant to object being rated, should mean the same thing to all respondents and should be within the respondents frame of reference. Response categories should be relevant to purpose Q - likert, semantic differential or tank order.

Question 45

problems with scales

Answer 45

Error of central tendency - tendency for respondents to avoid using the extremes of scales. This can be mitigated by ensuring that extremes do not appear too extreme by combining the two top categories at each end of the scale at the data processing stage.

Halo effect - in responding to items on a scale a like / dislike for the object being rated may influence respondents rating. This may be overcome by designing qnn so that the rating scales are spaced apart. Another manifestation of the halo effect is an automatic response syndrome which can occur if the scale eis laid out in such a way that all positive scores line up on one side of the page and negative line up on the other or if all statements / items in scale are pos / neg. If the respondent notices this pattern there may be temptation to reply automatically without really thinking about the answer. Solution is to include both pos & neg statements.

Logical error occurs when respondents give a similar rating to an object on attributes or attitudes that they think are somehow related. To overcome this you should ensure that such attributes or attitudes do not occur close together on the scale.

Question 46

Grids? what are they? why use?

Answer 46

If you want to understand how respondents describe or evaluate a product, service or brand - useful in understanding how consumers perceive the market / brand and what effect marketing activity has on perceptions of brands, rating products or services against a set of criteria can be useful. An association grid, which allow the respondent to choose which statements are associated with particular brands, is a useful way of collecting a lot of info quickly and allows scope for analysing data in a variety of ways, from calculating proportion of sample who associate a particular statement with a brand or product, through comparisons of the profiles of each brand across all statements to more complex multivariate mapping techniques.

Question 47

How to curate grid qns?

Answer 47

To measure attitude towards an object, product or service, the first step is to develop a set of evaluative or descriptive statements to reflect attitudes / beliefs about the object. Descriptive attitude statements can relate to particular properties of, perhaps those that have been emphasised in marketing / advertising activity. Evaluative attitude statements relate more to opinion or attitude based characteristics such as ‘reliable’ ‘good quality’ ‘suitable for children’.
Being choosing statements it is important to be clear about what it is you are measuring and the purpose to which the findings will be put. The end use of data should determine the choice of criteria; statements that distinguish between products may not necessarily be the same as those that are used to make preference or purchase decisions or those that underlie attitudes. Determine the relevant or salient beliefs about or characteristics of a product, service or brand. Using these will help write much better attitude and belief statements and will give a more sensitive understanding of the market. One way to do this is qual research - obtain a list without prompting of characteristics and attributes of brand, product or service and use it to develop evaluative or descriptive statements.

Question 48

gird pos / negs

Answer 48

Quant methods are particularly suitable for collecting data on attitudes when less detailed understanding is required. If measures used are grounded in solid qual work it is likely that they will be reasonably valid measures. If well designed they can produce reliable (replicable and consistent) measures which can be used in statistical analysis e.g. cluster and factor analysis. In the course of developing attitude there is a tendency to oversimplify; risking loss of richness and detail and understanding the nature of attitude. Using scales and rankings can mislead us into thinking that attitudes fall on a continuum with pos at one and neg at the other which may not be a useful or valid way of thinking about attitudes.

Main concern for choosing items in likert scale is that all items should measure aspects of the same underlying attitudes - should be unidimensional. Neural items and those at the extremes of the attitude continuum should be avoided. List of statements may have some positively phrased items along with some negatively phrased items - these should be consistent in how they are scored and analysed. In the scoring attitude scale to suit you may have to make changes at the data processing or analysis stage. Once you have decided on attitudinal statements and response format, pilot the Q and examine respondents reactions to each of the statements. Check whether they answer extremes, neutrals or DKs. this information should tell you whether your attitude statements are working or not. You should begin to see patterns, with respondents falling into different groups according to their responses.

Question 49

process of reviewing qns? overview

Answer 49

Reviewing questions

Once set of questions have been designed it is useful to review them against relevant research objectives and if necessary amend them. Should ask:

Does it give info you want
Doe sit reach research objectives
Is the purpose of the question clear
Is it necessary 
What assumptions have i made in this question 

In addition check whether Qns are suitable for the target group, for method of data collection and for how data are to be analysed.

Question 50

reviewing qns - target group?

Answer 50

Review wording of questions to ensure that vocabulary used is suitable for respondents; review response format to ensure that respondents will have no difficulty answering qns, check to make sure that Q&As make sense

Question 51

reviewing qns; method of data collection?

Answer 51

To prevent confusion, misunderstanding it is best to avoid long questions, scales and long descriptions - particularly in phone interviews where respondents cannot see the interview and qnn prompts and scales must be read out.

For self-completion methods such as online, much depends on how it looks. Thus it must be visually appealing and reflect professionalism of research org. With no interview present to clarify the meaning of Qs and instructions, the qnn must look easy to fill in and be easy to fill in. typically most qns will be pre-coded - to make the process easy. Questions and instructions must be written in clear unambiguous language; routing must be easy to follow. Topics should be interesting and relevant to participant due to lack of interviewer to establish and maintain interest. If they are not respondent may not complete qnn or ask someone else who they think will be able to answer qns.

Question 52

order of questions - importance and why

Answer 52

Now that you have set of qns that address research objectives, and a suitable response format for these questions - they must be put into an effective and logical order. Interviews are a form of conversation so should be easy to follow in order to keep interest and cooperation; no jarring non sequiturs or illogical jumps between topics. Qnn should create a positive impression of particular pieces of research and research generally. Order of topics and qns are important in establishing and building rapport with respondents even when no interviewer is present. Asking sensitive or difficult qns too early can destroy rapport and lead to withdrawal from interview or refusal to answer specific qns. Respondents may not feel comfortable enough to give accurate replies so data quality is compromised.

Question 53

order of questions - how

Answer 53

Useful to draw up flow charts from a list of draft qns and group qns that relate to each topic. Each group or set of questions is a module; these are then ordered into straightforward, non-challenging topics first, more difficult or sensitive topics, inc classification qns on age, income and so on towards the end (ensuring that personal data collected is relevant and non-excessive). To help flow you may want to include an intro to each module - brief! In terms of flow you should also consider balance between types of questions - too many of the same can be boring and repetitive and will adversely impact data.

One decided order of modules, the order of Qs within them should be decided; moving from more general qns to more specific ones - the funnel approach. Difficult or sensitive qns should again appear later on. Earlier qns may bias responses to later ones.

Question 54

layout and appearence - self-administered

Answer 54

Has effect on completion rates and on quality of data collected.

Self-completion should be laid out in an engaging way to maintain interest. This means visually appealing, such as headings & signposts, progress bars as an indicator (motivate respondents to get to the end of qnn). Due to no interviewer must be laid out in a way that the respondent understands swhat is needed (instructions and clear / unambiguous Q&As), easy to read and accessible (text size) and easy to fill in. This means testing surveys on all sorts of devices to ensure that participants don’t have to scroll too much and can be off-putting and so decrease completion rates. May have to design versions for each device. You can use ‘one page layouts’ or limit numbers and include a continue or next button to take respondent to next page.

Main adv of one-page layout is that it is quicker to fill in; but respondents can go back and change answers in light of later questions, you also have fewer routing options and filtering. Multi-page must decie on # of qns on each page, which should be guided by content of qnn (modules on each page). A greater number of pages means a greater number of clicks and so greater amount of time so should be limited as much as possible. Must considera allowing ‘back’ button as whilst gives resp a larger degree of control it allows them to correct or amend answers.

Question 55

layout and appearence - interview administered

Answer 55

Interview-administered qnn must be set out so that the interview can read easily, follow routing and record answers accurately. Interviewer questions are typically included in caps / bold / different colour text so that interviewers can easily distinguish their instructions and what they should be reading.

Question 56

Qnn length

Answer 56

Should be long enough to cover research objectives, meet research budget, suit choice of data collection method, allow time to build rapport needed to address relevant topics. Should not be long enough that it bears the burden of the respondent or so long that respondent does not want to partake at all. Besides cooperation rates, length of qnn may impact quality of data collected, with poorer quality towards to the end due to survey fatigue = high cognitive load = lower quality answer.
Recommended max length for in home F2F interview is 45 mins to an hour, online survey is about 30 minutes, telephone interview is about 20 minutes, street interview 5 to 10 minutes.

Question 57

Checking the qnn

Answer 57

In finalising qnn, have it checked thoroughly by fieldwork experts and by people involved in data processing. Should be proofread to ensure that routing and coding instructions are clear and accurate and that there is enough space to record and code answers.

Question 58

Conducting pilot study

Answer 58

Can be conducted at any stage of development - from conceptualisation (to explore meanings of concepts and understand language used by target aud) to fully developed draft (check it drivers information it is designed to deliver, to check there are no software glitches, routing works, no logic errors and so on). Style of pilot study will rely on how developed qnn is; pilot interviews undertaken in early stages of development might take the form of informal qual in-depth interviews. Those undertaken with more fully formed qnn are likely to resemble formal quant interviews.

For F2F and telephone surveys, interviewers from fieldforce conducting surveys should do some pilot interviews from the final draft. It is invaluable to get feedback from experienced interviewers as well as relatively new ones - each will have a different view of the interview process and effectiveness of pilot qnn. New interview will have insights into how it works from respondents point of view; both will give feedback on timing and overall manageability of the interview.

People involved in designing qnn or discussion guides should conduct or sit in on some pilot interviews - this allows development of qnn design skills and you hear and see how qns work with real respondents. If full-scale proper study is not possible, ask someone you work with who are not directly involved in the project to do pilot interviews with / for you.

Useful to conduct pilot interviews F2F with target pop, regardless of whether final version is designed for telephone or self-completion. F2F interview enables the interviewer to observe and note respondents non-verbal reactions to questions. After this you can pilot test in medium in which it will be delivered. Survey / qnn; 12 interviews should be given, or 3 / 4 pilot studies for discussion guide.

One approach is to go as you would in a real interview, and then go back to comment on each question as it is asked. May even give respondents a copy of research objectives for evaluation of questions. Useful to record pilot studies and listen to them again.

Question 59

Reviewing and revising post-pilot

Answer 59

Once complete, useful to think through how you would analyse responses. Check data against ROs to see whether you are getting the sort of information you need/ for quant projects, worth preparing a coding frame based on responses to questions, editing qnns, entering data and producing hole count. Allows to check for inconsistencies in logic or coding that might hamper data processing. Make the necessary changes to qnn that pilot work suggests. If they are substantial it may be worthwhile conducting another pilot study with a new set of respondents.

Things to look out for:

Clarity of purpose - did it measure what it was supposed to, was it interpreted in way you anticipated, were questions relevant and meaningful
Wording of questions - were they understood? Any unfamiliar words or misunderstandings?
Question content - were resps discouraged by any questions / embarrassed a=or prefer not say / skipped?
Order of qn modules and questions within - did it run smoothly? Did order seem logical to interviewer / resp
Layout / appearance - was it suited to data collection? Were instructions clear and easy to follow? How did it look on resps device?
Length - how long it took to finish? Interview / resps perceptions of length?