Questionnaire Design

Question 1

What type of research are questionnaires crucial for

Answer 1

Individual differences
Intelligence tests
Attitudes

Question 2

What ethics need to be met with questionnaires

Answer 2

Equal oppotunties and cultural biases

Question 3

What does the questionnaire measuresneed to be

Answer 3

Reliable and valid

Question 4

What is the process of creating your questionnaire

Answer 4

Question formats
Writing your questions
Clarity of questions
Avoid leading questions
Reverse wording
Response formats
Clear instructions

Question 5

Define an open formt question

Answer 5

asks for some written detail, but has no determined set of responses, e.g.,

“Tell us about the occasions when you have been academically vindictive.”

Question 6

What are the advantages of open format questions

Answer 6

Leads to more qualitative data

Question 7

What are the disadvantages of open format questions

Answer 7

Time consuming to analyse

Question 8

Define closed format. questions

Answer 8

Short questions or statements followed by a number of options

Question 9

What are the three ways you could based the writing of your questions on

Answer 9

Theoretical literature
experts
colleagues

Question 10

What could theorterial literature help with when writing questions

Answer 10

Ideas that appear in the theoretical literature should be used as a basis

Question 11

What could expects help with when writing questions

Answer 11

Recruit experts in the are to suggest items

Question 12

What can colleagues help with

Answer 12

Can help you generate more items

Question 13

Why is clarity of questions important

Answer 13

Respondents may concentrate on opportunity, resources, and ability to different extents.

Question 14

What is the problem with leading questions

Answer 14

Leads to respondents in a particular direction by potentially excusing the behaviour

Question 15

Why is reverse wording used

Answer 15

To ensure people are reading the questions properly

Question 16

What type of scales are Yes/No, True/False questions

Answer 16

Dichotomous scales

Question 17

What type of scale are “I usually don’t do this at all
I usually do this a little bit
I usually do this a medium amount
I usually do this a lot” questions

Answer 17

Frequency of behaviour

Question 18

Does strongly agree-strongly disagree count as a response format

Answer 18

YES

Question 19

Can numbers be used with a statement as a type of response format

Answer 19

Yes, numerical scales

Question 20

Are instructions important

Answer 20

YES

Question 21

How do researchers draw participants attention to the key part of the instructions

Answer 21

Underlining

Question 22

Which of the following is a dichotomous scale
Strongly Disagree  DisagreeAgree
True/False
 I usually don’t do this at all 
        1         2         3

Answer 22

True/false

Question 23

In the classical theory of error in measurement what does the observer score equal

Answer 23

The true score + error

Question 24

Where does the classical theory of measurement error value come from

Answer 24

Standard error of the measurement

Question 25

How does the classical theory help with measurement

Answer 25

it is universal of items

Question 26

Which items correlate to the true score

Answer 26

All items correlate to some extent

Question 27

What is reliability related to

Answer 27

The average correlation between items and test length

Question 28

What are the 4 types of relaibility

Answer 28

Internal External Inter-rater Intra-rater

Question 29

How can internal reliability be measured

Answer 29

Split-half reliability Parallel forms Cronbach’s Alpha KR-20

Question 30

How can external reliability be measured

Answer 30

Test-retest

Question 31

How can inter-rater reliability be tested

Answer 31

Kappa

Question 32

How does split-half reliability measure of internal reliability work

Answer 32

Split items into 2 half, e.g. odds v evens, randomly selecting correlate the total scores for each half

Question 33

What level of correlation in a split half reliability indicates reliability

Answer 33

.80

Question 34

How does parallel forms reliability measure of internal reliability work

Answer 34

Create a large pool of items Randomly divide Administer 2 tests to same participants Calculate the correlation between the two forms

Question 35

What is the problem with parallel forms

Answer 35

Difficult to generate the large number of items required

Question 36

How is Cronbach's alpha measured

Answer 36

Average of all the possible split half estimates

Question 37

To what value does Cronbach alpha measure

Answer 37

Values up to +1.00

Question 38

What figure does Cronbach alpha usually have

Answer 38

+0.70

Question 39

What indicates good reliability in the Croncbach's alpah

Answer 39

The greater the figure

Question 40

What does Kuder-Richardson Formula 20 (KR – 20) measure

Answer 40

internal reliability for measures with dichotomous choices (i.e., 2 choices Yes/No).

Question 41

What leave of values does Kuder-Richardson produce

Answer 41

1.0

Question 42

What figure indicates reliability with Kuder-Richardson

Answer 42

0.70

Question 43

Which test of internal reliability involves the creation of two different versions of a questionnaire?

Answer 43

Parallel forms of reliability

Question 44

How does test-retest reliability measure external reliability

Answer 44

Perform the same survey, with the same respondents, at different points in time.

Question 45

What do the results of test-retest show

Answer 45

The closer the results, the greater the test-retest reliability of the survey.

Question 46

What is used as a measure of the test-retest relaibility in external validity

Answer 46

The correlation coefficient between the two sets of responses is often used

Question 47

Define inter-rater reliability

Answer 47

Inter-rater reliability determines the extent to which two or more raters obtain the same result when coding the same response.

Question 48

What measures can be used for inter-rater reliability

Answer 48

Cohen’s Kappa: Values up to +1.00, larger numbers indicate better reliability, used when there are two raters. Fleiss’ Kappa: An adaptation which works for any fixed number of raters.

Question 49

What does the inter-rater measure

Answer 49

Agreement not accuray

Question 50

What does intra-rater reliability measure

Answer 50

The same assessment is completed by the same rater on two or more occasions

Question 51

How does intra-rater reliability work

Answer 51

Different ratings are then compared generally by means of correlation

Question 52

What is a problem of intra-rater reliability

Answer 52

Since the same individual is completing both assessments, the rater's subsequent ratings are contaminated by knowledge of earlier ratings.

Question 53

What are some of the sources of unreliability

Answer 53

``` Guessing Ambiguous items Test length Instructions Temperature, illness Item order effects Response rate Social desirability ```

Question 54

Which measure of reliability measures stability over time?

Answer 54

Test-retest reliability

Question 55

Which factors can impact validity

Answer 55

``` Faith Face Content Construct Convergent Discriminant Predictive ```

Question 56

Define faith validity

Answer 56

Simply a belief in the validity of an instrument without any objective data to back it up, and the evidence is not wanted

Question 57

Define face validity

Answer 57

If something has face validity, it looks like a test that measures the concept it was designed to measure.

Question 58

Define content validity

Answer 58

The extent to which a measure represents all facets of the phenomena being measured.

Question 59

Define construct validity

Answer 59

Seeks to establish a clear relationship between the construct at a theoretical level and the measure that has been developed.

Question 60

What is convergent validity

Answer 60

That the measure shows associations with measures that it should be related to, e.g., academic vindictiveness should be related to other aspects of vindictiveness; such as a tendency to seek revenge, or spitefulness.

Question 61

What is discriminant validity

Answer 61

That the measure is NOT related to things that it should not be related to.

Question 62

Define predictive validity

Answer 62

Assesses whether a measure can accurately predict future behaviour.

Question 63

Which measure of validity states that a test should represent all facets of the phenomena being measured?

Answer 63

Content validity

Question 64

How is a questionnaire reliable

Answer 64

A questionnaire is reliable if all of the questions in your test are consistently measuring the same underlying concept, and that this remains stable over repeated times that the test is administered.

Question 65

What makes a valid test

Answer 65

A test is valid if it is actually measuring what you intend it to measure.

Question 66

What is necessary but not sufficient for validity

Answer 66

Reliability