Set-up good research

Question 1

What are Sources of Variability?

Answer 1

Precision of Measure: variability due to imperfections in measuring tool
Interrater Reliability: variability due to differences in judgments of observer
Margin of Error: variability due to individual differences within a sample

Question 2

What is test-retest reliability?

Answer 2

• Similarity of result if test is repeated

* Best for assessing stable characteristics

Question 3

What are problems of test-retest reliability?

Answer 3

• Remembering of questions
• Longer interval between tests needed
• Change in participant between tests

Question 4

What are alternate forms of reliability? Problems?

Answer 4

• “Equivalent” questions are asked
• Can be administered after each other

Problems:
• Equivalence of questions
• Change in participant between tests

Question 5

What is split-half reliability?

Problems?

Answer 5

• Test and retest administered in one sitting, with alternating items
  • Results then scored individually
  Problem:
  • Equivalence of questions

Question 6

What is accuracy of measure?

Answer 6

• Comparing to a standard
• Biased if scores differ on average from
standard

Question 7

What is the observer bias?

Solution?

Answer 7

• May arise of researcher has a desired outcome in mind

* Solution: Blind observers (Researcher do not know condition of participant during experimentation)

Question 8

What are different types of validity?

Answer 8

-Face Validity: test seems to test what it is explained to test. Perceived legitimacy of the test can influence participants’ attitude
-Content Validity: test actually tests what it is designed to test.
-Criterion-Related Validity:
• Concurrent: agreement with “golden
standard”
• Predictive: successful prediction of future
behavior
-Construct Validity
• Convergent (similar to instruments measuring
the same construct)
• Divergent (not similar to measure of a
different construct)

Question 9

Why take samples?

Answer 9

To make general statements about the

population, but it is only possible to ask a limited number of people

Question 10

What is important to keep in mind with samples?

Answer 10

• How many is enough

* Is my sample biased

Question 11

What is a simple random sample?

Answer 11

A group of people is randomly chosen from the whole population.
• Solves problem of bias
• In reality hard to manage to get a truly random
sample

Question 12

What is a stratified sample?

Answer 12

• Population grouped (gender, nationality)
• An equally sized random sample from each
group is taken
Example: Taken two random sample from male and female population

Question 13

What is proportionate sampling?

Answer 13

• Variant of stratified sampling
• Random samples from each group (strata) are
proportionally sized in respect to population proportions
Example: Population: 60% Dutch, 20% German and 20% Belgian. The sample of 20 will be drawn 12 Dutch, 4 German/Belgian

Question 14

What is systematic sampling?

Answer 14

• Often combined with stratified sampling
• Every k-th element is taken
Example: Every 10th student on a list will be chosen

Question 15

What is cluster sampling?

Answer 15

• Natural occurring groups are identified
• Several groups are chosen randomly
• Can be combined with other sampling
techniques (Multistage Sampling)
Example: From 40 first year student tutorials 4 are randomly chosen to take part in a survey

Question 16

What are methods of quantification? observational research

Answer 16

• Frequency
• Duration
• Interval

Question 17

How to simplify sampling?

Answer 17

• Time: alternate observing and recording
• Individual: one subject at a time
• Event: one behavior at a time
• Recording: taping the situation

Question 18

What are naturalistic observations?

Answer 18

• High external validity (compared to laboratory
experiments)
• No causal links can be established

Question 19

2 types of design of questionnaire

Answer 19

• Open-Ended Items: more detailed, but time
consuming to analyze
• Closed-Ended Items: easy to analyze, but
loss of details

Question 20

What are solutions for the nonresponse bias in surveys?

Answer 20

• Solutions: Several contacts (prenotice letter,
thank you postcard, replacement
questionnaire), small reward
• Non-factors: length, personalization,
anonymity, deadline

Question 21

What is the social desirability effect?

Answer 21

• People tend to alter answers if they feel
evaluated/watched
• Less of a problem in online-surveys

Question 22

Exam question:
A man stumbled out of the car that had just been stopped. Not able to walk straight and slurring his speech the police did not hesitate to conduct breathalyzer test. To their surprise it came out lower then they expected. They repeat the test and it came out high, so high in fact that the man should have died of alcohol poisoning. A each subsequent test gave a different result
It seems as if the breathalyzer test had

A. A low reliability and therefor a low validity
B. A low reliability but a high validity
C. A high reliability and therefor a high validity
D. A high reliability and but a low validity

Answer 22

Answer A

Question 23

Exam question:
The police is performing a new lie detector test. Their key measure is the stress hormone cortisol level. They had the impression their suspect would be lying and stressed but this was not reflected the results. They decided to check if the skin conductance test, another measure for stress levels showed similar results.
What where they checking

A. Test-Retest Reliability
B. Face Validity
C. Divergent Validity
D. Convergent Validity

Answer 23

Answer D