Set-up good research Flashcards
What are Sources of Variability?
Precision of Measure: variability due to imperfections in measuring tool
Interrater Reliability: variability due to differences in judgments of observer
Margin of Error: variability due to individual differences within a sample
What is test-retest reliability?
- Similarity of result if test is repeated
* Best for assessing stable characteristics
What are problems of test-retest reliability?
- Remembering of questions
- Longer interval between tests needed
- Change in participant between tests
What are alternate forms of reliability? Problems?
- “Equivalent” questions are asked
- Can be administered after each other
Problems:
• Equivalence of questions
• Change in participant between tests
What is split-half reliability?
Problems?
- Test and retest administered in one sitting, with alternating items
• Results then scored individually
Problem:
• Equivalence of questions
What is accuracy of measure?
• Comparing to a standard
• Biased if scores differ on average from
standard
What is the observer bias?
Solution?
- May arise of researcher has a desired outcome in mind
* Solution: Blind observers (Researcher do not know condition of participant during experimentation)
What are different types of validity?
-Face Validity: test seems to test what it is explained to test. Perceived legitimacy of the test can influence participants’ attitude
-Content Validity: test actually tests what it is designed to test.
-Criterion-Related Validity:
• Concurrent: agreement with “golden
standard”
• Predictive: successful prediction of future
behavior
-Construct Validity
• Convergent (similar to instruments measuring
the same construct)
• Divergent (not similar to measure of a
different construct)
Why take samples?
To make general statements about the
population, but it is only possible to ask a limited number of people
What is important to keep in mind with samples?
- How many is enough
* Is my sample biased
What is a simple random sample?
A group of people is randomly chosen from the whole population.
• Solves problem of bias
• In reality hard to manage to get a truly random
sample
What is a stratified sample?
• Population grouped (gender, nationality)
• An equally sized random sample from each
group is taken
Example: Taken two random sample from male and female population
What is proportionate sampling?
• Variant of stratified sampling
• Random samples from each group (strata) are
proportionally sized in respect to population proportions
Example: Population: 60% Dutch, 20% German and 20% Belgian. The sample of 20 will be drawn 12 Dutch, 4 German/Belgian
What is systematic sampling?
• Often combined with stratified sampling
• Every k-th element is taken
Example: Every 10th student on a list will be chosen
What is cluster sampling?
• Natural occurring groups are identified
• Several groups are chosen randomly
• Can be combined with other sampling
techniques (Multistage Sampling)
Example: From 40 first year student tutorials 4 are randomly chosen to take part in a survey
What are methods of quantification? observational research
- Frequency
- Duration
- Interval
How to simplify sampling?
- Time: alternate observing and recording
- Individual: one subject at a time
- Event: one behavior at a time
- Recording: taping the situation
What are naturalistic observations?
• High external validity (compared to laboratory
experiments)
• No causal links can be established
2 types of design of questionnaire
• Open-Ended Items: more detailed, but time
consuming to analyze
• Closed-Ended Items: easy to analyze, but
loss of details
What are solutions for the nonresponse bias in surveys?
• Solutions: Several contacts (prenotice letter, thank you postcard, replacement questionnaire), small reward • Non-factors: length, personalization, anonymity, deadline
What is the social desirability effect?
• People tend to alter answers if they feel
evaluated/watched
• Less of a problem in online-surveys
Exam question:
A man stumbled out of the car that had just been stopped. Not able to walk straight and slurring his speech the police did not hesitate to conduct breathalyzer test. To their surprise it came out lower then they expected. They repeat the test and it came out high, so high in fact that the man should have died of alcohol poisoning. A each subsequent test gave a different result
It seems as if the breathalyzer test had
A. A low reliability and therefor a low validity
B. A low reliability but a high validity
C. A high reliability and therefor a high validity
D. A high reliability and but a low validity
Answer A
Exam question:
The police is performing a new lie detector test. Their key measure is the stress hormone cortisol level. They had the impression their suspect would be lying and stressed but this was not reflected the results. They decided to check if the skin conductance test, another measure for stress levels showed similar results.
What where they checking
A. Test-Retest Reliability
B. Face Validity
C. Divergent Validity
D. Convergent Validity
Answer D
