Task 3 Flashcards
Dependent variable
• “Outcome variable” • Measurable • Influenced by change in condition Example: Is reaction time decreased by drinking coffee?
Independent variable
• “Condition”
• Directly altered by experimenter
Example:
Is reaction time decreased by drinking coffee (eg 1cup, 2cups, 3cups)?
Which question should you ask while thinking about the reliability of your experiment?
How similar would the result be if the experiment is repeated?
Which question should you ask by thinking about the validity of your experiment?
Does the measure actually measure what the researcher intends to measure?
Name 3 sources of variability
- precision of measure: variability due to imperfections in measuring tool
- Interrater Reliability: variability due to differences in judgments of observer
- Marion of error: variability due to individual differences within a sample
Test-retest reliability
- Similarity of result if test is repeated
* Best for assessing stable characteristics
Name 3 problems of test-retest reliability
- Remembering of questions
- Longer interval between tests needed
- Change in participant between tests
explain parallel/ alternate-forms reliability
-equivalent questions are asked
- can be administered after each other
2 tests with a bigger time difference
Name 2 problems of parallel / alternate-forms reliability
• Equivalence of questions
• Change in participant between tests
one bigger test at one time
Explain split-half reliability
• One bigger test is administered
• Equivalent questions are repeated
• Results of equivalent questions are scored
individually
What is the problem of Split-half reliability?
Equivalence of questions
What is the accuracy of measure by validity?
- Comparing to a standard
* Biased if scores differ on average from standard
What do you know about the observer bias? What is the solution?
- May arise of researcher has a desired outcome in mind
* Solution: Blind observers (Researcher do not know condition of participant during experimentation)
Name 4 types of validity
- Face Validity
- Content Validity
- Criterion-related Validity
- Construct Validity
Face Validity
test seems to test what it is explained to test. Perceived legitimacy of the test can influence participants’ attitude
Content Validity
test actually tests what it is designed to test.
Criterion-Related Validity
Compare your results with someone else
• Concurrent: agreement with “golden standard” (right now)
• Predictive: successful prediction of future behavior
Construct Validity
- Convergent (similar to instruments measuring the same construct) you compare to a other test from the same area and want to get similar results
- Divergent (not similar to measure of a different construct) you compare to a test of another area and want to have different results
What is internal validity?
- Ability of your research design to adequately test your hypothesis
- Showing that the variation in the independent variable caused the observed effect
Name 2 threats (Bedrohungen) of internal validity?
• Rival hypothesis: another alternative variable might explain the effect
Alternative explanations
• Confounding: combination of variables explains the effect and one cannot separate the effect of these variables
Name 8 sources of confounding (Störeinfluss)
- History: specific events other than treatment occur between observations (eg. 9/11)
- Maturation: fatigue or aging change performance over time
- Instrumentation: instrument calibration changes and confounds the effect
- Statistical regression: subjects selected for treatment on the basis of their extreme scores tend to move closer to the mean when retesting
- Biased subject selection: nonrandom
- Experimental mortality / attrition: dropouts (is mainly a problem if its a random dropout (experiment gets to heavy pat. drops out)
- Demand characteristics: participants know
the purpose of the study and behave differently - Placebo effect: participants get better because they believe that they receive an effective treatment
What is external validity?
ability of your research design to generalize the results beyond the research setting to real world situations
Name 4 threats of external validity
- reactive testing
- interactions between participant selection biases and independent variable
- reactive effects of experimental arrangements
- multiple treatment interference
reactive testing
pretests may affect reaction to the experimental test
Interactions between participant selection biases and the independent variable
participants that signed up voluntarily could be more extroverted, this could affect your study if you are measuring for example personality types
Reactive effects of experimental arrangements
artificial experimental setting and knowledge of being a study-participant change results
Multiple treatment interference
If participants are exposed to multiple treatments and later treatments are affected by former ones
Why taken samples?
To make general statements about the population, but it is only possible to ask a limited number of people
What is important to keep in mind by choosing a sample?
- How many is enough
* Is my sample biased
What is a simple random sample?
A group of people is randomly chosen from the whole population
• Solves problem of bias
• In reality hard to manage to get a truly random sample
Population
all individuals that are there (usually too big to test every individual)
sample
a smaller group chosen from big group
generalization
applying findings from a sample to a large population
representative sample
matches characteristics of the population
Non-representative sample
biased
Random sample
identify the population and then draw a random sample from it
Nonrandom sample
choosing special individuals/ subject pools
Only university students
Internet research
Animal subjects (same breed, sex etc.)
Name 5 kinds of sampling
- simple random sample
- stratified sampling
- proportionate sampling
- systematic sampling
- cluster sampling
simple random sample
randomly selecting a number of people
stratified sampling
dividing population into segments (=strata), then selecting a sample of equal size from each segment
proportionate sampling
proportions of people in the population are equal to proportions in your sample
systematic sampling
every Kth element after a random start
cluster sampling
naturally occurring groups of subjects and randomly selecting certain clusters
Multistage sampling
selecting individuals from those clusters
Laboratory research
- Take individuals into laboratory setting
- Control variables
- More ethical
- High internal validity and low external validity
Field research
• In participants natural environment
• Lesser control of confounding variables
• Results can easily be generalized to the real
world
• High external validity and low internal validity
Which options do you have for your research setting?
laboratory research or field research
Which negative effects can volunteerism of a participant have?
Volunteerism lowering internal validity
change in outcome not because of condition, but because participants want to “help” the researcher
Volunteerism lowering external validity because subjects that volunteer often differ in specific ways from the average population
Which 2 types of non-experimental research do you know?
observational research
surveys
Name 3 method of quantification for observational research
- Frequency
- Duration
- Interval
How can you simplify a sampling of observational research ?
- Time: alternate observing and recording
- Individual: one subject at a time
- Event: one behavior at a time
- Recording: taping the situation
Naturalistic observation
• High external validity (compared to
laboratory experiments)
• No causal links can be established
Which 2 designs of questionnaires do you know
• Open-Ended Items: more detailed, but time
consuming to analyze
• Closed-Ended Items: easy to analyze, but
loss of details
What do you know about nonresponse bias of a survey?
• Solutions: Several contacts (prenotice letter,
thank you postcard, replacement
questionnaire), small reward
• Non-factors (things that do not help to get participants): length, personalization, anonymity, deadline
What happens by the social desirability effect?
• People tend to alter answers if they feel
evaluated/watched
• Less of a problem in online-surveys
When do you have high internal validity?
if you hold everything expect of the independent variable constant (same age, same gender,..)
How can you control a placebo effect?
By adding a control group
What do you know about regression towards the mean?
- its a threat to validity
- only occurs if you select an extremest subgroup from a larger group
- there has to be random measurement error
- extremely positive= upward measurement error
- extremely negative- downward measurement error
- you have to re-measure the extreme subgroup will very likely be less extreme and be closer to the average of the entire group
What is the Solution for threats of internal validity?
adding a control group
What defines a real experiment?
- consists of minimally two groups: experimental and control group
- participants are randomly assigned to one of the two conditions / groups
- additional design choices affect validity
How must the control-group be?
Equivalent to experimental group
