Midterm Lec 6a Flashcards
1
Q
Quantitative research
A
Data are numbers and the people are subjects
Randomized control trials, chart studies
2
Q
Qualitative research
A
Field research, case studies, participant observation and the people they study are called participants
3
Q
3 key characteristics of experimental research design
A
1) independent variable that is manipulated
2) control of all other variables (dependent)
3) observation of the effect of the independent variable on the dependent variable
4
Q
Advantages of experimental design
A
CRAE
Convergence,
replication,
adjustment of variables,
establishment of cause and effect relationship
5
Q
3 criteria for establishing cause and effect
A
1) the cause must precede the effect in time
2) the cause and effect must be correlated with eachother
3) the correlation between cause and effect cannot be explained by another variable
6
Q
Disadvantages of experimental design
A
cost
inability to generalize
securing cooperation
depending on the type you choose, it can be quite complicated to design and implement
7
Q
Control or situational variables
A
Variables that the researcher may not be able to manipulate, exclude, remove, alter
—>Preferred way to overcome this is to hold the variable constant by the researcher
8
Q
An auto manufacturer wants to know now bright brake lights should be in order to minimize the time required for the driver of a following car to realize that the car infront is stopping
Independent? Dependent? Control?
A
Independent, intensity brightness of brake light (manipulated)
Dependent, time of onset brake light until depression of brake pedal ( observed)
Control, colour of brake light, shape, force needed to depress pedal (hold constant)
9
Q
A pigeon is trained to peck a key if a green light is illuminated but not if the light is red. Correct pecks get rewarded by access to grain. Independent? Dependent? Control?
A
Independent, colour of light red or green.
Dependent, number of pecks
Control, size of key and intensity of light.
10
Q
Experimental group
A
The group that receives the treatment
11
Q
Control group
A
The group that does not receive the treatment
12
Q
Subjects
A
The name for the individuals in experimental research
13
Q
Advantages of subjects/groups
A
Advantages; no chance that one treatment con contaminate another since the subject/group only receive one treatment
- each subject is compared to himself/ herself therefore differences observed are not because of differences between subject.
14
Q
Disadvantages of subjects/groups
A
Disadvantages; concern that there is a possibility that the subjects or groups are different enough to influence the effectsof treatment.
- carryover effect: testing subjects in one condition has an effect on their testing in another condition.
15
Q
How to overcome disadvantages in subjects and groups.
A
- Randomization of treatment levels
- counterbalancing the treatment levels, systematically vary the order of the conditions to distribute the effects of time eg practice effects, fatigue.
16
Q
Random sample
A
A sample from the population that has been selected in an unbiased way.
Each person has an equal chance of being selected.
17
Q
Random assignment
A
Subjects are assigned to conditions in an unbiased fashion.
18
Q
Random groups design
A
Subjects are randomly assigned to conditions in the between subjects1 group design
19
Q
Matching
A
- Treatment, experimental subjects are matched with a central subject
- common “matching” items: age, gender, weight, height, IQ, years of schooling
20
Q
Matched group designs
A
Subjects are matched on some variables assumed to be correlated with the dependent variable and then randomly assigned to conditions.
21
Q
Matching cautions
A
What characteristics or items do you match on?
Can you locate exact matches?
- true matching becomes almost impossible because there will be personal, genetic, envioumental, influences that the researcher simply can’t control.
22
Q
Blind assignment
A
-The subject does not know whether they are in the treatment or control group
-the researcher knows what group the subject is in.
23
Q
Double-blind assignment
A
Neither the researcher collecting the data or the subject know which group the subject belongs to.
24
Q
Pre-experimental design types
A
-Lacks random assignment
- uses short-cuts that are much weaker than a classic experimental design
-substituted for when a researcher cannot use all the parts of the classical experimental design
- has weaker internal validity
25
Classic/true experimental design has
- Random assignment
- control group
-experimental group
- pre-test and a post-test for each group (control + experiment)
26
Quasi-experimental designs
- This design is stranger than pre-experimental
- variations on classical experimental design
- used when the researcher has limited control over the independent variables
- some have... Randomization but lack pre-test, more than 2 groups
27
Randomized contra trial
- An experimental design for medical and pharmacutical investigation
- RCT is viewed as the best type of evidence on which to base decisions and establish guidelines
- 2 types of trials within RCT
28
Parallel group trial (rct)
- Only one set of patients receives the new drug
- comparison is between the two groups
29
Cross over trial (rct)
- All participants receive new drug
- patients are randomized into an intervention group or a control group, with the intervention group taking the medication for a selected period of time.
- after that period of time, the OG intervention group becomes the control group taking the placebo while the OG control group serves as the intervention group receiving new drug.
30
Populations
- The total number of people things, or events of interest
31
Samples
- A subset of the population
-small collection of units from a much larger collection or population
- smaller set of cases selected from a larger pool
32
Representative sample
-the tendency to display variations among its members that are proportional to the variations that exist in the actual population of interest
-the tendency does not exist, the sample is said to be biased
33
Variability
- Now little, or now much the sample varies when compared to the population from which it was selected
-if there's lots of variability, you will want a large sample in order to ensure that the sample is reasonable representative of the population
- little variability then small sample size will be sufficient.
34
Sampling techniques
Probability and non-probability
35
Non-probability sampling
Aka non-random sampling
-used by qualitative researches
-seven types
36
Probability sampling
- Used by qualitative researches
- sampling is based on theories of probability from mathematics
-often used in experimental studies
-random is the key to sampling
37
Random (sampling)
- Lacks predictability, without any systematic pattern
-random samples are must likely to yield a sample that truly represents the population
- lets researcher statistically calculate the relationship between the same population
38
Sampling error
- Now much a sample deviates from being representative of the population
39
Sampling element
- Unit of analysis or case in a population
- "what" are you going to study? People? Animals? Objects? Organization?
40
Target population
- Specific pool of cases that are to be studied
- a representation of a certain segment
- "who" or "what" specifically is the researcher interested in studying? People with 6 toes? Cats with no whiskers?
41
Sampling ratio
- ratio of size of sample to the size of target population
- can be expressed as a decimal value or %
- sample: target population
Example: 50,000 people in the target population researcher selects (samples) 150 people from the target population to study
150/ 50,000 = 0.003 or 3%
42
Sampling frame
- List of cases in a population, or the best approx. Of it.
- the list from where the researcher will attempt to draw his/her sample from
- examples can include: taxation records, telephone directors, drivers license records, voters list, membership lists, etc
- potential problems with sampling frames
43
Parameter
- A characteristic of the entire population that is climate from the sample
- a quantifiable characteristic or feature of a population
44
Probability sampling key words
- Sampling element1 target population, sampling ratio, sampling frame, parameter
45
Measurement
- A repeatable, objective procedure for generating a measure
46
Scale of measurement
- The set of possible numbers that may be obtained by the measurement process
- all scales have certain characteristics or ‘properties'
1Magnitude
2 Equal intervals
3 Absolute 0
47
Magnitude
- When a scale has a magnitude, one aspect of the attribute being measured can be judged greater than, less than, or equal to, another aspect of the attribute
Eg if you have an aggression scale of 0-10 and you assign Jane a value of 8 and Joe a value of 5, you would say that Jane is more aggressive
48
Equal intervals
- The unit of measurement on the scale is the same regardless of where on the scale the unit falls
Eg the difference between 2 inches and 3 inches on a measuring tape would be the same difference between 82 inches and 83 inches
49
Absolute zero
-a value that indicates that nothing at all is being measured
- eg if the height is measured with a tape measure 0 inches of height is a scale value that indicates no night whatsoever
It is absolute zero
50
Factors that will affect measurement
Continuous variable and discrete variable
51
Continuous variable
- Have an infinite number of values or attributes that flow aloes a continuum
- values can be divided into many smaller increments
Es temps ages income, crime rate.
52
-Discrete variable
-Have a finite number of values
-Limited number of distinct, separate categories
E.g., gender, religion, marital status
53
Levels of measurement (LOM)
-Suggests that some measures are higher or more refined, while others are crude, less precise.
LOM will:
-Depend largely on how a concept has been defined
-Affect the kinds of indicators chose
-Depend on the limits of the variables
54
Nominal level LOM (discrete)
-The lowest, least precise LOM for which there is a difference in type only among the categories of a variable
E.g., racial heritage – African, Asian, Caucasian
55
Ordinal level LOM (also discrete)
-Identifies a difference among categories of a variable AND allows for the categories to be rank ordered as well
E.g., grades – A,B,C,D
56
Interval level LOM (continuous)
-A LOM that identifies differences among variable attributes, ranks categories, and measures the distance between categories, BUT there is no true zero
E.g., temp – 5, 45, or 90 degrees celsius
E.g., water freezes at 32 degrees fahrenheit, 0 degrees Celcius and 273 Kelvin
57
Ratio level
-Highest, most precise LOM
-Variable attributes can be rank ordered, the distances between them precisely measured and there is an absolute zero
-The existence of a true zero allows for statements of proportions or ratios
E.g., money income – $0, $10, $100, $500
E.g., 0 means 0
-Discrete variables tend to be nominal and ordinal
-Continuous variables tend to be interval or ratio
-Interval level can always be turned into ordinal or nominal, but not vice versa
-Ratio level can be turned into interval, ordinal or nominal, but not vice versa
58
Internal Validity
-Ability of experimenters to strengthen the logical rigor of a causal explanation by eliminating potential alternative explanations for an association between the IV and DV
-IV relates to how well a study is conducted
59
Threats to Internal Validity
-Selection bias
-History
-Maturation
-Testing
-Instrumentation
-Experimental mortality
-Statistical regression
-Diffusion of treatment
-Compensatory behaviour
-Experimenter expectancy
-Sequence effects
60
Selection Bias
-Groups in an experiment are not equivalent at the beginning of the experiment with regard to the dependent variable
- E.g., hot peppers / heartburn – some subjects may already have a history of heartburn before starting the study
61
History
-An event or situation that occurs and affects the dependent variable during an experiment
-It is unplanned and outside the control of the experimenter
-More likely to occur in experiments of long duration (e.g., longitudinal studies)
- E.g., Hurricane Katrina, The Great Depression
62
Maturation
-Natural processses of growth, boredom, etc. that occur during an experiment and affects the DV
- E.g., study measuring cartoon preference
- E.g., study wherein the subjects must complete several written tests over the course of a day
63
Testing
-The very process of measuring in the pre-test can have an impact on the DV
- E.g., study for hypertension
64
Instrumentation
Changes in the characteristics of the measurement instrument over time
- E.g., wear and tear of equipment
65
Experiemntal mortality
-Subjects failing to participate through the entire experiment
-Why is it a problem?
-Very problematic if there are a lot of subjects who withdraw (w/d)
- E.g., study starts with 100 subjects and then 75 w/d
-Very problematic if there are a lot of subjects within a specific group
- E.g., three groups of 20 subjects; 10 people from ⅔ groups w/d
66
Statistical regression
-Subjects who are at the extreme end of the spectrum on a measurement will tend to move towards the middle
-Aka regression towards the mean (or average)
- E.g., scores on an exam – when there is little room to go up, you tend to come back down due to random errors
67
Diffusion of treatment / contamination
-Treatment “spills over” from the experimental group into the control group and then the control group subjects modify their behaviour because they learn of the treatment
68
Compensatory behaviour
-Subjects in the control group modify their behaviour to make up for not getting the treatment
69
Experimenter expectancy
-Can happen whenever the experimenter indirectly makes subjects aware of the hypothesis or desired results
70
Sequence effects
-Seen in w/in subject designs
-Response or performance in a later condition may be the result of the subjects’ role in a previous condition
71
Controlling the Threats to Internal Validity
-Randomization: Real randomization
Matched pairs (not matched groups)
Randomizing treatments or counterbalancing
-Placebos
-Blind setups: Single blind vs double blind
-Reactive effects of testing: Eliminate pretest
-Instrumentation: Calibration and test reliability, Halo effects.
-Experimental mortality: Keep participants
72
External Validity
-The ability to generalize the findings beyond the study
73
Location of experiment
-Laboratory experiment: Takes place in a specialized setting or laboratory
-Field experiment: Experimental research that takes place in a natural setting
74
Population validity
-Refers to the question of whether the responses or behaviours of sx in the research sample can be generalized to the target population
E.g., can you generalize the reaction time of healthy university students crossing the street to an elderly population?
75
Ecological validity
-Refers to the question of whether the research findings can be generalized to all of the environmental contexts of interest
E.g., can testing sunscreen in a Canadian summer be generalized to the caribbean environment
76
Experimental realism
Not always a bad thing…
-The experiment is made to feel so realistic that experimental events have a real impact on subjects
-Becomes a problem when the design is weak and the events do not impact the sx
77
Reactivity
-Arises because sx are aware they are in an experiment and may react differently
78
Hawthorne effect
-A reactivity effect named after a famous study at an American electrical company
-Subjects responded to the fact they were in a study more than they did to the treatment
79
Demand characteristics
A type of reactivity in which subjects in an experiment pick up clues about the hypothesis and alter their behaviour accordingly
80
Placebo effect
-When subjects who do not receive the real treatment and instead receive a placebo or imitation treatment but respond as though they have received the real treatment
81
Threats to External Validity
Location of experiment
Population validity
Ecological validity
Experimental realism
Reactivity
Hawthorne effect
Demand Characteristics
Placebo effect
82
Controlling the Threats to External Validity
Selecting from larger population:
- Participants
- Treatments
- Situations
Ecological validity : Does the setting capture the essence of the real world?
83
Measurement Validity
-How well an empirical indicator (measurement) and the conceptual definition of the construct that the indicator is supposed to measure “fit” together
84
Reliability
-Reliability means dependability or consistency
-Reliability is consistency across time (test-reliability) across items (internal consistency) and across researchers (interrater reliability)
-Suggests that when the same things are repeated or recur under identical or very similar conditions, the outcome or results should be identical or very similar
-Researchers will often compute statistical analysis to identify / ensure more “specific” types of reliability-based on their research question, their type of research (quantitative vs qualitative) and their discipline of study (e.g., science, arts, humanities, engineering etc)
85
Stability reliability
-A measure that yields consistent results at different time points assuming what is being measured does not itself change
E.g., if standardized intelligence tests are used to measure IQ, then they will have stability-reliability over the years; however, if starting in 2010 IQ is measured by how many toes you have, the standardized test will no longer have stability-reliability
86
Representative reliability
-A measure that yields consistent results for various social groups
-Do you think that IQ tests have ‘representative reliability’? Why?
87
Equivalence reliability
-Reliability across indicators
-A measure that yields consistent results using different specific indicators, assuming that all indicators measure the same construct
E.g., culture specific IQ tests
88
How to Improve Reliability
-Clearly conceptualize all concepts: Develop clear, unambiguous definitions
-Increase: Try to measure at the most precise level possible -Use refined categories
E.g., high or low (not so good)
E.g., on a scale of 1-10 (much better)
-Use pre-tests, pilot studies, and replication: Test out your measurements before actually implementing them – use “drafts” -Do it once, do it again, and then again.
Use multiple indicators of a variable:
-Several different tests or measures to provide empirical evidence
89
Probability
-The odds that a certain event will occur
-The likelihood that something will happen
-Given that all things are equal, what are the chances that an event will happen
90
Probability in Research
-What are the odds that the results we found were not due to chance?
-When researchers state a level of probability, they are indicating how confident they are that their findings are not simply due to chance
-The extent to which the researcher is sure that changes seen in the outcome were because of the intervention, not chance
91
Probability Math
Probability is expressed as p
-The numerical value that follows is referred to as alpha
-Often when you are reading, you will come across values like:
- p < .05
- p < .01
- p < .001
- p < .05
- 5/100 = five times out of 100, the results you see are because of chance and not the intervention
- p < .001
- .1/100 = one times out of 1000, the results you see are because of chance and not the intervention
- Findings with this alpha level are considered to be very significant
92
Probability / Alpha Level Example
- If the researcher say their statistical analyses revealed significant results with a p < .01, they are saying:
- There is a less than 1 chance in 100 the results are because of chance
- To have a probability statement with this alpha level, they are very confident about the statistical findings
- Consequently, there is a very good chance that the intervention has an effect
93
Example
Dr. Neversleep would like to conduct a study on exam performance and levels of sleep. He believes that students need at least 8-10 hours of sleep each night to perform well on exams. He is going to randomly sample 1000 university students and follow them throughout the December final exam period
-State the hypotheses for the research question:
Ho: Lack of sleep has no effect on the outcome of exame
Ha: Lack of sleep does affect the outcome of exams
-The results for Dr. Neversleep’s study are very interesting. It would appear that with a p <.01, he can conclude that lack of sleep can have an impact on exam outcomes
94
Why do Type I Errors Happen?
Most often, subjects with extreme scores affect the data analyses
95
Consequences of Type I errors
-The researcher claims there is an effect and there is not – can lead to change in clinical practice, which in turn could be harmful
-Researcher may try to replicate the findings but does not get the same results because there was no effects the first time around
-False results can lead to other researchers to develop theories based on false information – leads to a dead end for everyone
96
Why do Type II Errors Occur?
-Treatment effect may have been too small to have a noticeable effect on the sample
-It is possible by chance that the sample was extreme in some way before the study started
97
Consequences of Type II errors:
-The experimental design may not be good enough
-Experiment may be abandoned when in fact it has potential (provided modifications are made)
-Could mean giving up a good line of research that could lead to important findings
98
