Statistics and Measurement Flashcards
1
Q
How is a quasi-experimental research design different from a true experimental design?
A
In a quasi-experimental design random assignment doesn’t/can’t occur.
2
Q
What contributes to a non-experimental research design?
A
The independent variable is pre-existing and there is no manipulation or intervention.
3
Q
What is required in a true experimental design?
A
There must be at least 1 manipulated independent variable
Subjects must be randomly assigned to their group
4
Q
What is another name for a nomothetic research design?
A
Group Research Design
5
Q
What are the 3 group research design strategies?
A
Between-Groups
Within-Subjects
Mixed
6
Q
What is a between-groups research design?
A
Comparing groups where the groups and data are independent of one another
7
Q
What is a within-subjects research design?
A
comparing groups in which the data is related/correlated/repeated
8
Q
What type of research design is used when two groups of subjects are matched on some characteristic/variable?
A
Within-Subjects Design
9
Q
What are carryover effects?
A
Occurs in a within-subjects design when subjects are exposed to multiple conditions and their performance naturally changes
10
Q
Carryover effects occur in what type of research design?
A
Within-subjects
11
Q
How should a research counteract carryover effects?
A
Use a latin square design
12
Q
What is a mixed research design?
A
Contains components of both between-groups and within-subjects designs. One component is independent and the other is repeated
13
Q
What is another name for an idiographic research design?
A
Single Subject
14
Q
What is one problem of a single subject research design?
A
Autocorrelation
15
Q
What is the AB research design?
A
A is baseline
B is treatment
16
Q
What is the biggest threat to an AB treatment design?
A
History
17
Q
How is an ABAB treatment design better than an AB design?
A
It removes the threat of history
18
Q
What is one ethical concern about the ABAB design?
A
Treatment B may be effective and then you remove it for a return to baseline
19
Q
What is one general concern about the ABAB design?
A
Participants may not return to baseline a second time
20
Q
What is a drawback of a multiple baseline design?
A
Time Consuming
Expensive
21
Q
What is a benefit of a multiple baseline design?
A
Avoids the problems of AB and ABAB designs
22
Q
What is the simultaneous treatment approach?
A
Simultaneous treatment occurs, but at two different times of day
23
Q
What is a changing criterion research design?
A
Used when gradually trying to change behavior
work towards achieving a criterion and the criterion changes as time progresses
24
Q
When would you use interval sampling?
A
When the behavior being measured does not have a specific or identifiable beginning or end
25
What are the two types of interval sampling?
Momentary Sampling
| Whole Interval Sampling
26
What is momentary interval sampling?
Choose specific times and if the behavior is being exhibited at that time you get a tally
27
What is whole interval sampling?
Choose and interval of time and only get a check if the behavior is begin exhibited that entire time interval
28
What is analog research?
Approximates a clinical situation
29
What is a problem with analog research?
Limits generalizability
30
What are the three time frames of research design?
Cross-Sectional
Longitudinal
Cross-Sequential
31
What is one problem with a cross-sectional research design?
Cohort Effects
32
What is a problem with longitudinal research?
Expense
| Dropout Rates
33
What is a cross-sequential research design?
Break groups into cross-sections and then monitor them over shorter periods of time
34
Simple Random Sampling
Everyone in the population has an equal change of being chosen
35
Stratified Random Sampling
Dividing the population into strata and then sampling equal numbers within each strata
36
Proportional Sampling
Sample people in proportion to their representation in the population
37
Systematic Sampling
Creating a system and sampling that way (every 10, odd SSN's, etc.)
38
Cluster Sampling
Sampling naturally occuring clusters
39
Name some threats to internal validity
```
History
Maturation
Testing/Test Practice
Instrumentation
Statistical Regression/Regression to the Mean
Selection Bias
Attrition/Experimental Validity
Diffusion
```
40
What is the best way to control for threats of history and maturation?
Having a control group
41
What is the threat of history?
Any incident or occurrence that happens while in treatment, but outside the experiments that affects the outcome
42
What is the threat of maturation?
Something happening within the subject due to the passage of time
43
What is the treat of testing?
People do better due to familiarity with the test instrument
44
What is the best way to control for the threat of testing?
Use a Solomon 4 Group Design
45
What are the four groups in a Solomon 4 Group Design?
Pre-test, Post-test, with Intervention
Pre-test, Post-test, without Intervention
Intervention and Post-test
Post-Test Only
46
What is the threat of instrumentation?
Calibration of instruments causing change
47
What is the threat of Statistical Regression?
Tendency for extreme scores to move closer to the mean
48
How can you control for the threat of statistical regression?
Use a control group
49
What is the threat of selection bias?
Non-random assignment to groups
50
What is the threat of attrition?
When people withdraw during the experiment
51
What is the threat of diffusion?
When the treatment diffuses into the control group
52
What are some threats to external validity?
Sample Characteristics
Stimulus Characteristics
Contextual Characteristics
53
How can sample characteristics be a threat to external validity?
Most research relies on volunteers, so there may be a certain type of person that volunteers versus one who does not.
54
How can stimulus characteristics be a threat to external validity?
The characteristics of the research set-up may not generalize to the population.
55
How can contextual characteristics affect be a threat to external validity?
People are participating in research and being observed so they may respond differently (Hawthorne Effect)
56
What are some threats to Construct Validity?
Experimenter Expectancies/Rosenthal Effect
Demand Characteristics
John Henry Effect/Compensatory Rivalry
57
What can experimenter expectancies be a threat to construct validity?
The experimenter may inadvertently communicate something to the subjects about how to respond
58
How can a researcher control for experimenter expectancies?
Double-blind studies
59
How can demand characteristics be a threat to construct validity?
Subjects might pick up on something that cause them to respond differently
60
How can a researcher control for demand characteristics?
Don't tell a participant which group they are in
61
How can the John Henry Effect be a threat to construct validity?
When people in the control group try even harder than the experimental group
62
How can a research control for the John Henry Effect?
Don't tell a participant which group they are in
63
What is a threat to statistical conclusion validity?
Low Power
64
What might cause low power in a research study?
Small Sample Size
Unreliable Measures
Inconsistent Procedures
Heterogeneous Subjects
65
The higher the internal validity, the ____________ the external validity
Lower
66
Positively skewed data has a higher proportion of ________ scores.
Low
67
Negatively skewed data has a higher proportion of ___________ scores.
High
68
Which measure of central tendency is highest in positively skewed data?
Mean
69
Which measure of central tendency is the lowest in positively skewed data?
Mode
70
Which measure of central tendency is the highest in negatively skewed data?
Mode
71
Which measure of central tendency is the lowers in negatively skewed data?
Mean
72
What percent of the population falls within 3SD of the mean?
99%
73
What percent of the population falls within 2SD of the mean?
95%
74
What percent of the population falls within 1 SD of the mean?
68%
75
What does a z-score tell you?
How many SD's above of below the mean someone scores?
76
What are the Mean and SD of T scores?
Mean=50
| SD=10
77
What are the mean and SD of IQ scores?
Mean=100
| SD=15
78
What is the central limit theorum?
If we take equal sized samples from the population, and we take an infinite number of them and plot the mean, we get the normal distribution
79
What is Type I Error?
Incorrectly Rejecting the Null Hypothesis
80
What represents the probability of making a Type I error?
Alpha
81
What is Alpha?
The probability of making a Type I error
82
What is Type II Error?
Incorrectly accepting the null hypothesis
83
What represents Type II Error?
Beta
84
What is Beta?
Type II Error
85
What is Power?
Correctly rejecting the null hypothesis
86
1-Beta=?
Power
87
What are some examples of non-parametric tests?
Chi Square
Mann Whitney U Test
Wilcoxin
88
What are the three requirements of parametric tests?
Interval/Ratio Data
Homogeneity of Variance/Homoscedasticity
Normal Distribution of Data
89
What is Homoscedasticity?
Homogeneity of Variance
90
What are 3 types of T-tests?
Single Samples T-Test
Independent Samples T-Test
Matched Samples T-Test
91
What type of post-hoc tests are available when significance is found on an ANOVA?
Pairwise Comparisons
Sheffe
Tukey
92
Post-hoc tests for ANOVAs lead to an increased chance for what?
Type II Error
93
When would you use ANCOVA?
When you find that something affects your outcome that you weren't planning for, you can statistically remove it using ANCOVA
94
When do you use trend analysis?
When an ANOVA is significant
95
What is a trend analysis?
Allows you to look at patterns of change so you can see how the DV changes with various continuous levels of the IV
96
What is the coefficient of determination?
The square of the correlation (r squared); it determines shared variability
97
What is a 0 order correlation?
X and Y are correlated and we don't think anything else is interfering with that relationship
98
What is a first order correlation?
There is a relationship between X and Y, but you think variable Z also affects this relationship
99
What is a semi-partial correlation?
When you have a first order correlation, but remove the effects of Z on either X or Y.
100
What is used to create the line of best fit?
Least Squares Criterion
101
What type of data is needed for a Pearson R correlation?
X and Y both need to be interval or ratio data
102
What type of data is needed for a Point Biserial correlation?
One variable is dichotomous and the other is interval or ratio data
103
What is the difference between point biserial and biserial?
Point biserial is a true and naturally occurring dichotomy. Biserial is an artificial dichotomy.
104
What is the problem of multicollinearity?
When there are multiple predictor variables (Xs) and they are all highly correlated with one another
105
What is a stepwise regression?
The computer decides which predictor variable is processed first, second third etc. based on how strongly related each variable is with the criterion
106
What is Hierarchical regression?
The researcher uses an underlying theory to determine the order of predictor variables
107
What is canonical correlation?
Two or more predictors and two or more criteria
108
What is discriminant function analysis?
A special case of multiple regression equation used when Y is nominal
109
What is logilinear analysis?
Looks like discriminant analysis with nominal Y data, but X's are all nominal
110
What is a moderator variables?
Affects the strength/nature of the relationship between X and Y
111
What is a mediator variable?
Explains why the relationship between X and Y exits
| If you remove the mediator variable, the correlation between X and Y no longer exist
112
What is a path analysis?
A way of using correlations to determine if there's a cause/effect relationship
113
What's the most common method of structural equation modeling?
LISERAL
114
What is LISERAL?
A method of structural equation modeling
115
What is a factor analysis?
Look at all factors and determine which cluster together to explain the outcome
116
What is an eigenvalue?
Tells you the strength of a factor in factor analysis
117
What are the two rotations used in factors analysis?
Orthogonal and Oblique
118
What is the difference between orthogonal rotation and oblique rotation?
Orthogonal rotation leaves you with factors that are uncorrelated with one another. Oblique rotation leaves you with correlated factors
119
When would you use principal components analysis?
When the researcher has no theoretical expectation of what the factors may be
120
What is classical test theory?
Any given score is the combination of truth and error
121
What is true score variability?
Reliability
122
What are 3 sources of test error?
Content sampling
Time Sampling
Test Heterogeneity
123
How does content sampling lead to test error?
When items on the instrument happen to be something you do or do not know
124
How does time sampling lead to test error?
Passage of time can naturally affect scores
125
How can test heterogeneity affect test error?
When an item taps more than on domain it can lead to test error
126
What affects instrument reliability?
```
Number of test items
Homogeneity of items
Range of Scores
Population Sample
Ability to Guess
```
127
How does the number of items on an instrument affect its reliability?
More items=More reliable
128
How does the homogeneity of items affect an instrument's reliability?
More homogeneous=more reliable
129
How does score range affect instrument reliability?
larger range of scores=more reliable
130
How does the population sample affect instrument reliability?
Heterogeneous Sample=more reliable
131
How does the ability to guess the answer to questions affect a test's reliability?
If items are harder to guess, the test is more reliable
132
What are the ways of measuring reliability?
Test-Retest
Parallel Forms
Internal Consistency
Interrater Reliability
133
What causes error in test-retest reliability?
Time
134
What causes error in parallel forms reliability?
Time
| Content Sampling
135
How does split half reliability affect overall reliability?
It artificially reduces reliability
136
What is the Spearman-Brown Prophecy Formula used for?
Uses the split-half reliability, but tells you how much more reliable your test would be if you used the full number of items
137
What is the primary source of error is split half reliability?
Content Sampling
138
Split half reliability should never be used with what type of test?
Speeded tests
139
What is Kuder Richardson Coefficient Alpha?
It's a way of determining split-half reliability. The computer takes the test and splits it in every possible half, generates a correlation for each of these, and averages them
140
What is the standard error of measurement?
The average change in scores if the same person were to take the same test repeatedly
141
What are some types of validity?
Content Validity
Criterion Related Validity
Construct Validity
142
What is content validity?
Does the test cover all the basic content knowledge/skills of what you want to measure?
143
What is criterion related validity?
Does the test accurately predict what we are trying to predict?
144
What are some types of criterion related validity?
Concurrent Validity
| Predictive Validity
145
What is an expectancy table?
Tells you the probability that a specific criterion score will fall in a range given the range in which a predictor score falls?
146
What are Taylor-Russel Tables?
Tells you how much of an improvement you will make in a selection/hiring decisions when using a predictor test as compared to using no test
147
What do you need to know to use a Taylor-Russel table?
Base Rates of successful hires and selection ration
148
What is selection ratio?
Number of opening to number of applicants
149
What impacts incremental validity?
Criterion Related Validity of the Instrument
Base Rate
Selection Ratio
150
How would you optimize incremental validity?
Have a moderate base rate (.5) and low selection ratio (.1)
151
What is decision-making theory?
It looks at predictions based on using a predictor test and compares them to actual results
152
What are the four possible outcomes of a predictor test?
True Positive
False Positive
True Negative
False Negative
153
what is easier to change, the predictor or the criterion?
The predictor. Sometimes you cannot change the criterion.
154
What is item discrimination?
How well an items discriminates between high scorers and low scorers
155
What is item validity?
How well an individual scored on that item and the entire test
156
What is the item characteristic curve?
Relationship between how you do on an item and your total score
157
What is item response theory?
It allows the researcher to calculate to what extent a specific item correlates to the underlying trait being measured and is used to create individually tailored adaptive tests
158
What creates shrinkage?
Cross-validation
159
What factors affect the validity coefficient?
Range of scores
Reliability of the predictor
Correction for Attenuation
Criterion Contamination
160
How does the range of scores affect validity?
Broad Range=Better Validity
161
How does the reliability of the predictor affect validity?
Reliability puts an upper limit on validity
| Validity is less than or equal to the square root of reliability
162
What is correction for attenuation?
Because X and Y are both not perfectly reliable, validity is lessened. This formula tells us, if X and Y were perfectly reliable, how much better our validity would be
163
What is Criterion Contamination?
This occurs when Y is subjectively scored and the rater knows how people did on the predictor. This artificially inflates validity
164
In the multitrait multimethod matrix, what is convergent validity?
Monotrait Heteromethod
165
In the multitrait multimethod matrix, what is divergent vaidity?
Heterotrait Monomethod
166
What are the subtypes of construct validity?
Convergent Validity
| Divergent Validity
167
What is the Standard Error of the Estimate?
How much error there is in estimating what we are trying to predict
