767 Flashcards
Learn to be a scientist!
How does science differ from a layperson’s approach to science?
- We are explicit in what we’re doing and why we’re doing it. 2. We make a fundamental distinction between observations and inferences. 3. REPLICATION
What’s the difference between observations and inferences?
Observation is the data collected. Inferences are anything else, including results (e.g. tentative conclusions, abstractions about reality, putting words to concepts)
How does the scientific model apply to clinical work?
- Think scientifically 2. Check inferences 3. Identify and be explicit about assumptions we make 4. The goal is to be value-explicit, not value-free
Why should I become aware and explicit of my worldview and assumptions?
The alternative of being explicit and conscious is the risk of imposing values on others and making inaccurate assumptions.
Why do we live in an inductive world in Psychology?
There are no steadfast rules. Must infer from the data we collect.
What is the central problem of inductive science?
Error. All samples have error. Our efforts are all aimed at reducing this error.
What is the aim of science?
Theory.
What is the relevance of the stocastic model to clinical work?
Clients present a sample of their behavior from the population of their experiential world. We must recognize that this is limited, not impose our worldviews, and attempt to understand from multiple vantage points (different theories)
What are Mike Ellis’ current 8 steps of scientific research methods?
- Identify/observe phenomenon 2. Formulate problem. 3. Explicate theorizing. 4. Research hypothesis. 5. Empirical test. 6. Data analysis.7. Interpretation. 8. Revise theory and retest.
What is the impact of paradigms in scientific inquiry?
They affect how we: 1. Construct theory 2. Explain what happens 3. Choose phenomenon to observe and ignore 4. Identify patterns 5. Identify what counts as data
What are the rules of categorization (partitioning)?
- Categories must be set up according to the research problem. 2. Exhaustive 3. Mutually exclusive and independent 4. Each category (variable) is derived from one classification principle 5. Any categorization scheme must be on one level of discourse
Why should I graph my data?
Shows relations and their nature. Can show things you would miss with simple statistics (e.g. bimodal data)
What might cause negative results?
- Incorrect theory and hypotheses 2. Inappropriate or incorrect methodology 3. Inadequate or poor measurement 4. Faulty analysis
What is the core procedure of statistical analysis?
- Set up chance expectation as hypothesis (null hypothesis) 2. Try to fit empirical data to chance model 3. If empirical data fit chance model, they are not statistically significant. If data do not fit chance model, they are statistically significant
What is the law of large numbers?
With an increase in the size of sample, n, there is a decrease in the probability that the observed value of an event, A, will deviate from the true value of A by no more than a fixed amount, k. (Reduction of errors)
What are characteristics of the normal curve?
- Unimodal 2. Symmetrical 3. Possesses certain mathematical properties
What is Popper’s major contribution to scientific inquiry?
Falsificationism!
What is the key principle of falsificationism?
Aim to disprove, not prove, theory.
What are Jon Stewart Mill’s five methods of elimination?
- Method of agreement (constant conjunction) 2. Method of differences (absence of cause or absence of effect) 3. Method of joint agreement of differences (basis for control group vs. experimental group) 4. Method of residuals (notion of error) 5. Method of concomitant variance *look at how variables vary together if can’t manipulate– groundwork for correlation)
What is Jon Stewart Mill known for in scientific method?
- Eliminating rival explanations 2. Testing multiple hypotheses 3. Control groups 4. Manipulating variables
What are Hume’s principles for inferring causality?
- Contiguity (events occur close together in space and time) 2. Temporal preference (A must come before B if it is cause of B) 3. Constant conjunction (A must always be present when B is present if A causes B
What is a better term for post-positivism?
Sophisticated Falsificationism
What are principles of post-positivism?
- Test with multiple competing hypotheses 2. Theory is viable to the extent to which it has been systematically been subject to and survived multiple attempts to confirm and disconfirm it 3. Disconfirmation is more effective
Two ways of defining a set.
- List - listing all members of a set. 2. Rule - giving a rule for determining whether objects belong to a set
Two basic set operations
- Intersection – overlap of two or more sets (A AND B) 2. Union - Set that contains all members of one set and all members of another set (A OR B)
What are the 7 steps of the General Linear Model (GLM)
- Specify the statistical hypotheses 2. Specify the statistical model 3. Estimate parameters of the model 4. Assess goodness of fit/effect size 5. Test hypotheses on the model parameters 6. Assess the adequacy of the model 7. Make a decision (to accept or reject the statistical hypotheses) based on this… accept or reject the research hypotheses
What are the four scales/levels of measurement?
- Nominal 2. Ordinal 3. Interval 4. Ratio
How do you graph nominal data?
Bar, pie, or histogram (histogram if variable is on a continuum)
How do you graph ordinal data?
Frequency distribution, bar, pie, or histogram
How do you graph interval data?
Line graphs, percentiles, histograms, frequency distributions, bar, pie, histogram
How do you graph ratio data?
Cartesian plots, anything you can do with lower levels of measurement
Central tendency
Mode, median, mean
Spread
Range, inclusive range (high - low +1), variance, standard deviation
What measures of central tendency are appropriate for nominal data?
Mode only
What measures of central tendency are appropriate for ordinal data?
Mode and median
What measures of central tendency are appropriate for interval data?
Mode, median, and mean
What measures of central tendency are appropriate for ratio data?
Mode, median, and mean (and some things we don’t use)
What measures of variability are appropriate for nominal data?
None
What measures of variability are appropriate for ordinal data?
Range only
What measures of variability are appropriate for interval data?
Range, variance, and SD
What measures of variability are appropriate for ratio data?
Range, SD, SD squared
Properties of Z scores
Mean = 0 V = 1 SD = 1
How are sums of squares inflated?
Dependent on sample size. Dependent on dispersion. Bigger sample size, bigger sums of squares
Types of probability
A priori and a posteiori
Why do we learn statistics?
- We live in an inductive world and all samples have error. 2. Statistics help us assign probability to effects being produced by chance alone.
Properties of line of best fit
- Will always pass through the data point representing mean of x and mean of y. 2. Can draw line of best fit if have intercepts and means.
Why is probability important?
It is the heart of science. What’s the probability of a result being accounted for by error?
Why is it important to operationalize your population?
Do know how representive the sample is of the population.
How do we operationalize population?
State who we are talking about with explicit inclusion and exclusion criterion. Make distinction between target population and sample population
What makes a sample random?
Identifying population BEFORE sampling and then systematically, randomly, choosing participants
What’s the difference between randomization and random selection?
Randomization refers to random assignment to conditions. Random selection refers to collecting the sample of a population. Can have randomization of a convenience sample. Ideally, have randomization of a random sample.
What is the best estimate of population mean?
Mean of means
Standard error of measurement
Always smaller than standard deviation. The smaller the better. Dependent on metric and SD.
What’s the difference between frequency distribution and sampling distribution?
Frequency is from your sample. Sampling distribution comes from a large number of samples plotted to a graph (probability density functions – purely mathematical models)
What is the number one rival hypothesis?
Error
B always refers to
Population. This is what we’re interested in.
Properties of statistical hypothesis
- Must be directly connected to research hypothesis. 2. Clear, logical connection 3. Statistical and research hypothesis must be consistent with one another.
How do you calculate a shrunken effect size?
1 - (1 - r^2)(N-1 / N - K - 1)
What happens to shrunken effect size when increasing sample size?
- Adjustment is lower 2. Error reduces at exponential rate
What is a t-test (general)?
Sample - parameter / SE statistics
What are degrees of freedom?
The amount of unknowns we can compute from knowing the mean and a set of known terms contributing to the mean
What does a confidence interval tell us?
There is a certain chance (usually 95%) that this range contains population mean (it is not a guarantee that the range includes the population mean).
What does a confidence interval containing 0 tell us?
Insignificant finding (we cannot exclude the possibility of chance).
What is the relation between sample size and confidence intervals?
Smaller sample size means larger confidence intervals.
What is Type 1 error?
Rejecting the null hypothesis when it is true in the population
What is Type 2 error?
Failing to reject the null hypothesis when there is a true effect in the population
Statistical power =
1- Beta
What group does Type 1 error apply to?
Sample
What group does Type 2 error apply to?
Population
Statistical power is a function of what (4 things)?
- Alpha level 2. Beta level 3. Sample size 4. Effect size
What is the relation between sample size and statistical power?
Positive – increase n, increase power
What is the problem with counseling psych research regarding power?
Average statistical power is .2 = 20% chance of finding effect)
What is the purpose of a power analysis?
To determine the amount of participants needed to achieve sufficient power in a study
What is a priori power analysis?
Occurring before collecting data. Determine alpha level, estimate power for population based on smallest effect size expected – do computations to deduce sample size
What are the two types of statistical power analysis?
- A priori 2. Post-hoc
What is post-hoc power analysis?
Given results, effect size, n, and alpha, what power did the statistical test have to determine an effect?
What is a large effect size in Counseling Psychology research?
.15
What is a medium effect size in Counseling Psychology research?
.058
What is a small effect size in Counseling Psychology research?
.01
How do you calculate lower boundary of a confidence interval?
X - (1.96 x SE)
How do you calculate upper boundary of a confidence interval?
X + (1.96 x SE)
Validity is a property of…
The inferences that we make– not research design/method
What do Cook and Campbell outline at the validity typology?
- Statistical conclusion validity 2. Internal validity 3. Construct validity 4. External validity
What is statistical conclusion validity?
Validity of inferences about the correlation (covariation) between treatment and outcome
What is internal validity?
The validity of inferences about whether observed covariation between A (the presumed treatment) and B (the presumed outcome) reflects a causal relationship from A to B as those variables were manipulated or measured.
What is construct validity?
The validity of inferences about the higher order constructs that represent sampling particulars
What is external validity?
The validity of inferences about whether the cause-effect relationship holds over variation in persons, settings, treatment variables, and measurement variables
heteroscedasticity
variances between multiple groups are not equal
Sphericity
Multivariate homogeneity of variance
Why can’t we predict success/failure of individuals
Probability (and Type 1 and Type 2 error) applies to groups, not individuals
What does it mean to fail to reject the null hypothesis?
Cannot conclude that there is no relation in the population because there are many potential reasons for this finding
What is the nil hypothesis?
Since in psych nearly all variables are related in some way, nil hypothesis states that variables will be minimally related (instead of not related as per null hypothesis)
Why should we not set up studies to affirm null hypothesis?
It is not possible to affirm null hypothesis – alternative explanations
When is it okay to seek to affirm null hypothesis?
When testing statistical assumptions (homogeneity of variance, means, etc.)
What are our priorities when seeking to affirm the null hypothesis?
Preserve Type II error rate at expense of Type 1 (we need to be able to find an effect if it exists)
What is research design (as per KL)?
Plan, structure, and strategy of investigation to obtain answers to research questions. Testing relatinos and the theories they are based upon.
All psychological studies are…
Correlational design in nature.
What is the purpose of research design?
Control and rule out rival explanations through logic
Can we label research design by statistical analysis?
No. Can use any statistical design in any research design. Design is broader than statistical procedure (they are distinct)
What are the types of research design (per KL)?
- Ex-post facto 2. Quasi-experimental 3. Experimental
By what 2 things are types of research design differentiated?
- Ability to manipulate 1 or more variables 2. Ability to randomly assign experimental conditions.
With regard to variable manipulation and random assignment, ex-post facto…
Does not possess either
With regard to variable manipulation and random assignment, quasi-experimental design
Can manipulate variables but not randomly assign
With regard to variable manipulation and random assignment, experimental design
Can both manipulate variables and randomly assign participants
What impact does random sampling have on determining level of research design (ex-post facto, quasi-experimental, experimental)
None. Random assignment is what matters for this distinction.
What is the purpose of reasoned argument in research design?
- Sets context for study 2. Identifies phenomena of interest 3. Establishes theoretical importance 4. Clearly defines constructs and consistently uses terminology 6. Explicates theorizing 7. Culminates in falsifiable hypotheses
What is Type 3 error (error of the third type)?
Mismatch between theory and research design
When creating a reasoned arguments, ask yourself:
- What will this study add to the literature? 2. Why should a journal publish this?
Why is it important to become more knowledgeable about statistical procedures and research design?
Become a better producer and consumer of research – help control confounds/error and gain sophistication in considering alternative explanations for results
What are threats to validity?
Sources of error that can be affecting the inferences that we make (reasonable, not merely plausible)
What are the most important things to check with regard to threats to validity in clinical work?
- Medical explanation 2. Fundamental attribution error – circumstances rather than disposition
What are four threats to hypothesis validity?
- Inconsequential research hypotheses 2. Ambiguous research hypotheses 3. Noncongruence of research hypotheses and statistical hypotheses 4. Diffuse statistical hypotheses and tests
Definition of reasoned argument
Coherent, internally consistent package that is both rigorous and well articulated (explicit) and systematically examines alternative explanations.
