Research And Program Evaluation Flashcards
What is the purpose of applied research?
Its conducted to advance our knowledge of how theories, skills and techniques can be used in terms of pratical application
What is the most valuable type of research?
Experiments aimed to discover cause and effect relationships
Whats a quasi-experiment?
Its when the researcher uses preexisting groups and the IV cannot be altered (ex: gender, ethnicity). Can determine cause/effect.
What is internal validity?
Refers to whether the DVs were truly influenced by the experimental IVs or whether other factors had an impact
Whats external validity?
Whether or not the experimental results can be generalized to large populations
What are some threats to internal validity?
- Maturation of subjects (psychological and physical changes including fatigue)
- Instruments used to measure bx
- Mortality (subjects withdrawing)
- Statistical regression
What does it mean when experiments emphasize parisomy?
Interpreting the results in the simplest way
Occams Razor suggests that experimenters…
Interpret the results in the simplest manner
Parisomy is equivalent to the word…
Parsimony
What is an “ex post facto study?”
Meaning “after the fact”
Whats an IV?
The variable the experimenter manipulates, controls, alters, or wishes to experiment with
Whats a DV?
Expresses the outcome or data in regards to factors you wish to measure
What is a control group?
They do not recieve the IV. Only the experimental group recieves the IV.
If you cannot randomly assign subjects, what type of experiment would it be?
Quasi-experiments
How many ppl do you need for a true experiment?
At least 30
Who’s work is closely related to hypothesis testing?
R.A. Fisher
Whats a null hypothesis?
Asserts there will not be a significant difference between the experimental group and control group (samples will stay the same)
When does a null hypothesis take place?
When an IV does not affect the DV
Whats an experimental hypothesis?
Suggests that a difference will be evident between the control and experimental group
Whats an affirmative hypothesis?
Asserts that the IV has indeed caused a change
Whats a percentile rank?
Its a descriptive stat that explains what percentage of the cases fell below a certain level
Whats a between subjects design?
When theres different subjects for each condition. Each subject recieves only one value of the IV.
Whats a within-subject design?
When the same subjects are employed (also referred to as repeated measures). 2 or more values/levels are administered to each subject
What does the letter P refer to in relation to a test of significance?
Probability/ level of significance
Whats a parameter?
Summarizes the characteristics of a population
Whats ethnographic research?
Involves research that is collected via interviews, observations and inspection of documente
What is the accepted probability level in social sciences?
.05 or less
Whats another way to refer to the level of significance?
Confidence level
Whats a type I error?
Alpha
Whats a type II error?
Beta
When does the type I (alpha error) occur?
When the researcher rejects the null hypothesis when its true
Whats a type II (beta) error?
When the researcher accepts the null when its false
What happens when you lower the statistical level (ex: .05 to .01)
Lowers Type I errors but raises the chance of committing type II errors
How do you test for significant differences between groups?
t Tests
How do you test for significance with more than 2 groups?
ANOVAs. They test when theres more than one level of the single IV.
Whats an ANCOVA?
Tests 2 or more groups while controlling for extraneous variables
What test do you use for two IVs?
Two way ANOVA
What table do you look at for ANOVAs?
Consult a table for F values
Whats a correlation coefficient?
Statistic that indicated the degree or magnitude of the relationship between 2 variables
Whats the range for correlation coefficients?
0.00 (no relationship) to 1.0 or -1.0
Whats a negative correlation coefficient?
Means that as one variable goes up, the other goes down
Whats a biserial correlation?
Indicates that one variable is continuous while the other is dichotomous
When is the term bivariate utilized?
When correlational data describe the nature of 2 variables
What does N stand for?
Number of people being studied
Whats a single blind study?
When participants dont know which group they are in, but the experimenter does
Whats a double blind study?
Both parties are unaware of groups and hypothesis
What are experimenter effects?
When the experimenter unconsciously communicates his or her intent or expectations to the subjects
Whats an ABA design?
Baseline (A) is secured, then intervention is implemented (B) then the outcome is examined via new baseline (A)
What designs are used when theres more than 1 target bx?
Multiple baseline designs
Whats the 68-95-99 rule?
In normal distribution, 68% of scores fall in the 1st SD of the mean, 95% within 2 SDs of the mean, 99.7% within 3 SDs
When do you use factorial designs?
When several experimental variables are investigated (2 or more IVs) sometimes the IVs are referred to as levels.
What will be the highest point in a distribution, regardless of shape, when displayed graphically?
The mode
What would a graphic look like if everyone did bad on a test?
Scores would fall on the left/low side of the distribution. Graphically the tail would point to right or positive side
What do graphs look like if everyone scores really high?
Scores would fall on the right side of the curve, which would give you a long tail that points to the left, indicative of a negative skew
How can you tell if a distribution is positively or negatively skewed?
The tail…tails on the right are positive, tails on the left are negative
Whats the benefit of standard scores vs raw data?
Standard scores allow you to analyze data in relation to the properties of a normal bell shaped curve
What type of distribution can be graphically displayed like a bar graph?
A histogram
Whats the mesokurtic?
Peak of the curve
What goes on the x axis?
The IV scores
What is the x axis also known as?
The abscissa
What goes on the Y axis?
The frequency of the DVs
Whats the Y axis also known as?
The ordinate
What’s a scattergram (aka scatterplot)?
A pictorial diagram or graph of 2 variables being correlated
What’s the John Henry Effect?
A threat to the internal validity of an experiment that occurs when subjects strive to prove that an experimental treatment that could threaten their livelihood really isn’t all that effective
What is the variance?
A measure of dispersion of scores around some measure of central tendency. It is the standard deviation squared.
How do you calculate the standard deviation?
It is the square root of the variance
What are Z-scores?
They are the same as standard deviations, they are sometimes called standard scores
What are T scores?
T scores have a mean of 50 with every 10 points landing at a standard deviation above or below the mean
What does platykurtic refer to?
The peakedness of a frequency distribution
What does a platykurtic distribution look like?
It is flatter and more spread out than the normal curve
What does a leptokurtic distribution look like?
It is when the curve is very tall, thin and peaked (distribution leaps tall buildings)
What is a nominal scale?
- It is qualitative
- Used to distinguish logically separated groups
- Classifies, names, labels and identifies by group
- No order, or zero point
What is an ordinal scale?
- Rank orders variables
- Relative distance between elements is not always equal
What is an interval scale?
- Has numbers scaled at equal distances but has no absolute zero point
- You can add or subtract
- Most tests used in school fall into this category
What’s an example of an interval scale?
IQ tests
What’s a ratio scale?
- An interval scale with a true zero point
- Can add, subtract, multiply or divide
What are some examples of a ratio scale?
- Time
- Height
- Weight
- Temperature
What does 2X3 refer to?
- Factorial notation
- First variable has 2 levels (ex: boy, girl)
- Second independent variable has 3 levels
Whats the Rosenthal Effect?
The experimenter expectancy effect. Occurs when the experimenters beliefs about the individual may cause the individual to be treated in a special way so the individual begins to fill the experimenters expectations.
What is the acceptable return rate for questionnaire surveys?
50-75% (with a sample size of at least 100)
What are some survey problems?
- Low return rates
- Lack of random subjects
- Not representative of the population
- Poor construction of the instrument
Whats the Hawthorne Effect?
When someones performance improves merely bc they are apart of an experiment. The subject reacts to the presence of the investigator.
Whats the Halo Effect?
The Halo effect occurs when a trait which is not being evaluated (ex: attractiveness) influences a researchers rating on another trait
Whats a trend analysis?
- Refers to a statistical procedure performed at different times to see if a trend is evident
- Describes an application of the ANOVA to see if performance on one variable mimics the same trend on a second variable
What is statistical regression?
Predicts that very high and very low scores will move toward the mean if a test is administered again
What is a quartile?
Refers to the points that divide a distribution into 4ths
What is a cross-sectional study?
Clients are assessed at one point in time
Whats ipsative mean?
Implies a within person analysis rather than a normative analysis between individuals (looking at an individuals own patterns)
What’s a chi-square used for?
Its used to determine whether an obtained distribution differs significantly from an expected distribution. Must have mutually exclusive categories.
What is random sampling?
Each subject has the same probability of being selected and the selection of one subject does not affect the selection of another
What is stratified sampling?
When you select based on special characteristics which need to be represented (ex: race, gender, educational degree, age)
What is quota sampling?
A type of stratified sampling where a specific number of cases are needed for each strata
What is a cluster sample?
Uses an existing sample or cluster of people or selects a portion of an overall sample (ex: high school seniors)
What is systematic sampling?
With this approach you take every nth person
What’s an operational definition?
It outlines a procedure so that other researchers can attempt to replicate the experimental procedure
What is a matched design?
The subjects are “matched” in regard to any variable that could be “correlated” with the DV
What is inductive logic/reasoning?
Research that goes from specific to generalization
What is deductive logic/reasoning?
Reduces the general to the specific
What is the standard error of measurement (SEM)?
Tells the counselor what would most likely occur if the same individual took the same test again
