Research and Program evaluation Flashcards
1
Q
What are some trends in counseling research?
A
- More studies seem to be sporting multiple authors and female authors
- Increased attention to multicultural issues
- Field-based professionals and practitioners submitting fewer contributions
- Meta-studies used to summarize findings related to a given topic or theme
- A majority of studies use grad students and adults as subjects
- qualitative research (as done by Freud and Piaget) seems to be making a comeback.
- N1= single subject designs seem to be making a comeback
- counselors and grad students feel they need more training in APA publication guidelines to submit to journals
- Kurt Lewin’s concept of action research is popular.
- Using the internet to conduct an experiment (rapid data collection, lower research costs, large sample sizes)
- Neuroscience is being used to guide diagnostic and treatment procedures.
2
Q
What is Cohen’s d effect size (ES) statistic?
A
This statistic is used to gauge how strong a relationship exists (small .2, medium .5, large .8)
3
Q
Why are N=1 single subject designs making a comeback?
A
- Only one person is required and counselors are interested in individual change
- The setting is usual real world vs. lab
- Generally easier for consumers of mental health services to understand this type of studies since typically don’t need complex statistical analysis
A common single-subject (N=1) design uses the ABABA design. This model tracks the client with an extended baseline throughout treatment to outcome. Single subject research is idiographic while studies of groups to discover general principles are called nomothetic
4
Q
What is Kurt Lewin’s concept of action research?
A
This is a type of research intended to improve the situation (vs. just advance knowledge) with local people/clients who will be better off at the end of the research. Self-surveys are often used to conduct Action Research. Action research bridges the gap between research and application/practice.
5
Q
What is the most valuable type of research?
A
the experiment, used to discover cause and effect relationships. Experimental research is the process of gathering data to make evaluative comparisons regarding different situations.
6
Q
What must an experiment have?
A
An experiment must have the conditions of treatment controlled via the experimenter and random assignments. An experiment attempts to eliminate all extraneous variables.
7
Q
What is a quasi-experiment?
A
A quasi experiment is like a true experiment BUT the groups are not ramdomly assigned. In a quasi experiment, you cannot state with any degree of statistical conference that the IV caused the DV (dependent variable). One popular type of quasi experiment is known as the ex post facto study or causal comparative study. Ex post facto means after the fact, connoting a correlations study or research in which intact, preexisting groups are utilized. This, however, can threaten internal and external validity.
8
Q
What is internal validity?
A
This refers to whether the dependent variables were truly influenced by experimental independent variable or whether other factors had an impact.
9
Q
What is external validity?
A
This refers to whether the experimental research results can be generalized to larger populations (I.e. other people, settings, or conditions).
10
Q
What is a factor analysis?
A
This is a statical procedure that uses the important or underlying “factors” in an attempt to summarize a lot of variables. So a test which measures a counselors ability may try to describe the 3 most important variables (“factors”) that make an effective helper, although literally hundersds of factors may exist. USung factor analysis procedures, a brief test that measures the 3 major factors may be able to predict who will be an effective counselor as accurately as 10 other tests that examine hundreds of variables/factors.
11
Q
What is a chi-square?
A
This is a nonparametric statistical measure that tests whether an obtained distribution differs significantly from the distribution that the researcher expected. You must have mutually exclusive categories to use a chi square.
12
Q
What is parsimony?
A
Parsimony is when you interpret results in the simplest way. In research, we strive for parsimony in that the easiest and least-complex explanation is said to be the best. The simplest explanation of finding is always preferred. For example, factor analysis is parsimonious in the sense that it is concerned with data reduction.
13
Q
What is Occam’s razor?
A
It suggests that experimenters interpret results in the simplest manner.
14
Q
What are “bubbles” in research?
A
This refers to flaws in research – like when you try to put a sticker on a car and there are always air bubbles.
15
Q
What is a confound?
A
This is when an undesirable or excess variable “confounds” or flaws the experiment (I.e. if there is an experiment testing a new form of therapy but the person is seeing another therapist on the side). The only experimental variable should be the independent variable (IV). A confounded experiment is, by nature, invalid.
All correlational research is said to be confounded.
16
Q
What periodical publishes more counseling research articles than any other periodical in our field?
A
The APA’s Journal of Counseling Psychology.
17
Q
What is the difference between basic research and applied research?
A
Basic research is conducted to advance our understanding of theory. Applied research (aka action research or experience-near research) is conducted to advance our knowledge of how theories, skills, and techniques can be used in practical application. Often counselors asserts that much of the research isn’t relevant to the actual counseling process.
18
Q
What is an independent variable?
A
An independent variable (IV) is a variable that the researcher manipulates, controls, alters or wises to experiment with.
19
Q
What is a dependent variable (DV)?
A
A dependent variable expresses the outcome of the data.
20
Q
What is a causal comparative design?
A
This is a type of experiment that is a true experiment EXCEPT that the groups are not randomly designed. Data gland from a causal comparative ex post facto (after the fact) design can be analyzed with a test of significant like a t test or anova, just like any true experiment.
21
Q
What does it mean for an experimenter to be guided by ethics?
A
In all Experiments, a counselor/researcher should be guided by ethics. This means:
- subjects are informed of any risks
- negative after-effects are removed
- subjects can withdraw at anytime
- confidentiality will be protected
- results of research reports will be presented in an accurate format that isn’t misleading
- the counselor/researcher will only use techniques they are trained in
22
Q
What are control groups and experimental groups
A
Both the control and experimental groups should have the same characteristics except the members of the control group will not have the experimental treatment applied to them. In an experiment, the control group does not receive the independent variable (IV). The experimental group receives the independent variable. The basic supposition is that the averages (means) of the groups do not differ significantly at the beginning of the experiment.
Note that if you cannot randomly assign the subjects to two groups, the research will be considered a quasi-experiment.
23
Q
What is an organismic variable?
A
This is a variable that a researcher cannot control, yet exists - such as heigh, weight, or gender. To determine whether there is an organismic IV, you must ask yourself if there is an experimental variable being examined that you cannot manipulate.
24
Q
What is hypothesis testing and who pioneered it?
A
Hypothesis testing was pioneered by RA Fisher. A hypothesis is a hunch or an educated guess which can be tested utilizing the experimental model – so it is a statement which can be tested regarding the relationship of the independent variable and dependent variable.
25
What is a null hypothesis?
This hypothesis suggests that there will not be a significant difference between the experimental group which received the IV and the control group which did not. Essentially, the null hypothesis is simply that the IV did not impact the DV.
26
What is a meta analysis?
A meta analysis is a study that analyzes the results of numerous studies.
27
What is an alternative hypothesis (or affirmative hypothesis)?
This asserts that the independent variable has indeed caused a change in the dependent variable.
28
What is a test of significance?
This is a type of statistical test that is used to determine whether a difference in the groups' scores is “significant” or just due to chance factors. In this case, a **t test** would be determined if a significant different between two means exists. This has been called the **two groups** or **two randomized groups** research design. In this study, the two groups were independent of each other in the sense that the change (or lack thereof) in one group did not influence the other group, those is is known as an independent group comparison. If the researcher had measured the same group of subjects without the IV and with the IV, it would be known as a **repeated measures comparison design.**
29
What is a correlation coefficient?
This is a way of measuring correlational research
30
What is a “between subjects design”?
This is when a research study uses different subjects for each condition. If the same subjects are employed, it could be referred to as a **within subjects design.**
31
What does *P* mean in relation to a test of significance?
*P* in this context means probability or the level of significance. Traditionally, the probability in social science research has been set at .05 or lower (I.e. .01 or .001). The .05 level indicates that differences would occur via chance only 5 times in 100. The significant level must be set before the experiment begins!
The smaller the value of *P* the more stringent the level of significance.
32
What is a parameter?
A parameter is technically a value obtained from a population which a statistic is a value drawn from a sample. A parameter summarizes a characteristic of a population (I.e. the average male's height is 5'9).
33
What is the accepted probability level in social sciences?
.05 or less. The two most popular levels of significant are .05 and .01
34
What does *P* = .05 really mean?
That there is only a 5% chance that the difference between the control group and the experimental group is due to chance factors. This could also be referred to as a **95% confidence interval** which means that the results would be due to change only 5 times out of 100. When *P* = .05, differences in the experimental group and the control group are evident at the end of the experiment and the odds are only 1 in 20 that this can be explained by change. **Reminder:** ***P*****= level of significance (aka level of confidence or confidence level)**
35
What are Type I and Type II errors?
* A **type I error,** also known as an **alpha error,** occurs when a researcher rejects the null hypothesis when it's true
* A **type II error,** also known as a **beta error,** occurs when you accept a null hypothesis when it's false.
The probability of committing a type I error (saying there is an experimental impact when there's not) equals the **level of significance (*****P*****).** Therefore the level of significance is often referred to as the **alpha level**.
36
What is the power of a statistical test
the **power of a statistical test** is what it's called when you calculate 1-**beta** (Beta errors). In this respect, “power” connotes a statistical test's ability to reject correctly a false null hypothesis.
Parametric tests have more power than nonparametric statistical tests. Paremetric test are used only with interval and ratio error.
37
How does increasing sample size impact Type I and Type II errors?
Increasing sample sizes helps to lower the risk of chance/error factors. Differences revealed via large samples are more likely to be genuine than differences revealed using a smaller sample size.
38
How do alpha and beta errors increase or decrease if a researcher changes the significance level from .05 to .001
**Alpha errors would decrease** (I.e. there is a higher standard to prove that the IV had an impact, so it is less likely that you would think something happened when it really hadn't) BUT because there's a higher threshold to prove that, so you risk thinking nothing happened when it really had, so **beta errors would increase.**
39
What is a *t* test?
A *t* test is a simplistic form of the analysis of variance. The *t* test is used to ascertain whether two sample means are significantly different. The researcher sets the level of significance and then runs the experiment. The *t* test is computed using 3 key data values: the difference between the mean values from each data set, the standard deviation of each group, and the number of data values in each group. This yields a *t* value.
The researcher then goes to a *t* table found in the index of most statistics' texts. If the *t* value obtained statistically is lower than the *t* value (sometimes called the **critical** ***t***), in the table, then you'll accept the null hypothesis. The computation must exceed the number cited in the table in order to reject the null.
The *t* test only applies when there are two groups. If there are more groups you need to use the **analysis of variance (ANOVA).**
40
What is ANOVA?
ANOVA, the **analysis of variance,** is used to figure out whether there are specific differences between more than 2 groups (if there are only 2 groups, can use a t test). The results of an ANOVA field an **F-statistic.** The researcher then consults an *F* table for a critical value of *F.* If the *F* obtained exceeds the critical *F* value in the table, then the null hypothesis is rejected.
This is considered a **one-way test** which is used for testing one independent variable. If there are two independent variables, you could use a **two-way ANOVA (analysis of variance),** if there are 3 IVs, you would use a **three-way ANOVA, etc.** These multi-way ANOVAs are also called **MANOVA.**
41
What are the major statistical tests (to test for significance difference)?
* ***t*** **test -** tests for significant difference between two groups. Compares *t* obtained to a *t* table for a **critical** ***t*** value
* **ANOVA (analysis of variance) -** tests for significant difference between more than two groups. Yields an ***F*** **statistic** - researcher compares to an *F* table for a critical value of *F.*
* **ANCOVA (analysis of covariance)** - tests two or more groups while controlling for extraneous variables that are often called **covariates**
* **Kruskal-Wallis** - used instead of the one-way ANOVA when data are nonparametric and you wish to test whether two correlated means differ significantly
* **Mann-Whitney** ***U*** **test -** used to determine whether two uncorrelated means differ significantly when data are nonparametric
* **Spearman correlation** or **Kendall's tau -** Used in place of the Pearson *r* when parametric assumptions cannot be utilized
* **chi-square nonparametric test -** examines whether obtained frequencies differ significantly from expected frequencies
42
What is MANOVA?
This is a test for statistical significnance. MANOVA stands for **multivariate analysis of variance** to test more than one dependent variable. Two-way and three-way ANOVAs are considered MANOVAs.
43
What is a correlation coefficient?
This is how the results of a correlation experiment is reported and is a statistic that indicates the degree of magnitude of linear relationship between two variables. The **correlation coefficient** is often abbreviated using an ***r*** **.** A coefficient of correlation makes a statement regarding the association of two variables and how a change in one is related to a change in the other. Correlations range from 0.00 (no relationship) to 1.0 or -1.0 which signify perfect relationships.
Note that a positive correlation is not a stronger relationship than a negative one of the same numerical value. A correlation of -.70 is still indicative of a stronger relationship than a positive relationship of .60. The minus sign merely describes the fact that as one variable goes up, the other goes down.
The **Pearson Product-moment correlation** ***r*** is used for interval or ratio data while the **spearman rho correlation** is used for ordinal data.
44
What are positive and negative correlations?
Positive correlation is evident when both variables change in the same direction (test scores go up when studying goes up). A negative correlation is evident when the variables are inversely associated - one goes up and the other goes down.
45
What are the terms bivariate and multivariate?
A **bivariate** correlational paradigm means it is a paradigm that is trying to describe the nature and relationship of two variables. If the researcher is looking at more than two variables, they would use the term **multivariate** to describe the correlational paradigm.
46
What is an N = 1 study and when might it be used?
This is a study with only one person in it. This is basically a case study approach and is popular with behaviorists who seek out overt behavioral change. The client's dysfunctional behavior is measured (a **baseline measure),** the treatment is implemented, and then the behavior is measured again (a new baseline). This is sometimes delineated using A, B, C in which A = baseline, B = intervention implementation, and C = second or alternative form of intervention.
These studies are often called **idiographic studies** or **single subject designs**.
The original case study methodology was popularized by Freud, though he did not rely on baseline measures. Case studies can be misleading because the results are not necessarily generalizable.
47
What are single-blind and double-blind experiments?
In a **single blind** experiment, the subject wouldn't know if she is a member o the control of experimental group - but the researcher does know.
A **double blind** study is one in which both the participant and the researcher don't know which group the participant is in.
48
What is the participant-observer model?
This is a study in which the researcher actually participates in the study while making observations about what transpired.
49
What are experimenter effects?
This something that can confound an experiment. Experimenter effects can flaw an experiment because the experimenter might unconsciously communicate his or her intent or expectations to the subjects
50
What is an AB or ABA time-series design?
Also known as **typical AB design,** this is the simplest type of single-subject research and was initially popularized by behavior modifiers in the1960s and 1970s. single subject case studies are one again gaining popularity.
In AB and ABA models that rely on **continuous measurement:**
* **A -** a baseline is secured
* **B -** the intervention is implemented
* **A -** the outcome is examined via a new baseline.
you can also use an **ABAB** design to better rule out extraneous variables – if the pattern for the second administration of AB mimics that of the first AB, it is more likely that the treatment (B) caused the changes than an extraneous variable. Some refer to ABA or ABAB paradigm as **withdrawal designs** because of the assumption that the behavior will move towards the initial baseline each time the treatment is withdrawn. When a researcher is looking at more than one target behavior, it is called a **multiple baseline design.**
51
What is a bell-shaped curve?
This is a normal curve that is symmetrical showing that something is normally distributed. Most physical and psychological traits are normally distributed, so most tests looking at physical and psychological traits will resemble a bell curve.
## Footnote
**Remember that in a normal distribution, 68% of the scores will fall within +/- 1 standard deviation of the mean, 95% will fall within 2 standard deviations of the mean and 99.7 will fall within 3 standard deviations of the mean – almost all scores will fall between 3 standard deviations of the mean.**
52
What are the mean, median, mode, and range?
* **Mean -** this is the arithmetic average
* **Median -** the middle score in the distribution of scores. If a distribution has an even number of scores, you take the mean of the two middle scores and use this as the median.
* **Mode -** the most frequently occurring score and the least important measure of central tendency.
* **Range -** the measure of variability – the distance between the largest and smallest scores. The larger the range, the greater the dispersion or spread of scores from the mean.
53
What is a bimodal distribution?
This is a distribution that has two modes. Graphically this would look like a camel's back with two bumps.
When a curve has more than two peaks, it is known as a **multimodal distribution.** A curve with a single peak (I.e. a normal curve) can be called **unimodal.**
54
What is the most useful measure of central tendency?
The mean, often abbreviated by an X with a bar over it. That said, if the distribution has a lot of extreme scores, the median is the statistic of choice (it is best for skewed distributions).
55
What is a skewed distribution?
This is when a distribution of scores is not distributed normally. In a skewed distribution, the left and right sides of the curve are not mirror images. And in a skewed distribution, the mean, median and more fall at different points whereas in a normal curve, they will fall at the same point.
56
What is a factorial design?
A factorial experiment includes two or more independent variables. Sometimes the IVs in a factorial design are called **levels** though have no correlation with hierarchy. In a factorial experiment, you can experiment several experimental variables and note several interactions.
57
What does it mean to be positively or negatively skewed?
If scores tend very low, something would be **positively skewed** (because most of the scores would fall on the left or low side of the distribution so the “tail” would point to the positive side or to the right). If scores tend very high, something would be **negatively skewed**.
58
What is the benefit of **standard scores** (percentiles, t-scores, z-scores, staines, or standard deviations) over **raw scores?**
A standard score allows you to analyze the data in relation to the properties of the normal bell-shaped curve. A raw score just gives you a number, but it's unclear what that number indicates (I.e. if someone gets an 82 on a test, we don't know how many questions there were, how many points per question, how things are weighted, etc)
59
What is a histogram?
This is a bar graph that shows how often a value in a set of data occurs. When bars are drawn horizontally, it is sometimes called a **horizontal bar chart.** A **double-barred histogram** can be used to compare two distributions of scores such as pre- and posttest scores.
60
What is mesokurtic?
Mesokurtic is when a curve peaks in the middle (meso). The normal Gaussian curve is mesokurtic since it peaks in the middle
61
What are other names for the x and y axes?
The **x-axis** can also be called the **abscissa** and is used to play the independent variable. The **y-axis,** also called the **ordinate**, is used as a scale for the dependent variable.
62
What might a bimodal distribution imply about the population you are working with?
It may imply that you are working with two different populations
63
What is a naturalistic observation?
Naturalistic observation occurs when clients are observed in a natural setting or situation.
64
What is ethological observation?
This refers to the observation of animals.
65
How do you calculate the range of a set of data?
The range is calculated by determining the difference between the highest and lowest scores (**high score - low score = range**). This is referred to as **exclusive range.**
BUT some define the range as: **high score - low score + 1 = range.** This is referred to as **inclusive range.**
66
What is a scattergram?
This is a pictorial diagram or graph of two variables being correlated. This could be used to visualize a **correlation coefficient.**
67
What is the John Henry effect?
The John Henry Effect, also called **compensatory rivalry of a comparison group,** is a threat to the internal validity of an experiment that occurs when subjects strive to prove that an experimental treatment that could threaten their livelihood really isn't that effective. (I.e. if an experimental treatment could put the subjects out of a job, they might try to make it ineffective).
68
What is the **Resentful Demoralization of the Comparison Group?**
This is another control group phenomenon that threatens internal validity in research, also called **compensatory equalization.** Here, the comparison group lowers their performance or behaves in an inept manner because they have been denied the experimental treatment. When this happens, the experimental group looks better than they should. If the comparison group deteriorates throughout the experiment while the experimental group does not, this could be an effect of demoralization. This can be measured pre and post test.
69
How does sample size impact range?
The range generally increases with sample size.
70
What is variance?
The variance is a measure of the dispersion of scores around some measure of central tendency. The variance is the **standard deviation squared.**
71
What is standard deviation?
The standard deviation is the amount +/- the mean that 68% of people fall within. in a test with a mean of 100 and a SD of 15, 68% of people would score between 85 and 115. If 100% of scores were the mean, then the SD would be 0.
## Footnote
**z-scores are the same as standard deviation. z-scores are also sometimes called standard scores.**
72
What is a **t-score?**
A t-score, also known as a **transformed score**, uses a mean of 50 with each standard deviation as 10. So a z-score (SD) of -1 would mean a t-score of 40. Or a SD/z-score of +1 would mean a t-score of 60.
73
What is a platykurtic distribution?
A platykurtic distribution is flatter and more spread out than the normal curve (flat = plat). It might look like the upper half of a hotdog lying on its side over the x-axis (**abscissa)**
74
What is a leptokurtic distribution?
This is a graphical distribution style that refers to a very tall, thin, peaked curve. (tip: a leptokurtic distribution leaps tall buildings)
75
What are stanine scores?
Stanine scores divide the distribution into 9 equal intervals, with stanine 1 as the lowest ninth and 9 as the highest ninth. In this system 5 is the mean.
So for example, a Binet IQ score of 101 would fall in stanine 5 because it is super close to the mean IQ score which is 100.
76
What is a nominal scale?
A nominal scale is a strictly qualitative scale that is used to distinguish between separated groups. For example, the DSM is a nominal scale, addresses, political parties, etc. A nominal scale has no true zero point and does not indicate order.
77
What are the four types of measurement scales, in order of complexity?
**NOIR:**
* **Nominal -** grouped categories, no zero point, doesn't indicate order
* **Ordinal -** rank orders variables, though doesn't have an equal distance between elements
* **Interval -** numbers are scaled at equal distance points, though there is no set zero point
* **Ratio -** an interval scale with a true zero point
78
What is an ordinal scale?
An ordinal scale rank orders variables, though the relative distance between elements is not always equal like, for example, a horse that comes in second in a race (not set distance between first and second). This is the second level of measurement and it DOES rank order data but does not indicate absolute distances.
## Footnote
**The ordinal scale provides relative placement or standing but does not delineate absolute distance.**
79
What is an interval scale?
The interval scale has numbers scaled at equal distances but have no absolute zero point. Most school tests fall into this point. You can add and subtract using interval scales but cannot multiple or divide. So, for example, an IQ of 70 is 70 points below an IQ of 140 but a counselor could not assert that a client with an IQ of 140 is twice as intelligent as a client with an IQ of 170.
Since the intervals are the same, the amount of difference can be stimulated. Using this scale, distances between each number are equal yet it is unclear how far each number is zero.
(for the IQ test, a zero on an IQ test does not equal zero knowledge so IQ tests provide interval measurements.)
80
What is a ratio scale?
A ratio scale is an interval scale with a true zero point – so radio measurements are possible using this scale. Addition, subtraction, and division can all be utilized on the radio scale.
This is the highest level of measurement. Time, height, weight, temperature on the Kelvin scale, volume, and distance all meet the requirements of this scale. Most psychological attributes cannot be measured on a ratio scale.
81
What is a 2x3 factorial notation?
This is an experiment design in which the first variable has two levels (I.e male or female) and the second independent variable has 3 levels.
82
What is a survey and what is a required completion rate for accuracy?
A survey is the simplest form of descriptive research. In order to be accurate, a survey must have a return rate of 50-75% and ideally have a sample size of over 100 people.
However, survey problems include poor construction of the instrument, a low return rate, and the fact that often subjects are not picked at random and thus are not representative of the population.
83
What is a nocebro?
A thing or medication that has a negative effect on an experiment because it harms the patient and/or creates negative expectations - like when a doctor comments that someone with a certain condition only has 6 months to live. However, note that a statistic does not actually connote whether or not something will happen to you.
84
What is the Hawthorne effect?
The Hawthorne effect is based on research that noticed that work production increased with both better and worse lighting – essentially, that performance can improve when participants simply know they are part of an experiment or with the attention of being in the experiment.
This is an example of a **reactive effect** or **reactivity**, when the subjects' behavior is influenced by the very presence of a researcher. This is also known as the **observer effect**
**Note that the Hawthorn Effect is on the participant.**
85
What is the Rosenthal Effect?
The Rosenthal effect or **experimented expectancy effect**, asserts that the experimenter's beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter's expectations.
Note that the Rosenthal effect is on the researcher.
86
What is the Halo effect?
The halo effect occurs when a trait that is not being evaluated (I.e. attractiveness, how well someone is liked) influences a researcher's rating on another trait (I.e. counseling skill).
87
What is trend analysis?
Trend analysis refers to a statistical procedure preformed at different times to see if a trend is evident. Some use the term to describe an application of the ANOVA to see if performance on one variable mimics the same trend on a second variable.
For example: you arranged 3 groups of subjects in regard to their ability to respond to reality therapy. Your hypothesis is that clients in each level will respond better to REBT than those on the level below them (I.e. good reality therapy clients will be better REBT therapy clients than average or poor reality therapy clients). ANOVA would allow you to statistically test this hypothesis.
88
What is ANCOVA (analysis of covariance)?
ANCOVA is similar to ANOVA but more powerful because it can help to eliminate differences between groups which otherwise could not be solely attributed to the experimental independent variables. In other words, although ideally each random sample will be equal to all other random samples, this is not always the case – a **covariate**, which correlates to the DV, could be present. (I.e. if you're testing for test anxiety but one group has a lot of athelets who are used to dealing with performance anxiety). The ANACOVA allows you to correct for the differences in the groups. By making the groups more alike upfront, it will enhance the possibility that the independent variables (I.e. meditation, self-hypnosis) rather than a covariate (athletic training) caused the differences in the groups.
## Footnote
**Essentially, ANVOCA tests a null hythpoesis regarding the means of two or more groups after the random samples are adjusted to eliminate average differences.**
89
What is causal comparative research?
This is a type of quasi-experiment in which you look at a correlation *after* something has occured. This is also known as an **ex post facto** design. This means that groups were not able to be randomly selected and you could not truly control the IV. (could be something like looking at effects of a group of people in a treatment center who elect to try one form of therapy).
90
What is a cohort study?
A cohort study examines people who were born at the same time or shared an event in regard to a given characteristic - I.e. e how many women born during the great depression later committed suicide.
91
What is the concept of statistical regression toward the mean?
Statistical regression predicts that very high and very low scores will move towards the mean if a test is administered again. This is a threat to internal validity. (I.e. unusually low scores on exams will improve whereas unusually high scores will get better).
92
What is standardized about standardized tests?
They always have formal procedures for test administration and scoring.
93
What is a longitudinal study?
This is a study where the same people are studied over a period of time. This can also be referred to as a **diachronic method**. In a longitudinal study, data are collected at different point in time – and age itself can be used as the IV.
94
What is a cross-sectional study?
In a cross-sectional study, data are indicative of measurements or observations at a single point in time and thus is is preferable in terms of time consumption. This can also be referred to as the **synchronic method.**
95
What is it called when the researcher has an accomplice pose as a client and act in a certain manner?
The person is known as a **confederate** or a **stooge.** Social psychology studies routinely employ confederates or stooges who are not real participants but in reality work with the researcher.
96
What are demand characteristics of experiments?
A demand characteristic refers to any bit of knowledge – correct or incorrect - that the subject in an experiment is aware of that can influence his or her behavior. Demand characteristics can confound an experiment.
97
What is summative or formative evaluation?
**Summative evaluation** is used to assess a final product (I.e. how many high school students are not indulging in alcoholic beverages after completing a yearly program about drug awareness). Summative research attempts to ascertain how well a goal has been met - while **formative process research** is ongoing while the program is underway (I.e. after 3 weeks of a proposed year-long program, how many students are taking drugs).
98
What is counterbalancing?
This is when you switch the order in which stimuli are presented to a subject in a study. This is used to control for the fact that the order of an experiment could impact its outcome – so the solution is simply to change the order of experimental factors.
99
What is the pygmalion effect?
The pygmalion effect is when the experimenter falls in love with his or her own hypothesis and the experiment becomes a self-fulfilling prophecy.
One way this can manifest is the **Rosenthal/experimenter effect** in which the experimenter's beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter's expectations.
100
What is ahistoric therapy?
Ahistoric therapy connotes any psychotherapeutic model that focuses on the here-and-now rather than in the past.
101
What is multiple treatment interference?
Multiple treatment inference is when a subject receives more than one treatment and the effect is that it is often tough to discern which modality truly caused the improvements.
102
What is ERIC?
This is the Educational Resources Information Center (ERIC) that is a resource bank of scholarly literature and resources to help you complete your literature search before writing. Includes both primary and secondary resource.
103
What is random sampling?
This is like sticking your hand in a fish bowl to pick a winning ticket. In a random sample, each subject has the same probability of being selected and the selection of one subject doesn't affect the selection of another subject. The simply random sampling procedure eliminates the researcher's tendency to pick a biased sample of subjects.
104
What is stratified sampling??
**Stratified sampling** is a type of sampling is used when a special characteristic (race, gender, educational degree, age, etc) needs to be represented. The stratification variable in the sample should mimic the population at large.
In a research situation where a specific number of cases are necessary from each stratum, the procedure is called **quota sampling**. Quota sampling is merely a type of strategies sampling procedure.
105
What is cluster sampling?
Cluster sampling is used when it is nearly impossible to find a list of the entire population. The cluster sample solves the problem by using an existing sample or cluster of people or selects a portion of the overall sample. A cluster sample will not be as accurate as a random sample, yet it is often used due to time and practical considerations – for example, it would be impossible to get a list of the entire population.
106
What is horizontal sampling?
Horizontal sampling occurs when a researcher selects subjects from a single socioeconomic group. Horizontal sampling can be contrasted with **vertical sampling** which occurs when people from two or more socioeconomic classes are utilized. Horizontal and vertical sampling are only relevant when socioeconomic factors are specified.
107
What is snowball sampling?
Snowball sampling or **chain-referral sampling** using subjects to drum up other subjects for the study. For example, if you were studying heroin addicts living on the street, you couldn't find them without the help of others in the same predicament.
108
What is systematic sampling?
With systematic sampling, you take every nth person. So if you have 10,000 people and want 1,000 people in your study you would pick the first person between 1 and 10 at random and then use every 10th person. According to some statisticians, your results will be virtually the same as if you used **random sampling**. While the random vs. true debate rages, you **can use systematic sampling in a true experiment.**
109
What is sampling error?
Small samples intended to mimic the population sometimes do not – the sample may be skewed and not big enough to account for this! The margin of error stated in political poll results is based on this concept.
110
What is an axiom?
An axiom is s universally accepted idea that needs no additional proof (I.e. gravity exists on earth).
111
What are some examples of nonparametric statistical measures?
In nonparametric testing, the curve is not a normal distribution. Some nonparametric tests are:
* Mann-Whitney *U* test
* Wilcoxon signed-rank test for matched pairs
* Soloman and the Kruskal Wallis *H* test
112
What is parametric and nonparametric testing?
In a **parametric test** the assumption is that the scores are normally distributed on a curve. In **nonparametric testing**, the curve is not a normal distribution.
113
What is a matched design?
In a matched design, the subjects are matched in regard to any variable that could be correlated with the DV which is really the post-experimental performance. For example, if you wanted to test a hypothesis about a new treatment for BPD but felt that IQ might be correlated with the adherence to the new treatment, you would try to match subjects based on IQ. So basically, this would make it so that IQ did not confound the study.
Special kinds of matched designs include **repeated measures** or **within-subjects** design in which the same subjects are used, once for the control and again for the experimental conditions.
**Note that unmatched/uncorrelated groups could be known as independent groups.**
114
What is inductive and deductive logic?
* **Inductive logic or reasoning** - research goes from the specific (I.e. a single case study) to a generalization
* **deductive logic or reasoning -** research reduces the general to the specific. (I.e. a client says he doesn't want to do a new addiction treatment program because he already knows the 12 steps – assumes that this specific one will be like the general one)
115
What is the standard error of measurement (SEM)?
The SEM tells the counselor what would most likely occur if the same individual took the same test again.
116
What is a Likert Scale?
A Likert scale is one that asks questions to figure out the overall degree of measurement, so would have options like,: strongly agree, somewhat agree, neutral, somewhat disagree, or strongly disagree.
117
What is internal vs. external validity?
**Internal validity** refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors or variables. (I.e. does it impact whether or not we can tell causality?)
**External validity** refers to the extent to which results from a study can be applied (generalized) to other situations, groups or events.
Threats to **internal** validity compromise our confidence in saying that a relationship exists between the independent and dependent variables.
Threats to **external** validity compromise our confidence in stating whether the study’s results are applicable to other groups.
118
What is internal vs. external validity?
**Internal validity** refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors or variables. (I.e. does it impact whether or not we can tell causality?)
**External validity** refers to the extent to which results from a study can be applied (generalized) to other situations, groups or events.
Threats to **internal** validity compromise our confidence in saying that a relationship exists between the independent and dependent variables.
Threats to **external** validity compromise our confidence in stating whether the study’s results are applicable to other groups.
