545- Graduate Research Design and Program Evaluation Flashcards
Construct
Part of research design, it is a collection of either behaviors, thoughts, or feelings that we define in some way. Behaviors are objective and able to be directly observed versus a construct which is more abstract and we have to define it. A behavior can be included as a part of the construct. So much of what we work with in psychology are constructs- can be conflated with biases because they are more subjective and abstract.
Clinical example: resiliency is a construct that we can define in behavioral terms.
Correlational research
Purpose is to just examine relationships between variables. Can do this in different ways, simultaneously or longitudinally. We can also answer the question about predictive relationships. Limitations include not being able to determine causation- one doesn’t make the other. It is important to be able to distinguish between correlational research and experimental research when researching treatments.
Example: A researcher is interested in examining the relationship between social media usage and self-perception among young adult populations.
Criterion-referenced scoring/tests
Criteria is based on cut scores of achievement versus norm-referenced where its just compared to the norm. With cut scores, there is a threshold and if they don’t meet the criterion but did better than everybody else, it is safe to say that they did better than their peers. Cut scores determined from some outside standard- no arbitrary criterion (also pretty much based on the norm).
Clinical example: looking about discrepancies between intelligence and achievement- both based on norm-referenced assessments, with criterion referenced, you would have to fall below a certain percent to even have that can be connected to learning disabilities.
Criterion-related validity
When you compare your measure to other measures to see where they converge or diverge. Convergent is where your measure aligns with a similar measure. Divergent is where your measure compares to an opposite measure. This is important for clinicians to be able to select and use measures in practice so that we can all be data-based decision makers.
Example:
Cross validation
Part of research methods, this is a procedure used to assess the stability of a statistical model. The process of cross-validation is dividing a data set into two subsets randomly where the first helps develop the model and the second tests the model. The first helps with consistency and reliability and the second set helps with the norming sample.
Example: In regression analysis, the first subset is analyzed to develop a regression equation and then applied to the remaining subset to see if it predicts the scores actually observed.
Cross-validation: This is a procedure used to assess the utility or stability of a statistical model. A data set is randomly divided into two subsets, the first of which is used to develop the model and the second of which is used to test it. In regression analysis, the first subset would be analyzed in order to develop a regression equation, which would then be applied to the remaining subset to see how well it predicts the scores that were actually observed.
Cross-sectional design
A type of observational study design. Data from both variables are collected at the same time. Simultaneous. Individuals are compared at a single point in time. Difficult to determine causal relationships. Not suitable for observing changes over time- would need a longitudinal study. Paired samples.
For example, any of our research projects. A group of five-year-olds is compared to a group of eight-year-olds.
Double-blind study
Part of research design, blind refers to a lack of awareness in a study and is employed to enhance experimental control. A double-blind study is one in which both the participants in the study as well as the experimenters interacting with them are unaware of experimental conditions.
Example: In a drug clinical trial, the participant nor the experimenter know if the participant receives the drug or the placebo.
Ecological validity
Part of applied research, ecological validity is the degree to which results obtained from a study apply to real world examples. Ecological validity is threatened by experimenter bias, oversimplification of a real-world situation, or poor sampling strategies. This is important because we want our research to be representative of the population it applies to in order to draw conclusions of whether something works or not.
Example: Research carried out on college students cannot be applied to the whole population, just other college students.
Experimental research
Research that randomly assigns participants to conditions and systematically manipulates variables in order to draw causal inference. It is typically conducted in a controlled environment like the laboratory setting to reduce influence from outside factors. Reducing this influence increases internal validity but decreases external validity. Requires that the independent variable be manipulated and some level of control must be present like some assignment to a group or something. Different types:
Example:
Hypothesis
An educated guess, a tentative answer to our question that could be based on other research. An empirical and testable assumption about a fact, behavior, relationship, usually based upon theory used to predict an expected outcome resulting from conditions or assumptions.
Longitudinal design
The same sample of participants will have data collected from them from multiple points of time. Different data could also be collected from the same sample to predict. Error is eliminated when the same people are looked at for correlations. Within samples. Attrition is a common threat to validity here.
Example: a multiyear, comparative study of children in urban and suburban schools to record cognitive development.
Program evaluation
Part of research methods, this is a process that contributes to decisions on installing, continuing, expanding, certifying, or modifying programs based on how effective they are. It is also used to gain evidence in support of or in opposition to the organization providing services and to contribute to basic knowledge in social or behavioral sciences, about interventions and experiments.
Example:
Program evaluation: This is discussed in graduate research design and evaluation and is a systematic assessment of the outcomes of a program, in order to evaluate its effectiveness and to inform modifications to the program. There is an emphasis on program operation and outcomes so this process if used to improve efficiency of programs. The steps to this are problem identification and definition (needs assessment), program development, program implementation (fidelity) and program evaluation. Needs assessment can occur because of a referred problem or a problem with the basic systems. The development process involves evaluating the severity of the problem, identify outcome goals, and find out more information regarding the issue at hand. Program implementation involves having some sort of checklist to make sure the program is actually being implemented, this might involve some training of the people implementing the new program. The program is evaluated through within or between systems evaluation, process indicators like social validity and treatment integrity, a pre/post-test and outcome measures.
This is important in a school setting if an issue arises among the way a system is working.
Quasi-experimental research
A type of research design in which the investigator cannot randomly assign participants to conditions. When someone comes to us based on a characteristic they can’t change or is not ethical to change.
Example: field research
Random sampling
Part of research conditions, this is the process of selecting the sample of participants from a larger group of eligible individuals so that each person has the same fixed probability of being included. (Every person in the population has a chance at being selected.) A chance procedure is used to determine specifically who is chosen. The group that goes through this particular selection process is then known as a random sample. Using this sampling method positively influences generalizability and external validity. It gives us our best chance at choosing a random sample with influences generalizability.
Example:
Scientific methodology
Ask a question, develop a hypothesis, develop your methods, collect your data, analyze the results. The scientific method and the process of it.
Example:
