Terms Flashcards
quantitative
to study relationships and cause-effect deductive approach; tests theory large samples numerical data narrow scope, statistical analysis, fixed
positivist (quan)
involves testing theories and hypotheses and generalizing findings to larger population
separate research from researcher
NEED MORE
behavioral research (quan)
study the actions or reactions of persons or animals in response to external or internal stimuli
naturalistic (qual)
observes and records some behavior or phenomenon in its natural setting while interfering as little as possible
qualitative
to examine a phenomenon in rich detail inductive; generates theory small sample contextual data broad scope, content analysis, flexible
ethnographic (qual)
involves an in-depth study of naturally occurring behavior within a culture or group
interpretive research
provides descriptive accounts
understand the world or experience of another
Attempt to understand phenomena through accessing the meanings participants assign to them
internal validity (quan)
goes with credibility in qual
the extent to which changes in a DV are related directly and only to the targeted IV
happens with extraneous variables
Threats to Internal Validity (12)
history (unrelated events during study)
maturation (changes within participants)
testing effect (post test)
instrumentation (unreliable or change in instru)
regression (skewed data)
selection (not equal groups)
mortality (loss of participants)
selection-maturation (diff maturation rates in sample)
experimenter effect (unintentional bias or behavior)
subject effect (attitude effect)
diffusion (treatment group talks to control group)
location
credibility (qual)
goes with internal validity in quan
results are believable and trustworthy form the perspective of a participant or subject in the research itself
prolonged engagement, frequent obs, peer deb, triangulation, collect referential adequacy materials, member checks
after: corroboration, referential adequacy, member checks
treatment verification
confirmation/verification of IV
Reasons: determines analysis of findings and drawing conclusion, critical to valid interpretation of effects and estimation of generalizability, needed for relicability
aptitude-treatment interaction in designs
???concept that some instructional strategies (treatments) are more or less effective for particular individuals depending upon their specific abilities
aptitude-treatment interaction in analyses
???should coordinate instructional materials and methods with students’ specific learning characteristics to maximize learning
random sampling vs random selection vs random assignment
sampling: implies that every member of a pop has an equal and indep chance of being selected to participate in the sample
selection: how you draw the sample from pop (external validity)
assignment: how you assign the sample to diff groups or treatments in the study (internal validity)
systematic sampling
population in list, pick each k-th element
convenience sampling
non-probability sampling
willing and available
selected because of their convenient accessibility and proximity to the researcher
purposive sampling
(judgmental sample) subjects selected bc of some characteristic for the purpose of the study
Ex: pick 2 of 10 schools bc think the 2 represent typical adolescent Americans
external validity (quan)
goes with trans in qual
extent to which the results of study can be generalized (apply findings to population)
transferability (qual)
goes with external validity in quan
degree to which the results of qual can be generalized or transferred to other contexts or settings
ecological validity
To what extent will the same findings be obtained under other experimental settings (environmental conditions) or other experimental treatments?
population validity
To what extent can I apply my results to the targeted population?
reliability (quan)
degree of consistency with which it measures whatever it is measuring
goes with dependability in qual
low reliability = high variance
obs score = true score + error
more reliable = less error
consistency or stability of measurement
How well does the instrument measure accurately and consistently?
Can our measure (or other form of observation) be confirmed by further measurements or obser
internal consistency reliability
used to assess the consistency of results across items within the test
a measure of reliability used to evaluate the degree to which different test items that probe the same construct produce similar results
inter-rater reliability (inter-scorer)
a measure of reliability used to assess the degree to which different raters agree in their assessment decisions
validity (quan)
goes with confirmability in qual
valid = unbiased
knowing if the inferences we make about the relationship between the targeted variables are valid
measures what it claims to measure
content validity
the extent to which the test’s items represent the entire body of content that the test is designed to measure
How adequately do the test items represent the entire body of content the test user intends to make inferences about?
Also deals with formatting: clarity of printing, type size, clarity of directions, adequate work space, etc
criterion validity
relationship between scores from one instrument and scores obtained from another instrument or measure
compares test score and an outcome
example: SAT score and GPA
Types: Predictive and Concurrent
construct validity
how well the scale measures the construct it was designed to measure (test is measuring what it is supposed to measure)
example: math test tests reading ability instead of math skills
parallel-forms reliability
used to assess the consistency of the results of two tests constructed in the same way from the same content domain
stability (test-retest reliability)
used to assess the consistency of a measure from one time to another on same test
measure of steadiness or sameness of scores over time
dependability (qual)
goes with reliability in quantitative
if research repeated you would get the same thing
the stability or consistency of the inquiry processes used over time
Careless mistakes in collecting the data?
keep an audit to share for your research
based on assumption of repeatability (but not possible in qual… only qual)
confirmability (qual)
goes with validity in quantitative
degree to which the results could be confirmed or corroborated by others
keep data audit that shows data collection and analysis procedures and makes judgments about the potential for bias or distortion
sampling unit
element or set of elements considered for selection in some stage of sampling
unit of replication
??? unit (usually class or student) where study is replicated
unit of analysis
the level from which you collect and analyze the data (students, teachers, parents, organizations, etc) function of the research questions or hyp you seek to answer or test
content analysis (both qual and quan)
word frequencies, space measurements, etc
Quan= perform as methodology or used to supplement data in anaylsis
Qual= used to gather numerical data. Content analysis used as way to analyze data
validity of instrument
does the instrument measure what it claims to measure?
interpretation and meaning of the scores derived from the instrument
reliability of instrument
consistency in respondents’ responses
How to help: Have more than one interviewer interview, repetitive questions, and repeat interview or questions after some time
secondary data analysis
use of data that was collected by someone else for some other purpose
replication studies
repeating a study using the same methods but with different subjects and experimenters
replication-and-extension studies
combine results from prior studies with results from a new study specifically designed to replicate and extend the results of the prior studies
meta-analysis (for synthesizing research)
statistical technique that combines effectsize compare findings of a LARGE number of studies and attempt to identify commonalties
descriptive research
measure of status, rather than prediction
describe “what exists”
gather data that describes events and then organize, tabulate, depict, and describe data
Often uses visual aids
survey research
participants answer questions administered through interview or questionnaires longitudinal (same pop at diff times) or cross-sectional (diff pop at same time)
observational research
observe human and animal behavior
naturalistic: observe in natural enviro
laboratory: more control, less time consuming
correlation research
attempts to determine the strength and direction of the relationship between two or more variables
prediction research
if two variables correlate, use one variable to predict the other; higher the correlation, the more accurate the prediction
ex post facto (causal-comparative research)
research to determine the cause for existing differences in groups of individuals
IV already occurred and, therefore, can’t be manipulated
Ex post facto: Effects type design (Proactive)
focus on examining a presumed cause of some effect
Are groups formed bc of presumed cause?
Does group membership have an effect on DV?
Grouping is on the IV
2 to 1
Ex post facto: Cause type design (Retroactive)
Try to identify possible cause of group membership
What are the possible causes that led to group membership?
Grouping on DV
1 to 2
experimental research
an attempt to maintain control over all factors that may affect the result of an experiment then tries to predict what will occur
Control, manipulation, and observation and measurement
pre-experimental research
(weak) lack threats to internal validity
true experimental research
control major threats to internal validity
randomly assigned
highly recommended for education
quasi-experimental research
use existing groups equivalent groups no random assignment (susceptible to internal and external validity) CANNOT intervene or change groups commonly used in education
ethical principles in human subjects research (6)
- Protect participants
- Ensure confidentiality and anonymity
- Deception?
- Research with children- informed consent by parents, do not pretend to be diagnostic or counselors, don’t coerce children to participate
- Informed consent
- Regulation of research (HHS dept and FIT Institutional Review Board)
Descriptive and Vote Count Methods for summarizing research
Descriptive= quantification (often frequency analysis) of body of research. Purpose- what extent does existing literature support or reveal interpretable pattern. Elaborate lit review
Vote counting= used for drawing qualitative inferences about a focal relationship. summarize results of studies with frequencies or votes. DOES NOT analyze effect size, uses outcomes of tests of hypotheses reported in individual studies
