Mid-term Prep Flashcards
Experimental Research
manipulated variables
Non-experimental Research
observational recording of data
Ex Post Facto Research
(After the Fact) causal-comparative: looks to the past for causes of current observations
Causal-Comparative Research
Ex Post Facto: looks to the past for causes of current observations
TRUE experiment is indicated by…
Random Assignment in experimental studies
_________ signifies experimental conditions
use of TREATMENTS
What is required in order to draw a cause and effect relationship?
Experiment
Independent variables
manipulations done by experimenter
Dependent Variables
Outcomes or results
Hypothesis
prediction of the outcome of the study/what you think will happen
Type 1 Error
Rejecting the Null Hypothesis when the Null Hypothesis is true
Type 2 Error
Rejecting the Research Hypothesis when the Research Hypothesis in True
Alpha
p = .05 or p = .01
chances of results being purely randomly caused
Power
rejecting the Null Hypothesis and accepting the Research Hypothesis when the research hypothesis is the true hypothesis
How to Maximize Power
- design powerful study
- increase the significance level
- Increase the sample size
- Employ powerful statistics
Two ways of designing powerful study
- manualize treatment
2. use vastly different treatments for comparison
Most powerful statistics are…
Parametric statistics
Levels of statistics
- Nominal
- Ordinal
- Interval
- Ratio
What are Descriptive Statistics?
mathematical procedures for: 1. organizing 2. summarizing 3. describing characteristics of a sample of data
What are Inferential Statistics?
mathematical procedures for deciding whether a sample relationship represents a relationship that actually exists in the population
What is Effect Size?
how consistently differences in the dependent scores (outcome scores) are caused by changes in the Independent Variable (IV)
What is Coefficient of Determination?
variance in the Dependent Variable (DV) that can be accounted for by variance in the Independent Variable (IV)
Definition of THEORY
a unified explanation for discrete observations that might otherwise be viewed as unrelated or contradictory
Two Functions of Research in Relation to Theory
- To test hypothesis derived from an existing theory
2. To provide observations and conclusions on which researchers can INDUCE theory
Three ways to test hypotheses derived from an existing theory
- researchers DEDUCE hypotheses that are consistent with the theory
- IF hypothesis is confirmed, it lends support to the theory
- IF hypothesis is NOT confirmed, it calls into question the theory (or parts of it)
IF a theory is called into question, what do the theorists do?
Theorists consider reformulating the theory to account for the discrepancy
Grounded Theory is…
- INDUCTIVE Method of analysis that can lead to theories of behavior.
- Found in Qualitative Analysis
- Evolutionary
No theory of human behavior is universal therefore…. (2)
- researchers usually examine trends across groups in order to test or develop theories
- Examine individuals or subgroups who do not perform as predicted
Quantitative Planning…
- Deductive approach
2. Review Lit leads to hypothesis
Qualitative Planning…
- Inductive approach
2. Observations/interviews lead to decisions on what further data is needed
Quantitative Measures produce…
numerical data
Qualitative Measures produce…
words
In Quantitative, can you adjust instruments during the study?
NO
In Qualitative, can you adjust instruments during the study?
YES
Quantitative participants…
large sample sizes
Qualitative participants…
small sample sizes
Quantitative Sampling method…
Random
Qualitative Sampling method…
purposive
Quantitative Data analysis…
statistics summarize all participants
Qualitative Data analysis…
Cite individual reponses
Quantitative Results…
generalized to one or more populations
Qualitative Results…
limits conclusions to only individuals directly studied
Quantitative Studies strive for…
objectivity
Qualitative Studies believe…
all observational processes are inherently subjective and open to interpretation
Quantitative vs. Qualitative major difference…
Numbers vs. discussion of themes/trends in words
Q vs. Q
Nature of the Question…
Quant: numbers easily collected about it
Qual: words describe it best
Q vs. Q
Little Known about topic…
Quant: Not preferred
Qual: Preferred
Q vs. Q
Participants belong to closed culture…
Quant: not preferred
Qual: Preferred
Q vs. Q
Participants available for limited time
Quant: preferred
Qual: NOT preferred
Q vs. Q
Time and funds limited
Quant: preferred
Qual: NOT preferred
Q vs. Q
Audience requires hard numbers
Quant: preferred
Qual: NOT preferred
Q vs. Q
Which captures emotions better?
Qualitative
Q vs. Q
Which captures hard to get to populations better?
Qualitative
Two Steps of Purposive Sampling
- identify a research topic of interest
2. Seek individuals who are likely to have relevant information
Number of selection criteria use in Qual
one
Purposive sampling is…
sample picked on purpose of people who have information on the issue being researched
Number of selection criterion use in Purposive CRITERION Sampling…
more than one criterion to choose the people that end up in the sample
Convenience sample is…
NOT purposive but rather just whoever happens to be available to you
Two options for when to collect demographic information…
- at the onset of the study
2. after participants have been selected
Two reasons for collecting demographic information
- to help researcher/reader to “see’ the participants
2. to describe the sample in detail for readers of the research
Use single source sampling if…
interested in applying results to single location
Use diverse sources in sampling if…
interested in applying results to many locations
Qualitative researchers usually use smaller samples than quantitative researchers for three reasons…
- Qual data collection is more expensive
- Qual data collection is more time-consuming
- Qual researchers are less interested in testing for statistical significance
Three criterion used to determine sample size…
- As the data is collected, researcher conducts a preliminary, informal analysis, noting the major and minor themes that are emerging
- Note the point at which several additional participants fail to respond with new information that leads to the identification of additional themes
- No additional participants are sought
How do you know when saturation is reached?
When you are not getting any new information from interviews and questionnaires
Phenomenological approach is..
an examination of perceptions
Two Settings for the Semi-structured Interview
- Face-to-face
2. Tape-recorded
Two components of an Interview Protocol
- written directions for conducting the interview
2. Standard set of predetermined questions to ask of all participants
Four components of Semi-structured Interview Format.
- rapport building questions
- Broad, open-ended questions
- Increasingly specific questions
- Demographic information
What makes the semi-structured interview?
- Interviewer does not need to ask ONLY predetermined questions
When to ask non-predetermined questions in a semi-structured interview…
- If participant does not seem to understand a question
- If a response is too terse
- To probe for additional information
Semi-structured interviewer characteristics
- skilled in interviewing
2. unbiased achieved through self-disclosure
Five Strengths of Semi-structured Interview
- can establish rapport and motivate respondents
- can clarify the questions
- can read nonverbal cues (eye contact, body posture)
- can use visual aids
- rich data can be obtained
Six Weaknesses of Semi-structured Interview
- takes personal time
- costs more when covering wide geographic region
- respondents may be concerned about confidentiality
- can introduce interviewer bias
- interviewers need to be trained
- respondents can terminate at any time
Four components of Focus Group Format
- 6-12 participants
- Lead by a facilitator or moderator
- Lasts for about an hour
- Two or more focus groups are typically used in a study
Two Intents of Focus Groups
- discussion of a topic to gain insight into opinions, attitudes and experiences
- perceptual changes of the individual within a social context
The major strength of Focus Group
Social Context: Presence of others changes perception of individual
Three Roles of focus group facilitator
- describes the topic and tries to create non-threatening environments
- Uses a predetermined set of questions (questioning route) to ensure that all relevant aspects of the topic are discussed
- Probes for additional information when necessary
Three strengths of Focus Groups
- reveals the evolution of perceptions in a social context
- relatively inexpensive
- Provides fairly dependable data within a short period of time
Weakness of Focus Groups
Requires a facilitator skilled in group work
Two components of Observation Format
- direct, physical observation
2. Field Research
Ethnography is…
when the focus is on cultural issues
Three Strengths of Observation
- data obtained is of events as they normally occur
- easier to note the effects of environmental influence on specific outcomes
- Easier to observe certain groups of individuals
Six Weaknesses of Observation
- necessary for observer to be physically present
- Moods, feelings and attitudes must be guessed
- cognitive thought processes not captured
- slow, tedious and expensive
- Observer fatigue
- observers must be trained
Five Observer Biases and Solutions…
- recording errors/check sheet
- memory lapses/ check sheet
- Errors in interpreting/ participant interviews
- fatigue and boredom/ limit observation time
- Multiple observers/ establish inter observer reliability
Respondent Bias and Solutions
Bias: Behave differently because being observed
Solutions:
1. observe unobtrusively
2. observe over longer period of time and discount early observations
Four Methods of Quality Control in Data Collection
- Data Triangulation
- Methods Triangulation
- Researcher Triangulation
- Audio Tape & Transcribe
Data Triangulation
- uses multiple SOURCES for obtaining data
2. to the extent that the information obtained is similar, the data can be said to be corroborated
Methods Triangulation
- uses multiple METHODS to collect data
2. To the extent that the various methods provide similar information, the data can be said to corroborated
Researcher Triangulation
- A team of researchers with diverse backgrounds
- Each member of the team collects and analyzes data
- Reduces the possibility that the results represent only the idiosyncratic views of one individual
Audio Tape & Transcribe
- conduct oral interviews or Focus Groups
- Audio-record and transcribe
- Checking the accuracy of a transcription helps to ensure the quality of the data
Three Methods in Quality Control of Data Analysis
- Inter Observer/ Inter Coder Agreement
- Peer Review/Auditor
- Member Checking
Inter Observer/ Inter Coder Agreement
- In analyzing data, each member of the research team works independently
- Then they compare their results
- To the extent that the team members agree, the results are dependable (reliable)
- If the team members disagree then they discuss until consensus reached
Peer Review/ Auditor
- uses an outside expert to:
a. examine the process used to collect data
b. resulting data
c. conclusions drawn - Peer/auditor provides feedback to the researcher
Member Checking
- based on the idea that the participants are ‘members’ of the research team
- have participants/members review the results of the analysis
- participants/members tell researchers if results ‘ring true’
- If results do not ‘ring true’, then adjustments can be made in the description of the results
Inductive Methods of Qualitative Data Analysis
- collect data
- analyze data
- derive theories that explain the patterns noted in the response of participants
Deductive Methods of Quantitative Data Analysis
- examine what is known
- deduce theories
- test theories by collecting data
Three Steps of Grounded Theory Approach
- Open Coding
- Axial Coding
- Core Category
Four Components of Open Coding
- Verbatim transcripts examined for distinct, separate segments
- Segments identified by type and ‘coded’ with category names
- subcategories are developed when possible
- make preliminary notes on any overarching themes noted
Axial Coding
- transcripts of interviews are re-examined with the purpose of identifying relationships between categories and themes identified during open coding
- look for relationships between sorted categories
Two Core Category components
- researchers develop the main overarching category under which the other categories and subcategories belong
- Researchers also attempt to describe the process that leads to the relationships identified in the previous stage of the analysis
Grounded Theory Approach Key Element
Constant Comparison
a. throughout analysis of the data
b. constant comparison of each new element of the data with all previous elements that have been coded in order to establish and refine categories
c. The analysis focuses on similarities and differences in the data that might be accounted for by a core idea
d. constantly comparing across coders and data
Consensual Qualitative Research Approach (CQR)
Strives to have a team of researchers arrive at a consensus on the meaning of the data collected
Three Steps to CQR Approach
- Code into domains
- Develop Core Ideas within Domains
- Cross-Analysis
Code Into Domains
- Segment the data into groups according to the topics they cover
a. done independently by each member of the research team
b. team members meet to discuss and refine the domains until they reach unanimous agreement on them
c. similar to grounded theory
d. each step is done individually but then the group comes to consensus together as a team
Develop Core Ideas within Domains…
- write short summaries that reduce the original ideas of participants to fewer words
a. done independently by each member of the research team
b. team members meet to come to consensus
Cross-Analysis
- core ideas are grouped into categories based on similarities
- results in higher level of generalization
- looks for a central theme
CQR External Stability Check
- examine data in addition to the interview transcripts (eye witness accounts, physical evidence)
- usually done by an outside auditor for quality control
CQR Internal Stability Check
- Determine extent to which each category was:
a. general : all participants
b. Typical: half or more of the participants
c. Variant: less than half, but more than two of the participants - Distinctive feature of CQR
a. the number of participants to which each domain applies should be provided in research reports
Purpose of CQR Auditor
- outside expert
- reviews the work of the research team after each major step of the study
- Provide in a research report
a. credentials of the auditor
b. steps at which the auditor supplied feedback
c. whether changes were made on the basis of the auditor’s feedback and what the changes were
Differences between Quant and Qual is found in what section?
Results section
Program Evaluation Compared to Experimental Research (3)
- Applied research vs. Basic research
- Preceded by a formal needs assessment vs. rarely preceded by a formal needs assessment
- Program subject to change during course of evaluation (formative and summative) vs. treatments rarely changed during course of experiment
What type of research can be used soon to solve an immediate problem?
Applied Research
What kind of research doesn’t necessarily solve a problem?
Experimental research
What kind of research is conducted primarily to solve an existing problem?
Applied Research
What kind of research is conducted primarily to obtain knowledge of underlying theories that explain behavior?
Basic Research
What kind of research is used for formal needs assessment?
PE/Applied Research
What kind of research rarely precedes formal needs assessment?
ER/Basic Research
What kind of research determines practical needs of those who will be served by the program?
PE/Applied Research
What is the major focus of Program Evaluation/Applied Research?
to estimate extent to which program has met needs revealed in needs assessment
Two components of Program Evaluation
- Formative
2. Summative
Formative Evaluation
- information collected during the course of the program that assists in the process of modifying the program while it is being implemented
- forms the program as we go along
- what parts of the process work?
Two types of information collected in Formative Evaluation
- about the process of implementing the program
2. About progress toward ultimate goals (summative evaluation)
Summative Evaluation
does the program ultimately reach goals?
Five components of the Program Evaluation Research Report
- overview of the needs assessment that led to the development of the program
- Results of the formative evaluation
- Data on the summative evaluation
- Recommendations regarding extending the program in the future
- Recommendation for improving the program’s implementation
