Exam 1: Non-Experimental Designs Flashcards
Straw-Man Hypothesis
• A null hypothesis that is not plausible and no one would actually believe
The null should be something that could plausibly happen!
Non-Experimental Design
- No manipulation of variables
- Examine effects of existing differences (Can be quantitative or qualitative)
• Goals
- Observe, describe, document behavior or characteristics
- Examine relations among behavior or characteristics
- Compare characteristics and how that impacts behaviors
ex: pure-tone averages are what they are, you cannot manipulate them
Types of Non-Experimental Designs
1) Descriptive
• Surveys, polls, case studies, prevalence studies
2) Relationship
• Correlational, predictive
3) Comparative
• Case-control, group comparisons
4) Causal-Comparative or Cohort
• Impact of causal factors over time
Descriptive: Surveys
1) Participants asked a series of questions
2) Who are the participants?
3) What are the questions?
• Types
• Sequence
• Valid/reliable
4) How will you give it?
• e.g., oral, written
Types of Survey Questions
- Yes/No
- Categorical
- Rating Scale
- Cumulative Response
- Open-Ended
Cumulative responses- you can choose more than one answer
Advantages of Surveys
- Easy to test and score
- Can test large numbers of people
- May be standardized (Provide basis for what is typical)
- Reliable but not necessarily valid
if you get a large enough group of people you can get a standardized score
PROMIS: assessment of these dimensions in survey form
within these are sub-measures that are really standardized
so you get a real sense of what’s normal or standard in a population
You can administer sub-components of this (i.e. fatigue) for projects
Disadvantages of Surveys
• Difficulty of crafting the questions
Leading questions
- Should concerned parents use a home program as part of their child’s speech therapy? - Implying that you should use this program- subtle implication of the value of the behavior
• Social desirability
• Ambiguity
- E.R. rate pain on scale from 0-10
• Voluntary/nonresponse bias
- Not a true sense of people and their beliefs b/c you would only get people who had time to answer their landline
Descriptive: Case Study
- Intense observation of an individual (person or setting) to understand their behavior
- Case studies can help us pose the right questions, but they may not be enough…Either need a lot of time points for 1 kid or a lot of kids
- Tensions between gathering evidence in a natural settings and having as much experimental control as possible
- May be quantitative or qualitative
- Focused or broad
- Intrinsic: understand the particular entity
- Instrumental: study a particular case that represents a larger phenomenon
- Collective: collect data on several instrumental cases
Piaget (1896-1989)
- In depth case studies of his own children
- Child reasoning, cognition, and perception of the world
- Used to conceptualize stages of development
- Children are “little scientists”
- Representation of reality is built on analyzing evidence, constructing theories about the world
- Errors accumulate, forcing a new representation
Object Permanence
Object Permanence
• Knowledge that objects exists when no longer in sight
• If you cover an object with a blanket, child believes it
disappears
• Piaget found that until 9-12 months of age, children lack object permanence
• Why children cry when mom leaves and peak-a-boo is fun
Correlations and Regressions
1) Correlation
• Two variables that vary along a dimension (not groups)
• Examine if the extent of one relates to the extent of the other
* studying the relationship between two variables - do a correlational analysis to determine the strength of the relationship
2) Regression
• Two or more variables
If a researcher is studying the relationship between several variables, do a regression analysis to determine the strength of the relationship and also which variables have the most predictive power.
** correlation does not establish causality
• Examine measures close in time or use one or more to predict the future occurrence of another
Comparative
- Groups: Examine the impact of existing differences between two or more groups on a measure of skill/characteristic
- Demographics
- Disorder vs. controls: Case-control study
- Example case-control study
- Comparing adolescents with SLI and typically developing peers on a measure of narrative language ability (Wetherell et al., 2007)
Example: Comparative vs.
Correlational
• Issue: Do different personality types differ in their use of hand gestures while communicating?
• Personality survey measured degree of extroversion/introversion
• Videotaped a conversation with each participant and counted the
number of hand gestures they made
• Option 1) Classify people as either introverted or extroverted and compare the two groups
• Option 2) Use their scores along the continuum of extrovert-to- introvert and see if this correlates with hand gestures
Advantages/Disadvantages of
Comparative/Correlation studies
- Comparative
- Assumes everyone is either one or the other category
- Problem if unequal sample sizes
- Correlational
- The degree to which people are one vs. the other
- Problems of restricted range, skewed sample (extreme participants)
Cross-sectional
Cross-sectional
Cross-sectional studies are descriptive studies (neither longitudinal nor experimental); a type of observational study that analyzes data collected from a population, or a representative subset, at a specific point in time.
- Groups may differ on factors besides age
- Faster to run
- Attrition not a factor
- Participants are naïve
- No concern that having taken part in the study at one age might change what the person would do at a later age
Longitudinal Research
An individual or group is followed for some extended period of time from about 1 year to 25 years. Longitudinal research is usually non-experimental, however, in some cases it can be experimental.
• Better control for other differences between groups; Slower to run; More participant drop out (attrition)
- Order effects
- If participants learn from being in the experiment, older kids will always do better than younger kids
Longitudinal: child development— you might want more than 5 years to look at an outcome
Causal-Comparative or
Cohort Studies
A cohort study is a type of longitudinal study that allows researches to study the emergence of disorders over time or study the long term effects of a treatment.
- researchers are investigating existing differences while obtaining information about several variables (not just the primary variable under study)
• Permit stronger inferences about possible cause/effect: e.g., risk factors
- can be retrospective or prospective
• Causal-Comparative: Obtain several measures per participant in order to control for possible alternative explanations
Cohort Study
- Longitudinal design
- Prospective (concurrent)
- Current information to predict future status
- e.g., Framingham Heart Study
Prospective- relationship between cholesterol and heart disease; people chosen to be studied before developing heart disease
• Retrospective (historical)
• How do past records explain present status • e.g., Carter et al. (2003): compared speech/
language impairment of children who had been exposed to malaria (vs. not) 2 yrs prior
Do people who get CIs feel self-conscious about them? - what non-experimental design to use?
not comparing them to another group
survey them
- When do children start using the past-tense ending correctly?
- Uses word-completion tasks, like Joey like to skip; he skips every day. Yesterday he _____.
what non-experimental design to use?
Relationship either cross-section or longitudinal
What non-experimental design to use?
Do people who see student clinicians feel more or less satisfied than those who see professional clinicians?
Comparison of 2 groups
What non-experimental design to use?
What is the relationship between amount of hearing loss and extent of hearing handicap?
correlational
Qualitative Research
- Data may include verbal statements, direct quotes, excerpts, open- ended responses, detailed descriptions of behavior, etc.
- Researchers rely on inductive reasoning to formulate a theory - Data is collected to identify trends, themes –> theory
- Natural environments/situations
- Small sample sizes, lot of time observing a fewer number of people or samples
Naturalistic Observation
- Ethology: the study of naturally occurring behavior
- Method: making detailed observations of animal or humans in their natural settings
- Konrad Lorenz
- Argued that one cannot understand animal behavior without the context of its environment
Animals and their environments
• Animals and humans have a predisposition to learn certain things
• Rats: Can be conditioned to associate ** New food - sickness ** Bright lights - electric shock Difficult to learn ** Bright lights - sickness ** New food - electric shock
• An ethologist would point out that this makes sense if you understand a rat’s environment
Ethnography
- Systematized form of participant observation- observe events and behaviors within a cultural context - spend considerable time in the culture you are observing (“in the field” or “field work”- immersion in the culture’s day-to-day activities)
- Describe patterns of behavior that characterize a culture in natural setting
- Fieldwork, behavior, artifacts, speech samples
- Hammer & Weiss (1999): Studied how low SES African-American mothers in an urban community viewed their children’s language development - Video recordings, insider point-of-view
Grounded Theory
• No presumptions; data direct the development of the theory and refine repeatedly
Goal is to develop conceptual or theoretical models to account for behaviors, events, situations, and so forth. Data often collected in an ethnographic approach (interviews, observations, etc.)
• Analysis
-Initial step is to use open coding: read through many times to summarize patterns, allow the data to generate codes for data that emerge from the data itself rather then pre-existing expectations of the researcher
- Axial coding: define relationships
- Selective coding: find core variable across all
Phenomenology
Study a phenomenon from the viewpoint of the participants- try to understand what participants believe or how they interpret a situation- the purpose is to develop a better idea of what the participants go through in their everyday life- research will read through whole transcripts in their entirety many times
• “Please describe for me a situation in
which you learned something.”
• Verbal transcript of narrative
• Holistic sense
• Identify parts (meaning changes)
• Re-wording to capture underlying meaning
• Generate a written description of the structure of the event
Conversation Analysis (CA)
• Recordings of conversational interactions
• Examine patterns of turn taking, organization, repair
widely used qualitative approach
• Measures
• Orthographic transcripts
• Codes for prosodic features, turn taking
(e.g., overlaps)
• Analysis
• Examine transcripts to identify intended
actions (e.g., offer help)
• Examine sequence that lead to that turn
• Turn “design”
• How other person responds
Scientific Rigor - Researcher Bias
- Experience, knowledge, or POV influences observations
- May not be able to eliminate, but should indicate how minimized
- Reflexivity: be self-aware of bias and minimize it
- Negative case sampling: try to collect data that doesn’t align with beliefs
- Researcher triangulation: More than one person observes/analyzes/interprets
hard to compare to reliability and validity
hard to “eliminate” any source of bias
agreement across people
Scientific Rigor:
Descriptive adequacy
Interpretive adequacy
Descriptive adequacy-
Demonstrate factual, accurate accounting of data
Strategy: Prolonged engagement
Strategy: Methods triangulation- use multiple methods
Strategy: Data triangulation
Strategy: Thick description
– highly detailed and specific description of a phenomenon
Interpretive adequacy: How well captured/conveyed the meaning of an experience
Primary Strategy: Participant feedback
The longer you work with people the more you may see them open up, or you may see more types of interactions
Clinician-Investigator Non-
Experimental Research
- Clinicians have access to SO MUCH DATA
- Quantitative and qualitative
- Consider IRB approval in order to conduct your own research or to share data with research collaborators
- Example (Shprintzen et al., 1978)
- 12 patients with similar symptoms: Submucosal left soft palate etc.
• Velo-Cardio-FacialSyndrome
Subsequent studies revealed that nearly 75% of such cases have the same chromosomal
etiology
• Improved understanding/diagnosis of common syndrome associated with clefting
Quantitative research
quantitative researchers employ deductive reasoning and begin with one or more theories about possible outcomes and they utilize large numbers of participants for a short period of time)
