MIDTERM 1 Flashcards
Describe the non experimental method
Both variables are measured and determined if they correlate
Why cannot causal statements be made from the non experimental method?
- No temporal precedence
- The third variable problem: extraneous variables may be causing the relationship
Types of non experimental relationships
- Positive linear
As one variable increases, so does the other - Negative linear
As one variable increases, the other decreases - Curvlinear
The direction of relationship changes at least once - No relationship
Circle scatter plot/horizontal line
What is an experimental method
Direct manipulation and controls of variables
Can describe a causal relationship between variables
Difference between an independent and dependent variable
Independent: manipulated
Dependent: measured
What is temporal precedence?
Causal variable comes first
How is causality established?
- Temporal precedence
- Covariation
- Eliminate plausible alternative explanations
Some also say the cause needs to be:
- Necessary
- Sufficient
Define Covariation
participants show a different effect between the control and experimental condition
How are alternative explanations eliminated?
- Use an experimental control
- Use random assignment so that any extraneous variables are just as likely to affect each group
How is high internal validity met?
When only the independent variable can be the cause of the results
- Temporal precedence, covariation and eliminating alternative explanations
Difference between necessary and sufficient?
Necessary: cause must be present for the effect to occur
Sufficient: cause will always produce the effect
Experimental design steps
- obtaining equivalent groups of participants
- Introducing the independent variable
- Measuring the effects of the independent variable on the dependent variable
Types of experimental designs to obtain equivalent groups
- independent group design
- repeated measures design
- match pair design
What is an independent group design?
Randomly assigning participants to experience one of the conditions (between-subject design)
Advantages and disadvantages of independent group design
Advantages:
- Avoid order effects and demand characteristics - Can use treatments with permanent effects - Similarity to "real world" setting
Disadvantages:
- Any detected difference between conditions may be attributed to group differences - Any true differences may not be detected due to low power - Need many participants - Partipant bias and experimenter expectations
How do you create equivalent groups in independent group designs?
- Random assignment
- Use matched pairs (ex. twins, IQ etc…)
- One can never be sure if matching was effective in creating equivalent groups
What is a repeated measures design?
Assigned to participate in all levels of the independent variable (within-subjects design)
- First choice as it reduces measurement error
When is a repeated measures design not possible?
When IV is a subject variable
- ex. transgenic vs. wild type mice, male vs. female
When order effects make it impossible
- ex. effects of 1 treatment is permanent (ex. surgery)
Advantages and disadvantages of a repeated measures design
Advantages:
- Fewer participants, maximize the data collected
- Sensitive to detecting differences between IV levels
- Less variance in data attributed to error
Disadvantages:
- Order effect - Demand characteristic - Participant bias - Experimental expectations
What are order Effects
The order of presenting treatments effects the dependent variable
- Practice effect
- Fatigue effect
- Contrast effect
Demand characteristics
- Any feature of a study that might inform the participant of the purpose and consequently affecting their behavior
(may deliberately act to confirm or undermine hypothesis)
Counteract demand characteristics:
- Deception - filler items - placebo
- Ask participant what they think the hypothesis is
Participant biases
Placebo effects:
- Use placebo group
- Waitlist control condition
Adaptive procedures:
- Staircase design: trials become more and more difficult if the participants gets the questions correct, incorrect = more easy
- For cognitive tasks ie. sound levels
- Randomize the order to decrease stress on animals (ie. trails don't get more and more difficult each time, the order is randomized)
Where do experimenter expectations come from?
- Treating participants in each condition differently
- Record or interpreting data differently in different conditions
How do you avoid experimenter expectations?
- Use repeated measure design
- Automated presentation of conditions and recording of data
- Double-blind experiment
Describe the types of order effects
Practice effect:
- Performance improves because of repeated practice of a task
Fatigue effect:
- Performance worsens as the participant becomes tired/bored/distracted
Contrast effect:
- The response to the second condition is altered because the conditions are contrasted (first condition can affect the experience of the second)
How do you deal with order effects?
- Counterbalancing
- Rest period may counteract the fatigue effect - Use independent group design
What is counterbalancing?
A way to deal with order effects by changing the order of IV levels
Complete counterbalancing:
- All possible orders of presentation are included in the experiment
- N! orders
Latin square:
- Each condition appears at each ordinal position
- Each condition precedes each condition once
- N orders for even N, 2N for odd N
Partial counterbalancing:
- Random order of IV
- Reverse counterbalancing (ex. ABCCBA)
What is a matched pair design?
First select pairs of participants that score equally on some variable of interest (use pretest), then use random assignment within each pair to assign to the independent variable
What is a pretest?
Test given to groups and scores of groups are compared to ensure groups were equivalent on the critical variable
What are experimental designs that include a pretest?
- Pretest-posttest design
- Solomon four-group design
When should you add a pretest and what are the disadvantages?
Add when:
- Small sample of mortality (dropout) is high
- Select appropriate participants
Disadvantages:
- Can be time consuming and awkward to administer - Can sensitize the participant to what you will be studying (can be distinguished using deception)
What is a pretest-posttest design
Measurements taken before and after the treatment
What is a posttest-only design
No pretest given, measurements only taken after the treatment
Describe a solomon four-group design
Pretest is treated as a second independent variable
- Randomly assign half of the participants to pretest or no pretest condition, then randomly assign to independent variable
What is a operational definition?
Definition of DV + IV with techniques used to measure and manipulate
Important for replication
Differences between a response, situational, and participant variable
Response variable:
Describes responses/behaviors of individuals
Can be measured in any design
Situational variable:
Describes characteristics of the situation or environment
Can be measured in any design but can only be manipulated in an experimental design
Participant variable:
Describes the characteristics that an individual brings
ex. intelligence, sex
What are the differences between measurement error, systematic error and random error?
Measurement error:
systematic error + random error
Extent that a measure is unreliable
Systematic error:
Created by faulty equipment or bias (error is always the same amount each trial)
Decreases validity
Random error:
Errors are unpredictable and cannot be reproduced.
Decreases reliability
Describe variability and reliability in terms of an operational definition
Variability:
Does the operational definition measure the concept it’s supposed to?
Reliability:
Is the operational definition based on observable, objective behaviors?
What makes a good operational definition?
Reliability and validity are the primary criteria to make a good operational definition, but it must also be:
- Absence of bias - Cost efficient - Practical - Objective - Hight acceptance
How would a hypothesis be unfalsifiable?
- No empirical evidence is obtainable
- predictions are so vague they can hardly fail
- it is upheld even though it is regulated by data by introducing new assumptions post-hoc
- no operational definition is given in prediction
What is the difference between a hypothesis and prediction?
Hypothesis:
- Statement may or may not be true
- written in present tense
- Derived from a broader theory
Prediction:
- related to specific methodological details
- written in future tense
- derived from a general hypothesis
- includes operational definitions
What is bias in a measurement?
Average error (difference between measure and true value) over many measurements (systematic error)
Define reliability
Consistency of measurements (how close measurements are to each other)
Based on concrete observable behaviors
How is reliability increased?
Multiple observations of variable
Using careful measurement procedures
What does high reliability increase?
- Precision
- Consistence
What does high reliability decrease?
- Variation
- Random error
- uncertainty
- measurement error
What are measures of reliability?
- Interrater reliability
- Test-retest reliability
- Internal Consistency Reliability
Define interrater reliability
Consistency of observations across different raters
Define test retest reliability
Giving participants the same measurement multiple times, if many people have similar scores both times, the measure reflects a more true score
Memory of the tester or test taker can skew results, therefore alternate forms of the same test given
Define Internal Consistency Reliability
Assesses how well a set of items relate to each other
Should yield similar results in each item
Includes split-half reliability, item total, and chronbach’s alpha
Define split-half reliability
Randomly select half of items and compare scores to the other half
Define an item total
Correlate each item score with the rest of the test and total score
Allows to evaluate each item individually
Define item
Repeated attempt to measure the same concept
Define Chronbach’s alpha
How well each item relates to every other item
Define validity
How close central tendency of measurements are to actual value
Whether the operation definition measure what it is supposed to
What does high validity increase?
Accuracy
What does high validity decrease?
- Bias
- Systematic error
What is internal validity?
Degree to which all confounding variables have been controlled and causality and be inferred
What is external validity?
Extent that findings can be generalized to a greater population
How can high external validity be created?
Ensure sample represents population (use random sampling + obtain all the responses)
Define construct validity
Adequacy of the operational definition and that it actually reflects the theoretical meaning of the variable
What is face validity?
Type of construct validity
Extent to which measure appears to accurately assess the intended variable
Not sufficient to conclude that a measure has construct validity
What is content validity?
Type of construct validity
Compare content of a measure with the theoretical definition of the construct
Not sufficient to conclude that a measure has construct validity
What is predictive validity?
Type of construct validity
Use the measure to predict future behavior
Construct validity is supported when scores of a measure predict future behaviors relevant to the construct
What is concurrent validity?
Type of construct validity
Extent to which scores on a measure are related to scores from a criterion measure administered at the same time
What is convergent validity?
Type of construct validity
Extent that scores on target measure are related to scores on other measures of similar construct (measures of similar constructs should converge)
What is discriminant validity?
Type of construct validity
Scores of measure are not related to other measures that capture theoretically
different constructs.
Describe the differences between concurrent/predictive validity and convergent/divergent validity
Concurrent and Predictive Validity
Base on a gold standard, measure to agree with a criterion
ex. self report measure for drinking and blood alcohol (concurrent), SAT scores and GPA in uni (predictive)
Convergent and Divergent Validity;
Measure to compare to other measures of a construct
- ex. new IQ test to correlate with existing established IQ tests
What are quantitative variables
- Continuous
- Discontinuous
Can be differentiated using the midway test (ie if the value can be halved)
What is a continuous variable?
A real number
ex. driving speed, age of individual
What is a discontinuous variable?
Only integers
ex. number of siblings, action potential (all-or-none)
What is an interval variable?
Automatically a continuous variable, does not have a “true” 0 value
ex. temperature, intelligence
Numeric properties are literal, assume equal intervals between values
What is a ratio variable?
Depends on having a true 0, therefore, value can be doubled
ex. weight, age
0 indicates an absence of variable measure, assume equal intervals between values
Types of continuous variables
Interval and ratio variables
What are categorical variables
- Nominal
- Ordinal
What is nominal variable?
Type of categorical variable
Categories with no numeric scales and no obvious relationship
ex. color, sex
What is an ordinal variable?
Type of categorical variable
Rank order, numeric values have limited meaning
ex. degree of burns (1st, 2nd, 3rd), education level
What is a confounding variable?
Intertwined with the independent variable so you cannot determine which causes the dependent variable
What are common confounding variables
- Operational definition: poor validity
- Participant factors: gender, race, intelligence, socioeconomic status etc..
- Order effects: practice, fatigue
- Group factors: self-selection (letting people sing up for the study themselves)
How can confounding variables be limited
Random assignment
Types of frequency distributions
Bar Graph:
Separate bars for each piece of information with categorical variable on x-axis
Pie Charts:
Slices represent relative percentages
Nominal scales
Histogram:
Displays frequency distribution for continuous variables
Bars are connected
Frequency polygons:
Alternative to histograms using a line instead
Continuous variables
Shapes of frequency distributions
Symmetry:
- Symmetric: Divided into 2 halves as mirror images
- Positive Skew: Many scores with low numbers that trail off towards positive numbers
- Negative Skew: Many scores with high values that trail off towards negative numbers
Modality:
- Unimodal: 1 peak - Bimodal: 2 peaks - Uniform: No well defined mode, all values are equally likely to occur
Descriptive statistics
make precise statements that summarize the data
Central Tendency:
- Value corresponding to the center of distribution - Allows for comparison to other distributions
Variability:
Characterizes the amount of spread in a distribution of scores measured on an interval or ratio scale
Types of central tendency
- Mean
- Median
- Mode
Describe mean
Type of central tendency
Advantages:
- Commonly used and good for many data sets - Enables use of sophisticated statistical tests
Limitations:
- Poor measure of central tendency for highly skewed distributions (sensitive to outliers)
Preferred for:
- continuous data
- Symmetric distribution
Describe median
Type of central tendency
Scored that divides the group into half
Advantages:
- Robust against outliers
- Better summary of skewed data
- Can be used with ordinal data
Limitations:
- Limits use of many statistical tests
Preferred for:
- Ordinal, interval, ratio data
- Skewed distribution + outliers
Describe mode
Type of central tendency
Most frequent score
Advantages:
- Used with any data type
Limitations:
- Ignores much of the data
Preferred for
- Categorical data
- Multimodal distributions
Types of variability in descriptive stats
Standard Deviation:
Indicates how far scores tend to be from the mean
Calculate variance first
Range:
Highest - lowest score
Describe random assignment
- Each participant has an equal chance of being placed in any of the experimental groups
- Ensures that any potential confounds are just as likely to affect one group as the other
Describe the law of large numbers
As the sample size increases, sample statistics become less variable and more closely estimate the population values
Describe the differences between non probability and probability sampling
Non probability Sampling:
- Phenomena similar across the population
- ex. AP threshold, limit of STM
Don’t know the probability of any member being chosen
Probability Sampling:
- Phenomena vary across population
- ex. beliefs, values, political view, etc…
Each member of the population has an equal chance of being chosen
What are the types of non probability sampling?
Purposive Sampling:
Obtain a sample of people who meet predetermined criterion
Quota sampling:
Chose a sample that reflects numerical composition of various subgroups in population
Similar to stratified sampling but without randomness, collect data with convenience techniques
- ie. there are 40 participants, 20 will be female, 20 will be male
Convenience Sampling:
Individuals are recruited wherever you can find them
Introduces sample biases
Efficient, inexpensive
What are the types of probability sampling?
Simple Random Sampling:
Each member has an equal probability of being selected for the sample
Resulting sample is called a random sample
Stratified Random Sampling:
- Used when certain parts of the population make up a smaller percent and researcher wants to make sure the sample reflects this
Population is divided into subgroups, then use simple random sampling to selected members from each group
Cluster sampling:
- Used when the researcher does not have a list of every participant
Researcher identifies clusters of people and randomly samples whole clusters
What is a complex experimental design?
More than 1 IV level or 2 or more IVs
- Ex. multiple doses of a drug
Reasons for using multiple IV levels:
1. Detect non-linear relationship between IV + DV 2. Rule out alternative explanations (eliminate confounds)
Define confidence interval
Range of plausible values for the population value; “margin of error”
Helps account for error existing in estimate because only a sample and not the entire population is used (sampling error)
- Decreases with an increase of sample size (law of large numbers)
What are 3 types of survey research questions?
- Attitudes and beliefs
- Facts and demographics
- Behaviors
Types of surveys
Mail:
- Reach large group, many don’t respond
Internet:
- Reach large group, allows for coding and analysis of data
Questionnaires vs interviews:
- Questionnaires less costly and ensure anonymity
- Interview increases participant understanding and decreases boredom and distraction
What is some problematic question wording in surveys?
- Unfamiliar technical or imprecise terms
- Ungrammatical sentence structure
- Phrasing that overloads working memory
- Embedding questions with misleading information
- Double barrel questions: asking 2 questions at once
- Loaded question: written to lead people to respond one way. Uses emotionally charged words. (Negative wording: decreases reliability and validity)
- Yea/Nay saying: responding the same answer to all questions. Limit by asking questions that can result in both yes and no answers to the same conceptual concept
Describe a response set
Tendency to respond to all question from a particular perspective rather than provide answers that are directly related to the questions
Ex. Social desirability response set
- Individual answers in most socially acceptable way
Describe rating scales
Used in surveys
- Fixed number of responses - Fully labelled are more reliable than partially labelled (decreases measurement error)
Semantic differential scale:
Measure meaning ascribed to concepts
ie. Likert scale:
- Odd number gives a neutral point while an even one forces the participant to choose a side
- Evaluation (adjectives; good-bad...)
- Activity (slow-fast, active-passive...)
- Potency (weak-strong, large-small...)
What is a factorial design
Used in complex experimental methods with more than 1 IV with more than 1 level each
What are marginal means
Come from a factorial design and shows the main effect of IV
- For 2x2 there are 8 combinations of main effects and interactions
- Main effect of Factor A and B
- Interactions of AxB
Interpretation of factorial design tables
Interaction of A x B: Is there a difference in range of values between columns or rows? (If yes, there is an interaction)
Main effects of A and B: Are mean values from row/column different? (if they are, there is an effect)
Interpretation of factorial design graphs
Interaction of A x B: If lines are parallel, there is no interaction, if they cross, there is
Main effects of x-axis variable: Average the first y value of each separate line, and do the same to the second values to create a new line (If the new line has a slope, then there is a main effect)
Main effects of non-x-axis variable: Make a horizontal line from the two values from the same line and compare the two horizontal lines (If the horizontal lines overlap, there is no main effect)
