RESEARCH AND GOALS OF MEASUREMENT Flashcards
REMINDERS > INTERNAL AND EXTERNAL VALIDITY
> To determine whether exclusive relationships exist between certain variables ( internal validity)
Does playing Sudoku improve working memory?
> To determine whether such relationships translate to individuals
and situations in the real world ( external validity)
Would playing Sudoku improve memory capacity in children and adults?
Causation
Determining causation is the ultimate research goal
•The first step in determining possible causation is to
choose a causal model
•A causal model (theoretically) should include
variables which are related in a sensible way
•That is, it is possible that one variable could be
causing (leading to changes) in another
CAUSAL MODELS
The most basic causal models only have two variables
Causal Variable > Consequence Variable
Independent Variable > Dependent Variable
Predictor Variable > Outcome Variable
Causal, Independent Predictor variables are thought to cause changes in another variable
Positive relationship: An increase in the values of the causal produces an increase in the values of the consequence variable
Negative relationship: An increase in the values of the causal variable produces a decrease in the values of the consequence variable
Independent Variables: Levels
Can have multiple levels when a variable is used
Can be male/female ( 2IV)
Can be playing or not playing sudoku (2IV)
Can be, playing for 1 hours, 2 or 3 ( 3IV)
COVARIATES
Variable that may also affect the dependent (usually a weaker association)
E.g., Education (in addition to Sudoku) may improve working
memory
Typically not of primary interest to the researcher
Should be taken into account (measured or controlled) when designing the research
CONFOUND
> If the covariate is a more sensible explanation then it is a confound
That is, the effect we are supposedly interested in is being confused for the effect of another variable
Gender/Sex as a Causal Variable
•Gender is a poor choice of causal variable
•While it might be convenient, it represents too many possible confounds
•We are often interested in gender/sex differences, but the explanation for those differences
is complex
•Being male or female is accompanied by complex number of factors
Socialisation
Recreational activities
Hormones (gender vs. sex)
Cultural norms
Occupations
Proximity
•Ideally, you want to specify models which include two very proximate variables
i. e., they are not separated by other potential variables
i. e., the causal variable is very closely associated
Causation vs. Association
•In research settings we are not always able to demonstrate or support a causal relationship
This is often because of practical and ethical constraints
Sometimes, we can only measure variables
E.g., rather than manipulate them
This leaves us only able to determine if they are associated but not if one likely causes the other
So we have to be careful about how we talk about causation
LECTURE ATTENDANCE»_space;> ACADEMIC PERFORMANCE
> Not good causal model > other variables which might affect performance
DEPRESSION»_space; ANXIETY
> strong evidence there is a + association
> however more variables involved
PARENTAL STATUS»_space;> EXAM PERFORMANCE
>Not good model
>Other confounding variables could exist
Models are never “proven”
•In science, we typically do not use the word “prove” when talking about models •Instead, we use terms like: “support" “find evidence for" “positively link
•Just because the data supports a model, it doesn’t mean that model is proven as it could just be a random finding
Advanced Causal Models
General Aggression Model
(Anderson & Bushman, 2002)
The General Aggression Model (GAM) is a comprehensive, integrative, framework for understanding aggression. It considers the role of social, cognitive, personality, developmental, and biological factors on aggression.
Uses multiple dependent and independent variables
Causation needs Explanation
- Psychology is about explaining phenomena
- Not just describing it
- Look for explanations not just descriptions
Description: Women who read more fashion magazines have more eating disorder symptoms
Explanation: Frequent exposure to images of women who possess idealistic physical characteristics communicates a normalized body image which does not represent most women.
Women who adopt this normalised image may then engage in compensatory behaviours, such as
restricted eating and purging, in an attempt to narrow the perceived discrepancy between their
own physical appearance and that of an ideal body image.
- An explanation can be the difference between psychologicaland non psychological research
- When reading papers, look for explanations
It can be the difference between a good paper and a weak paper
Sometimes the quality of the journal can help
Another helpful tip is to look at the title of the journal or the aims and scope
CAUSAL MECHANISM
•When you write an introduction for an
assignment, you should include an explanation
> It must clear why how one variable is thought to
affect the other
> E.g. what are the underlying processes or pathways
that lead to the outcome you are expecting?
•This explanation is referred to as the causal
mechanism
Sometimes there are multiple causal mechanisms
Focus on the simplest or easiest to explain
You want to be succinct and clear
You will need to cite other research to support any causal mechanisms you propose in your assignments
There is almost always some evidence available to help you explain the association between two variables
DATA
Data can be classified in a number of ways:
1.By the TYPE of instrument used to collect it
>Different instruments will yield different data depending on the scales (units) they employ (i.e., volume vs.
2.The SIZE of the set of data collected
E.g., single case, multiple case & metadata
3. STAGE of the process
Raw or transformed
Standardised
Summed
Averaged
Percentages
Frequencies
DATA INSTRUMENTS
•Questionnaires
SURVEY
>mixture of questions
>Often unrelated (gender, age, ethnicity)
INVENTORY
>questions capturing a similar concept
>usually published
>good measurement properties
•Tests APTITUDE (e.g., WAIS, WISC) problem solving perceptual reasoning working memory ACADEMIC/SKILLS PERFORMANCE >assessment of prior learning >general knowledge
DATA SOURCES
•Archives
e.g., education, criminal, and medical records
•Experimental tasks
a.Published measures
Iowa Gambling Task ( Bechara et al., 1994)
Wisconsin Card Sorting Task (Milner, 1963)
Implicit Association Task (Greenwald et al., 1998)
b.Novel measures
Can vary depending on researcher requirements
c.•Physiologocal Measures
Physical indicators of internal/external states
heart rate/blood flow
skin conductance
salivation
EEG, MRI, fMRI
body temperature
perspiration
VARIABLES IN DATA
A property of something that can take different values (i.e., it can vary)
•Temperature is a property of the environment
Can take different values on different scales (Fahrenheit, Celsius)
•Height is a property of trees
Can take different values on different scales (metres, feet)
•People have properties
>Easily observable
Colour
Size
Sex
> Not so easily observable
Intelligence
Personality
Mood
Variables need to be operationalized so we can measure them
Example: Stress
Emotions and thoughts captured using an inventory
Nervous arousal captured by monitoring heart rate and skin conductance
Stress operationalized as: •Difficulty relaxing •Nervous arousal •Easily upset/agitated •Irritable/over reactive •Impatient
DASS 21 items for stress: captures subjective experience
SCALES OF MEASUREMENT
•Data can possess different properties depending on what was measured and how it was measured
•The more properties, the more flexible our options are (i.e., we can perform more manipulations to evaluate and interpret our data)
•The scales of measurement are:
>Nominal/Categorical
>Ordinal
>Interval
>Ratio
Scales of Measurement >
Nominal/Categorical
•The most limited scale of measurement
•No property of magnitude
•Cannot perform operations (subtraction, addition, etc.)
Cannot do anything but collect the data
•Examples
Sex (male, female)
Political preference (Liberal, Labor ,
Race/Ethnicity (Caucasian, Asian, African)
Nationality (Australian, Yemeni, Mauritian)
Colours (Red, Blue, Brown)
Scales of Measurement > Ordinal
•Property of order but values do not represent magnitude
•Still cannot perform operations (subtraction, addition, etc.)
•May still represent categories
•Examples
Placing in a competition (1 st , 2 nd , 3 rd
Descriptive categories
low drinker, medium drinker, high drinker
very slow, slow, normal, fast, very fast
very ugly, ugly, average, attractive, very attractive
Scales of Measurement > Interval
•Property of magnitude but not relative magnitude
•Meaningful (equal) differences between intervals
•No true zero point (can have negative values)
Zero does not represent the absence of a property
• There is not start and finish
•Can perform some operations
Subtraction, addition but not multiplication or division
•Examples (there are not very many)
Temperature > The differences are the same at equal intervals > zero does not mean that there is no longer a temperature
12 hour clock > time between intervals is the same
Scales of Measurement > Ratio (one of best scales)
•Property of magnitude (including relative magnitude)
•Meaningful (equal) differences between intervals
•True zero point (cannot have negative values)
•Zero represents the absence of a property
•Can perform many operations
Subtraction, addition, multiplication and division
The most useful scale of measurement for analysis
Most flexible scale to analyse data
•Examples Height Weight Reaction time Exam score
Discrete vs. Continuous Variables
•Discrete
No intermediate values between data values
Nominal and ordinal scales are discrete
i.e., there cannot be infinite decimals between points of the scale
example race placing, 1st,2nd,3rd > there are no values in between
•Continuous
Intermediate values between data values
Common in interval and ratio scales (but not always)
There can be infinite decimals between points of the scale
example > weight and temperature
Displaying Data
Simple but effective way to review the distribution of data
Similar or the same values are grouped together in bars or bins (bar graph)
Allows the central tendency and spread to be examined visually
Measures of Central Tendency
MEAN MEDIAN MODE
> Common for mean and median to be similar
> all attempt to summarise data and provide some sense of the middle
> if data has extreme variances, we use median instead
Measures of Variability
•Range
The difference between the smallest and largest value in the dataset
•Variance
The mean squared deviation of each data point from the mean
•Standard Deviation
The square root of the mean squared deviation of each data point from the mean
> same calculation as variance calc but you square root again to bring number back down to an understandable number
Mean and low SD > means probably an accurate study
Mean and high SD > means that not accurate due to extreme variables at end of scale
Reliability and Validity
Reliability
The consistency of the data yielded from an instrument across multiple collections
Validity
Whether the data yielded from an instrument can be interpreted as representing the intended construct (i.e., does it measure what it claims measure)
A test can be reliable without being valid (that is, it can yield consistent but invalid scores)
A test cannot be valid if it is not also reliable
Questions to ask yourself when assessing RELIABILITY AND VALIDITY
1.Does it look like a VALID TEST?
2.Do experts think it looks valid?
3.Do scores on this test correlate with scores on other tests?
4.Do people of differing abilities perform differently on this test (i.e., do children
perform poorer than adults)?
5.Can it predict performance on other tasks or skills that involve the particular thing which is being measured?
