Psychometrics Flashcards
1
Q
What is the Individual Differences Approach?
A
ID is a subclass of Differential Psychology ID seeks to understand and establish psychological dimensions that apply to everyone, yet allow for or create individual differences Attempts to discover what makes a person that person
2
Q
What are the assumptions of ID?
A
- Assumes a connection between normal and abnormal expression of traits
- Assumes intelligence and personality are fluid not static
3
Q
What are the 2 major conceptual focus areas of ID?
A
- The Structural Model (the ‘what’):
- How individuals differ, what is the structure of personality/intelligence - What are the societal, cultural, psychophysiological, experiential differences
- The Process (Dynamics) Model (the ‘why’):
- When, why and where people differ, the functional factors of the differences - Causes and consequences of differences
4
Q
What is the purpose of tests in ID?
A
- Tests in ID are defined as “systematic applications of a few relatively simple principles which attempt to measure personal psychological attributes”
- Allow for indirect assessment of hidden psychological attributes
- Used to make decisions around people (treatment, education etc)
5
Q
What are some of the general limitations of psychological testing?
A
- Most important: Minimisation of Error (unaccounted variance) -
- Precision/accuracy - High specialisation: focused not exhaustive questions -
- Administration and interpretation limitations
- Ethical Limitations: tests are required to meet ethical standards
- respectful treatment
- informed consent,
- privacy,
- confidentiality,
- withdrawal from testing
6
Q
What are some of the defining characteristics of psychological tests?
A
- A psychological test is a ‘sample of behaviours, attitudes, thoughts and feelings’.
- Value is largely determined by representativeness, biasedness, error, sample size, stability of attributes, response rate
- Sampling is obtained through standardised conditions
- random sampling: simple, systemic, cluster
- non-random sampling: convenience, quota
- Involves established rules for scoring and data collection
- Objective (standardised questionaiires, blood tests), vs Subjective (interview)
7
Q
What are some chief concerns regarding the interpretations of psychological test results?
A
- Are attributes real?
- cultural biases, administration biases, faking, framing and research biases
- Are attributes important?
- difference between statistical and practical importanc
- Do tests help or hinder?
- issue of labelling
8
Q
What is a psychological measurement (and its limitations)?
A
- The process of assigning numbers to a person such that some attributes of the person are accurately represented
- Assumptions & Limitations
- Personality exists and is real,
- Personality can be represented by numbers (imperfectly),
- Individual differences have relative stability (predictive variability)
- cannot measure whole person,
- assesses a single attribute
9
Q
What are the four basic scale types (NOIR)?
A
-
Nominal (categorical): an attribute is nominated an abitrary numerical value
- Dichotomous: Yes[1] No [0]
- Polytomous: Answer A[0] B[1] C[2]
-
Ordinal (categorical): attributes are rank ordered on an underlying quality. Distances between points unknown.
- Balanced: neutral scale point in the middle
- Unbalanced: either no neutral point or non middle neutral point
- eg. Likert scale: 5 point agreement scale (balanced)
- Interval (numerical): attribute scaled on even intervals (uncommon) eg temperature
- Ratio (numerical):ratios between measured numbers = ratios between desired attributes. eg reaction time
10
Q
What is scale transformation and standardisation? How and Why?
A
- Why transform or standardise?
- raw scores often insufficient for comparison (different scales/versions)
- standardisation -> creation of universal indexes
- generation of population norms (indicate population distribution, norm based interpretation
- Common transformations
- z-scores: standard deviations
- t-scores: z-score transformation
- area transformations: quartiles, percentiles
- stanines: standard nines 1-9 point scale
11
Q
What is validity and some of its related issues?
A
- What is validity?
- The degree to which a test measures what it claims to measure
- The appropriateness of test-scores and their interpretations
- The degree to which measurements are a true representation
- Level of logical and statistical biases in tests and conclusions
- Some issues with validity
- Psychological constructs are abstract and hidden
- Declaring an instrument is either valid or invalid is unjustifiable in absolute sense
- What is meaningful and useful in psychological testing?
12
Q
What is content validity?
A
- Whether scores represent the content area; scores should be whole or unbiased representation of the domain
- issues:
- bias, sampling bias, cluster bias, *systematic error* (accuracy bias)
- ceiling/floor effects
- expert judges
13
Q
What is Criterion related validity?
A
- The degree to which a test correlates with one or more parallel outcomes
Correlation is called the validity coefficient - Two types:
1. Concurrent: criterion in present i.e. long form v short form
2. Predictive: criterion in future
14
Q
What is Construct (factoral) validity?
A
- The degree to which assessed constructs possess a sane theoretical foundation which is operationalised through measureable descriptors
15
Q
What is convergent and disciminant validity?
A
- Convergent: High level of correlation between items that make up the same constructs or related constructs
- Divergent: Low levels of correlation between items that make up unrelated concepts
16
Q
What is external and internal validity?
A
- External: consistency across settings, samples, populations and time periods
- ecological validity = the degree to which a score reflects reality
- Internal: The degree of confidence on the nature of asymmetrical (causal) relations between measured constructs (controlled variables)
- Designs
- Naturalistic designs: good EV and low IV
- Experimental designs: good IV and low EV
17
Q
What is the dimensional conceptualisation of validity?
A
Any validity type can be conceptually represent on two dimensions
- Focus of concern:
- Outcomes - practical applications,
- Process - conceptual interpretations
- Generality:
- Internal - validity with in a given setting
- External - validity in other settings
18
Q
What are some empirical validation techniques?
A
- Distinct-groups approach
- assessing validity by differentiating between levels of construct in dissimilar samples
- Multi-method validation
- using multiple construct indicators via multiple methods
19
Q
What is reliability?
A
- What is reliability?
- Degree of consistency or stability of measurement scores across time or context
- Degree of absence of construct fluctuations unaccounted for
- Degree of random error in the observed variability
- Assumptions of reliability indices
- Additivity (problem of multiplicity, in-addable)
- Independence of item scores (responses)
- Coding (scoring) consistency
- Random subject assignment
- Scale-items use same dimensionality
20
Q
What is Classical Test Theory?
A
- Observed score (X) is the sum of a true score (T) and the error (E)
- X = T + E
- Expressed in terms of change (variance): σ2X = σ2T + σ2E
- Reliability index (r), is a rough estimate of theoretical reliability
- r = σ2T / σ2X
- Issues
- True scores: assumes existence of a construct, assumes stability of construct (ignores temporal instability and fluctuations)
- The randomness of error: Error may not be random
- Systematic error: ignores bias and validity issues
21
Q
What are common sources of measurement error?
A
- Individuals
- Idiosyncratic: language, mood, fatigue, memory
- Generic: faking, acquiescence and nay-saying bias, floor/ceiling effects, random responses
- Items
- Content-related: lack of clarity, leading/biased questions
- Format-related: range and bias in content domain, number/type of response categories
- Administration-related: Learning/training, Distractive settings
22
Q
What is Internal consistency reliability?
A
- The degree of consistency in responses to scale items measuring the same construct
- Measured using Cronbach’s alpha: weighted average scale-item ntercorrelation
- values between 0 and 1 (0 = only error, 1 = only true scores)
- ideal scores 0.6 - 0.8
23
Q
What is split-half reliability?
A
- The estimated reliability based on the correlation of 2 equal random parts of a measurement
- Measured using the Spearman-Brown coefficient (rxy)
24
Q
What is Test-retest (temporal) reliability?
A
- Assesses the stability of scores over time
- Measured using Pearson correlation (rank-order variation), mean drifts
- Issues
- Dropouts, temporal instability of constructs, time interval for retest
- Alternatives
- parallel forms: similar to split half, based on 2 equivalent forms of a measurement
25
Q
What is inter-rater reliability?
A
- The estimated reliability based on the correlation between 2 or more independant judges ratings of an item or scale
- Measured using Cohens Kappa coefficient for 2 judges
26
Q
What is the Standard Error of Measurement (SEM)?
A
- An index of the average degree of random error of an individuals or measurements observed score
- Allows identification of a confidence interval
27
Q
What is Factor Analysis? What are the main types of FA?
A
- Multivariate statistics used to find hidden dimensions from a set of measurements
- Condenses a large number of observed attributes smaller more meaningful groups called factors
- Major FA Types
- Exploratory (EFA): used to identify potential structures eg PCA
- Confirmationary (CFA): used to confirm an already hypothesised structure
28
Q
What are the characteristics of the FA dimensionality?
A
- The factors (dimensions) are linear combinations of observed attributes
- Must have numerical or continuous structure
- FA does not asses causal relationships - Maximising explanatory not predictive power
29
Q
What is factorability?
A
- Factorability is the suitability of an item to be included in factor analysis
- Barlett’s Test of Sphericity determines whether item correlation is significantly different from matrix with no correlation (identity)
- in practice depends on degree of numerical association between items
- items with low (< |0.5|) and high (> |0.9|) need to be considered carefully for inclusion/exclusion
30
Q
What is a factors/component and what are their different types in FA?
A
- A factor/component is a latent dimension composed of group of similar items
- meaningful if items related in both qualitative (conceptual) and quantitative (numerical) sense
- Orthogonal factors: independent dimensions
- Oblique factors: related dimensions
31
Q
What is factor loading?
A
- Factor loading is the correlation between an item and a factor
- When item has a loading factor of > |0.4| belongs to factor
32
Q
What is rotation and what are the different types of rotation in FA?
A
- A rotation is a geometric transformation of factors in order to generate a model with a simpler structure (simple structure = highly distinct groups)
- Varimax rotation (orthogonal):
- rotates factors to provide maximum variance
- Rotates unrelated (orthogonal) factors
- Direct/Oblimin/Oblique Rotation
- rotation used for related factors
- provides a better conceptualisation but seriously distorts the space
33
Q
How many factors should be retained?
A
- The Kaiser Criterion : Retain any factor with eigenvalue greater than one
- Screen Plot Rule
- standardised variance of items explained by single factor
- Variance Explained Rule
- Retain all factors that can collectively explain 80-90% of variance
- Joliffe Criterion
- Retain all factors with eigenvalue greater than or equal to 0.7
- Comprehensibility Rule
- Retain all factors that are meaningful and clearly interpretable
34
Q
How can you maximise psychometric values?
A
- Increase sample size
- SEM is inversely proportional to square root of sample size
- Allow for sufficient (item/participant) meaningful variability
- Discriminability
- Minimisation of serial effects
- Minimise acquiescent responses
- Conceptually and empirically valid dimensionality
- Should make sense
- Develop a “sane research design/methodology
- Analysis cannot account for design errors
- A constant process
- Constructs redefined
- Measurements created and refined
- Latent structures expanded and clarified
35
Q
What are some alternatives to CTT?
A
- Generalisability Theory
- Focus on how well generalised
- Attempts to map and control systematic error (adds to observed score)
- Item Response Theory
- Mathematically maps characteristics of measurement items against participants ability
- Can accurately predict response patterns
36
Q
What are some applications of differential psychology?
A
- Criminal Personality Profiling
- remove suspects from list
- useful with unusual crimes
- adaptive interrogation techniques based on personality
- identify unknown offenders
- Psychography, psychobiography, and psychohistory
- Freud laid foundations with ‘proscriptive guidlines’
- Triple Booking Approach
- Body
- Ego
- Family/Culture
