Psychometrics Flashcards

1
Q

What is the Individual Differences Approach?

A

ID is a subclass of Differential Psychology ID seeks to understand and establish psychological dimensions that apply to everyone, yet allow for or create individual differences Attempts to discover what makes a person that person

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the assumptions of ID?

A
  • Assumes a connection between normal and abnormal expression of traits
  • Assumes intelligence and personality are fluid not static
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the 2 major conceptual focus areas of ID?

A
  • The Structural Model (the ‘what’):
    • How individuals differ, what is the structure of personality/intelligence - What are the societal, cultural, psychophysiological, experiential differences
  • The Process (Dynamics) Model (the ‘why’):
    • When, why and where people differ, the functional factors of the differences - Causes and consequences of differences
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the purpose of tests in ID?

A
  • Tests in ID are defined as “systematic applications of a few relatively simple principles which attempt to measure personal psychological attributes”
  • Allow for indirect assessment of hidden psychological attributes
  • Used to make decisions around people (treatment, education etc)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are some of the general limitations of psychological testing?

A
  • Most important: Minimisation of Error (unaccounted variance) -
  • Precision/accuracy - High specialisation: focused not exhaustive questions -
  • Administration and interpretation limitations
  • Ethical Limitations: tests are required to meet ethical standards
    • respectful treatment
    • informed consent,
    • privacy,
    • confidentiality,
    • withdrawal from testing
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are some of the defining characteristics of psychological tests?

A
  • A psychological test is a ‘sample of behaviours, attitudes, thoughts and feelings’.
    • Value is largely determined by representativeness, biasedness, error, sample size, stability of attributes, response rate
  • Sampling is obtained through standardised conditions
    • random sampling: simple, systemic, cluster
    • non-random sampling: convenience, quota
  • Involves established rules for scoring and data collection
    • Objective (standardised questionaiires, blood tests), vs Subjective (interview)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are some chief concerns regarding the interpretations of psychological test results?

A
  • Are attributes real?
    • cultural biases, administration biases, faking, framing and research biases
  • Are attributes important?
    • difference between statistical and practical importanc
  • Do tests help or hinder?
    • issue of labelling
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is a psychological measurement (and its limitations)?

A
  • The process of assigning numbers to a person such that some attributes of the person are accurately represented
  • Assumptions & Limitations
    • Personality exists and is real,
    • Personality can be represented by numbers (imperfectly),
    • Individual differences have relative stability (predictive variability)
    • cannot measure whole person,
    • assesses a single attribute
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the four basic scale types (NOIR)?

A
  • Nominal (categorical): an attribute is nominated an abitrary numerical value
    • Dichotomous: Yes[1] No [0]
    • Polytomous: Answer A[0] B[1] C[2]
  • Ordinal (categorical): attributes are rank ordered on an underlying quality. Distances between points unknown.
    • Balanced: neutral scale point in the middle
    • Unbalanced: either no neutral point or non middle neutral point
    • eg. Likert scale: 5 point agreement scale (balanced)
  • Interval (numerical): attribute scaled on even intervals (uncommon) eg temperature
  • Ratio (numerical):ratios between measured numbers = ratios between desired attributes. eg reaction time
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is scale transformation and standardisation? How and Why?

A
  • Why transform or standardise?
    • raw scores often insufficient for comparison (different scales/versions)
    • standardisation -> creation of universal indexes
    • generation of population norms (indicate population distribution, norm based interpretation
  • Common transformations
    • z-scores: standard deviations
    • t-scores: z-score transformation
    • area transformations: quartiles, percentiles
    • stanines: standard nines 1-9 point scale
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is validity and some of its related issues?

A
  • What is validity?
    • The degree to which a test measures what it claims to measure
    • The appropriateness of test-scores and their interpretations
    • The degree to which measurements are a true representation
    • Level of logical and statistical biases in tests and conclusions
  • Some issues with validity
    • Psychological constructs are abstract and hidden
    • Declaring an instrument is either valid or invalid is unjustifiable in absolute sense
    • What is meaningful and useful in psychological testing?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is content validity?

A
  • Whether scores represent the content area; scores should be whole or unbiased representation of the domain
  • issues:
    • bias, sampling bias, cluster bias, *systematic error* (accuracy bias)
    • ceiling/floor effects
    • expert judges
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is Criterion related validity?

A
  • The degree to which a test correlates with one or more parallel outcomes
    Correlation is called the validity coefficient
  • Two types:
    1. Concurrent: criterion in present i.e. long form v short form
    2. Predictive: criterion in future
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is Construct (factoral) validity?

A
  • The degree to which assessed constructs possess a sane theoretical foundation which is operationalised through measureable descriptors
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is convergent and disciminant validity?

A
  • Convergent: High level of correlation between items that make up the same constructs or related constructs
  • Divergent: Low levels of correlation between items that make up unrelated concepts
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is external and internal validity?

A
  • External: consistency across settings, samples, populations and time periods
    • ecological validity = the degree to which a score reflects reality
  • Internal: The degree of confidence on the nature of asymmetrical (causal) relations between measured constructs (controlled variables)
  • Designs
    • Naturalistic designs: good EV and low IV
    • Experimental designs: good IV and low EV
17
Q

What is the dimensional conceptualisation of validity?

A

Any validity type can be conceptually represent on two dimensions

  1. Focus of concern:
    • Outcomes - practical applications,
    • Process - conceptual interpretations
  2. Generality:
    • Internal - validity with in a given setting
    • External - validity in other settings
18
Q

What are some empirical validation techniques?

A
  • Distinct-groups approach
    • assessing validity by differentiating between levels of construct in dissimilar samples
  • Multi-method validation
    • using multiple construct indicators via multiple methods
19
Q

What is reliability?

A
  • What is reliability?
    • Degree of consistency or stability of measurement scores across time or context
    • Degree of absence of construct fluctuations unaccounted for
    • Degree of random error in the observed variability
  • Assumptions of reliability indices
    • Additivity (problem of multiplicity, in-addable)
    • Independence of item scores (responses)
    • Coding (scoring) consistency
    • Random subject assignment
    • Scale-items use same dimensionality
20
Q

What is Classical Test Theory?

A
  • Observed score (X) is the sum of a true score (T) and the error (E)
    • X = T + E
    • Expressed in terms of change (variance): σ2X = σ2T + σ2E
  • Reliability index (r), is a rough estimate of theoretical reliability
    • r = σ2T / σ2X
  • Issues
    • True scores: assumes existence of a construct, assumes stability of construct (ignores temporal instability and fluctuations)
    • The randomness of error: Error may not be random
    • Systematic error: ignores bias and validity issues
21
Q

What are common sources of measurement error?

A
  • Individuals
    • Idiosyncratic: language, mood, fatigue, memory
    • Generic: faking, acquiescence and nay-saying bias, floor/ceiling effects, random responses
  • Items
    • Content-related: lack of clarity, leading/biased questions
    • Format-related: range and bias in content domain, number/type of response categories
    • Administration-related: Learning/training, Distractive settings
22
Q

What is Internal consistency reliability?

A
  • The degree of consistency in responses to scale items measuring the same construct
  • Measured using Cronbach’s alpha: weighted average scale-item ntercorrelation
    • values between 0 and 1 (0 = only error, 1 = only true scores)
    • ideal scores 0.6 - 0.8
23
Q

What is split-half reliability?

A
  • The estimated reliability based on the correlation of 2 equal random parts of a measurement
  • Measured using the Spearman-Brown coefficient (rxy)
24
Q

What is Test-retest (temporal) reliability?

A
  • Assesses the stability of scores over time
  • Measured using Pearson correlation (rank-order variation), mean drifts
  • Issues
    • Dropouts, temporal instability of constructs, time interval for retest
  • Alternatives
    • parallel forms: similar to split half, based on 2 equivalent forms of a measurement
25
Q

What is inter-rater reliability?

A
  • The estimated reliability based on the correlation between 2 or more independant judges ratings of an item or scale
  • Measured using Cohens Kappa coefficient for 2 judges
26
Q

What is the Standard Error of Measurement (SEM)?

A
  • An index of the average degree of random error of an individuals or measurements observed score
  • Allows identification of a confidence interval
27
Q

What is Factor Analysis? What are the main types of FA?

A
  • Multivariate statistics used to find hidden dimensions from a set of measurements
    • Condenses a large number of observed attributes smaller more meaningful groups called factors
  • Major FA Types
    • Exploratory (EFA): used to identify potential structures eg PCA
    • Confirmationary (CFA): used to confirm an already hypothesised structure
28
Q

What are the characteristics of the FA dimensionality?

A
  • The factors (dimensions) are linear combinations of observed attributes
  • Must have numerical or continuous structure
  • FA does not asses causal relationships - Maximising explanatory not predictive power
29
Q

What is factorability?

A
  • Factorability is the suitability of an item to be included in factor analysis
  • Barlett’s Test of Sphericity determines whether item correlation is significantly different from matrix with no correlation (identity)
  • in practice depends on degree of numerical association between items
    • items with low (< |0.5|) and high (> |0.9|) need to be considered carefully for inclusion/exclusion
30
Q

What is a factors/component and what are their different types in FA?

A
  • A factor/component is a latent dimension composed of group of similar items
    • meaningful if items related in both qualitative (conceptual) and quantitative (numerical) sense
  • Orthogonal factors: independent dimensions
  • Oblique factors: related dimensions
31
Q

What is factor loading?

A
  • Factor loading is the correlation between an item and a factor
  • When item has a loading factor of > |0.4| belongs to factor
32
Q

What is rotation and what are the different types of rotation in FA?

A
  • A rotation is a geometric transformation of factors in order to generate a model with a simpler structure (simple structure = highly distinct groups)
  • Varimax rotation (orthogonal):
    • rotates factors to provide maximum variance
    • Rotates unrelated (orthogonal) factors
  • Direct/Oblimin/Oblique Rotation
    • rotation used for related factors
    • provides a better conceptualisation but seriously distorts the space
33
Q

How many factors should be retained?

A
  • The Kaiser Criterion : Retain any factor with eigenvalue greater than one
    • Screen Plot Rule
    • standardised variance of items explained by single factor
  • Variance Explained Rule
    • Retain all factors that can collectively explain 80-90% of variance
  • Joliffe Criterion
    • Retain all factors with eigenvalue greater than or equal to 0.7
  • Comprehensibility Rule
    • Retain all factors that are meaningful and clearly interpretable
34
Q

How can you maximise psychometric values?

A
  • Increase sample size
    • SEM is inversely proportional to square root of sample size
  • Allow for sufficient (item/participant) meaningful variability
    • Discriminability
    • Minimisation of serial effects
    • Minimise acquiescent responses
  • Conceptually and empirically valid dimensionality
    • Should make sense
  • Develop a “sane research design/methodology
    • Analysis cannot account for design errors
  • A constant process
    • Constructs redefined
    • Measurements created and refined
    • Latent structures expanded and clarified
35
Q

What are some alternatives to CTT?

A
  • Generalisability Theory
    • Focus on how well generalised
    • Attempts to map and control systematic error (adds to observed score)
  • Item Response Theory
    • Mathematically maps characteristics of measurement items against participants ability
    • Can accurately predict response patterns
36
Q

What are some applications of differential psychology?

A
  • Criminal Personality Profiling
    • remove suspects from list
    • useful with unusual crimes
    • adaptive interrogation techniques based on personality
    • identify unknown offenders
  • Psychography, psychobiography, and psychohistory
    • Freud laid foundations with ‘proscriptive guidlines’
    • Triple Booking Approach
      1. Body
      2. Ego
      3. Family/Culture