PSY598 Flashcards
Testing and Assessment (125 cards)
1
Q
Psychological Assessment
A
An objective procedure for sampling and quantifying human behaviour describes
how a person thinks, acts and feels
2
Q
Organisational Testing- Purpose
A
Diagnosis -What to do next
What to focus on – (to use resources time money personnel -wisely)
As a benchmark for a longterm project research
To help Individuals develop and change
To recruit the best possible person for the role
3
Q
Organisational Testing Methods
A
Benchmark: Quantitative standard
Best Practice: Qualitative standard
HR Audit: cultural functional and effectiveness factors
4
Q
Organisational Types of Assessment
A
Observation Questionnaires and Surveys Interviews Focus groups Tests and inventories Existing data – turnover absenteeism data safety stats etc
5
Q
Hollands Theory
A
People search for environments that will let them use their skills and abilities express their attitudes and values behaviour is determined by an interaction between personality and the characteristics of environment. TYPES: Realistic Investigative Social Conventional Enterprising Artistic
6
Q
Criticism of Educational Testing
A
Serves to perpetuate SES differences no evaluation of soft skills- teamwork, leadership, emotional IQ- are important (if not more) in determining academic success
UMAT: measures reasoning, and intrapersonal skills but does not better than school grades in predicting outcomes in medical training
7
Q
Organisational: Selection Tools
A
Application (resume/ cv) Biodata Psych Test Work Sample Interview Reference Checking
8
Q
Steps in a Clinical Interview
A
1 Referral question
2 Case History Data
3 Mental Health Examination
4 Psychological Testing (if needed)
9
Q
Mental Status Examination
A
an assessment of a patient’s level of cognitive (knowledge-related) ability appearance emotional mood speech and thought patterns at the time of evaluation used in psychiatry neurology gerontology
10
Q
Components of MSE
A
Appearance Behaviour Speech Mood Affect Thought form Thought content Perceptions Cognition Insight and Judgement
11
Q
Assessments Used in Clinical Settings
A
Intelligence: WAIS Personality: MMPI
Psychopathology: Personality Assessment Inventory/ Millon Adolescent Clinical Inventory
Depression and Anxiety: Becks Depression/ Anxiety Index State-Trait Anxiety Index
12
Q
Hallucination vs. Illusion
A
Hallucination: False sensory perception
Illusion: misinterpretation of actual stimuli
13
Q
Derealisation vs. Depersonalisation
A
Derealisation: The feeling that ones surroundings are not real
Depersonalisation: a state in which one’s thoughts and feelings seem unreal or not to belong to oneself estranged from the body
14
Q
Summative Assessment
A
purely evaluative function- (end of year exam)
15
Q
Formative Assessment
A
an assessment aimed at facilitating and evaluating learning
16
Q
Educational Aptitude Test & Why?
A
WISC WHY? Giftedness Special needs Admissions
17
Q
Clinical Interview
A
technique for collecting info about a client psychoanalytic- detailed exploration of the personal family history pyschosocial development conflict defence self interpersonal processes
18
Q
Behavioural Assessment- Methods of Analysis
A
Frequency: # of times behaviour occurs
Duration: time behaviour occurs
Interval: observation time partitioned into small intervals
19
Q
Behavioural Assessment: The Observed
A
Antecedent
Behaviour (duration frequency intensity)
Consequences (reinforcement: punishment/reward)
20
Q
Behavioural Assessment: The Observer?
A
Individual
Significant other (parent/teacher/ partner)
Objective observer
Electronic device (video recorder)
21
Q
Behavioural Assessment
A
Method of data collection (Skinner) observing covert behaviour (direct observation self-report checklists questionnaires checklists and interviews significant-other ratings)
22
Q
MMPI
A
assesses major patterns of personality and emotional disorders using empirical-keying approach 567 items
23
Q
Social Cognitive Approach to Personality
A
examines the relationship between peoples behaviour the situations in which these occur and their cognitions about them
24
Q
Multivariate (trait) Approach to Personality
A
Personality oldest approach- number of dimensions of individual differences that people have in common and that serve to specify the individual’s personality
25
Personological Approach to Personaity
Murray- personality in terms of the psychogenic needs (un/conscious) of the individual and extent to which the environment promoted/inhibited those needs
26
Interpersonal Approach to Personality
proposes that personality exists only in the interaction between people and that the study of interpersonal processes is therefore central to personality assessment
27
Psychoanalytic Approach to Personality
Freud: role on unconscious motivational processes in normal and abnormal personality functioning
28
Bases for Personality Assessment
Theoretical
Empirical
Diagnostic
29
Paradigms in Personality Assessment
```
Psychoanalytic
Interpersonal
Personological
Multivariate (trait)
Empirical
Social-cognitive
Positive psychology
```
30
Subscales of WAIS
```
FSIQ: Full Scale IQ-
Verbal Comprehension
Perceptual Reasoning
Working Memory
Processing Speed
```
31
Weschler Definition of Intelligence
the aggregate of global capacity of the individual to act purposefully to think rationally and to deal effectively with his environment.
32
Wechsler Adult Intelligence Scale (WAIS)
most widely used individually administered intellectual assessment batteries- (verbal comprehension working memory processing speed visual-spatial abilities fluid reasoning
33
Triarchic Theory of Intelligence
Sternberg
1. Componential (analytical)
2. Experiental (creative)
3. Contextual (practical)
34
Crystallised Intelligence
Accumulated knowledge and skills resulting from educational/life experience (achievement test)
35
Fluid Intelligence
More pure inherited aspects of intelligence used to solve novel problems and situations (aptitude test)
36
Explicit Theories of Intelligence
devised by psychologists and other scientists validated using scientific methods sometimes informed by implicit theories
37
Implicit Theories of Intelligence
generated by individuals and based largely on their observations and opinions of how the world works
38
Norm Referenced Test
Test that uses performance of a representative sample on the test for evaluating performance of the test taker
(percentile ranks)
39
Criterion Referenced Test
Test that uses a predetermined empirical/ absolute standard as an objective reference point
e.g. >50% pass <50% fail
40
Expectancy Tables
method of interpretation that is used when scores on one measure (the predictor; ATAR) are used to predict performance on another measure (the criterion; GPA)
Also used to establish cutoff scores (percentiles z-scores stanine)
41
Z-score
linear transformation M= 0 SD= 1 number of standard deviations one data point is from the mean
42
Sten Score
non-linear standard score with a M= 5.5 SD= 2
43
Percentiles
non-linear transformation expression of the position of a score within a distribution (divide the distribution into 100 parts)
Can be used to 'normalise' non-normal distributions of test scores
44
Non-Linear Transformation
A transformation that preserves the order but not the equivalence of distance between the raw scores (percentile) used for norm-referenced test
WHY? raw score distribution are bunched in the middle- little discrimination draws out the differences in the middle but leaves the value at the tails unchanged
45
Linear Transformation
a transformation involving + - ÷ × of a constant to all raw scores- preserves the order and equivalence of distance of the raw scores(z-score) used for norm-referenced test
46
Standard Score
where a score lies in with reference to its mean can be read as a proportion of a standard deviation
47
Question Formats
Dichotomous (yes/no; true/false)
Polychotomous (more than two options- MCQ)
Likert (uni: 1 to 5 /bipolar -5 to 5 )
Q-Sorts (cards to sort into piles- not like me to alot like me) Open-ended
Constructed response
48
Levels of Measurement
Nominal/Categorical: categorise people-(Nationality/ DSM IV)
Ordinal: rank order people- (Pay bands: A B C D)
Interval: distance is meaningful (Likert Scale)
Ratio: Meaningful 0 point (Height)
49
ICC, Fairness and IRT
Construct validity test-retest reliability internal consistency factor loading of items, item correlations, and total score and ICC should be the same across all groups- p value may differ
ANOVA should now show any group x item interactions
50
Differential Test/ Item Functioning
Test/Items may behave differently for different group of respondents
51
Item Characteristic Curve
graph of the probability of answering an item correctly with the level of ability on the construct being measured demonstrating
items difficulty
items discriminatory power
probability of correct answer by guessing
x-axis: total
score y-axis: probability of success on a specific item
IRT: assumes people with high scores have a greater ability than those with low scores
52
Item Response Theory
Functional relationship between a response to a single item and the underlying trait,
detects item/test bias- DIF/DTF
53
Classical Test Theory
Believes that individuals observed score (on the test)= true score (actual measurement) + error (things that cause things difference)
doesnt not test for biases only reliability
54
Predicted and True Values- Contingency Table
Predicted Positive Value: a+b
Predicted Negative Value: c+d
True Positive Value: a+c
True Negative Value: b+d
55
Specificity (Sp)
Extent to which test correctly identifies people who do not have the condition of interest (valid negatives: d/b+d)
56
Sensitivity
Extent to which test correctly identifies people who have the condition of interest (valid positive: a/a+c)
57
Utility
Usefullness of test
As validity of selection tool increases- the mean performance of sample increases
Low selection ratios= largest increase in performance
58
Concurrent Validity
Form of predictive validity in which the test and the criterion are administered jointly.
59
Criterion-Related Validity
Form of predictive validity in which scores predict/ relate to future performance or test score.
60
Predictive Validity
The extent to which a score on a test allows estimates to be made about scores on a criterion external to the test itself (criterion/concurrent validity)
61
Diagnostic Validity
Extent to which test correctly classifies people who do and do not have specific disorders (specificity/ sensitivity)
62
Convergent/Divergent Validity
Types of construct validity- specifies where correlations should be small and large in terms of psychological constructs (used multitrait multi method matrix)
63
Construct Validity
Extent to which scores on a test are related to the underlying construct it is measuring and correlates with appropriate variables (convergent/discriminative)
64
Incremental Validity
Extent to which knowledge of a score on a test adds to that obtained by another, pre-existing test score or psychological characteristic
65
Content Validity
degree to which the content of a test is representative of the domain it’s supposed to cover, through inspection of the items on the test.
66
Types of Internal Validity
Construct Validity (Convergent/Divergent)
Diagnostic Validity (Specificity/Sensitivity)
Predictive- (Concurrent/Criterion)
Instrumental Validity
67
External Validity
the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity) and over time (historical/temporal validity).
68
Validity
Extent to which a test measures what it purports to and to which evidence supports meaning and use of test
69
5 Steps to Validity
1. Careful theory specification
2. Development of informative hypothesis tests
3. Use of sound research design
4. Examination of the degree to which observation confirm hypotheses
5. Ongoing revision of both theory and measures
70
Test Specification
A written statement of the attribute or construct that the test constrcter is seeking to measure and test conditions under which it will be used
71
Construct/Attribute
Specific idea or concept about a psychological process/underlying trait that is hypothesised on the basis of a psychological theory and target of a psychological test
72
Empirical Approach
Constructing a psychological test that relies on how the items behave when administered (discriminate between no/high scorers) and tool for personality prediction
73
Rational-Empirical Approach
constructing a pyschological test that relies on
1) . reasoning from what is known about the construct
2) . collecting and evaluating data about how the test and the items actually behave when administered to a sample
74
Decision Theory
approach to predictive validity the likelihood of someone displaying certain characteristic or not (above/below cutting point incremental validity criterion-related validity)
75
Statistics of Regression
Predicting one variable from a score on another- incremental validity
76
Standard Error of Estimate
Standard deviation of the distribution of the differences between actual and predicted scores
77
Multitrait-Multimethod Matrix
the pattern of correlations resulting from testing all possible relationships among two or more methods of assessing two or more constructs
78
Method Variance
variability among scores arising because of the method rather than the constructs of interest
79
Measurement Error
Standard Deviation: (√ of variance)
Variance: (arithmetic mean of the 2 differences between each value and mean value)
Standard Error of Measurement: (standard deviation of error- tells us how much error is in the score)
80
Factor Analysis
Identifies clusters of related items on a test- way to increase reliability
81
Discriminability Item Analysis
Examines the relationship between item response (right/wrong) and total score on all other items- way to increase reliability
82
Item Analysis
Analyses of an item following its administration and how well item relates to construct measured- way to increase reliability
83
Increase Reliability
Increase number of items
Factor and Item analysis
Discriminability Item Analysis
84
Parallel Forms Reliability
Extent to which a person obtains a similar score on an equivalent form of the same test
85
Inter-rater Reliability
Extent to which different raters agree in their assessment of same sample of test takers
86
Internal Consistency (Cronbach's Alpha)
Examines correlation between pairs of items in every possible combination
87
Types of Reliability
Internal Consistency (Cronbachs Alpha)
Inter-rater Reliability
Test-retest Reliability
Parallel Forms Reliability
88
Reliability
The consistency stability and accuracy of a test to measure what it purports to in any given circumstance
89
Psychological Testing vs Assessment
Testing: the process of administering psychological test and obtaining and interpreting scores (IQ of a child) Assessment: more complex answer to a referral question which may include (but not limited to) psychological testing (why does my child have learning difficulties)
90
Achievement Test
A test designed to predict past learning- crystallised intelligence (end of year exam)
91
Aptitude Test
A test designed to predict a persons future learning potential-fluid intelligence (GAMSAT)
92
Objective Tests
Stimuli is less ambiguous require minimal clinical interference in scoring. Have reliability validity & normative database
93
Projective Tests
Use ambiguous stimuli to indirectly uncover unconscious needs (ink blots)
94
Generalisability Theory
ideas and procedures based on the idea that consistency of psychological tests depends on specifying the desired range of conditions over which this is to hold
95
Domain-Sampling Method
sees the test as a representative sample of the larger domain of possible items that could be included
96
Stratified Sampling
non-probability sampling method where the population is divided into strata and participants randomly selected from each stratum
97
Local Norms
Derived and developed from and for specific population groups or geographical location- local (as opposed to national norms)
98
Norms
Average or expected values unique to specified groups allow interpretation of an individual score by comparison to the score of the relevant group
99
Culture-Fair Test
Test designed to minimise distortion of scores resulting from differences in cultural backgrounds of the test-takers
100
Types of Psychological Tests
```
Projective
Objective
Aptitude
Achievement
Personality
```
101
Defining Characteristics of Psychological Tests
1. To make inferences about an individual in a specific context (sample of behaviour/ sign of disposition)
2. Objective procedures of administration and scoring (standardised)
4. Provides an objective reference point (criterion/ norm reference)
5. Must meet 'psychometric properties' criteria (reliable and valid)
102
Test Obsolence
A result of continual development of test result: loses its utility because theory or content of items is wrong/no longer appropriate
103
Developmental Norms
Norms derived on the basis of any trait, ability, skill, or other characteristic presumed to develop, deteriorate or otherwise be affected by stage of life (chronological age)
104
Convenience Sample
is one which is convenient or available for use - not necessarily the most representative of the population being studied
105
Wellbeing at Work
Work conditions (rosters)
Impacts (diet/stress)
Employee Assist Programmes
Clinical Incident Management
106
CAPA
Computer Assistant Psychological Testing
107
Accommodation
Adaptation of a test, procedure, or situation, or the substitution of one test for another to make the assessment more suitable for an assessee with exceptional needs
108
KSAO
Knowledge
Skills
Abilities
Other characteristics
109
Base Rate
AKA prevalence: Proportion of individuals in the population who show the behaviour of interest in a given psychological testing or assessment
(sum of valid positives and false negatives)
110
Bias
Factors inherent in a test which systematically prevents accurate impartial results
111
Fallacy of Bias
Egalitarian Fallacy: any mean difference between groups is evidence of test bias
Culture-Bound Fallacy: group differences are due to the culture-bound nature of items
Standardisation Fallacy: test standardised in one population is biased if used on another.
112
Sources of Error
Test construction
Test Administration
Scoring & Interpretation
113
Variance
squared differences between mean and each score
Total Variance is made up of:
True Variance: from differences assumed to be stable (consistent in retest and parallel forms)
Error Variance: irrelevant random sources (can affect reliability)
114
Variance Goals
To maximise the proportion of total variance attributed to True variance, and minimise proportion of error variance
115
Central Tendency
Rating error: rater exhibits reluctance to rate at either the positive or negative extremes- ratings are in the middle
116
Confirmatory Factor Analysis
Employed when an explicitly hypothesised factor structure is tested for its fit with an observed relationship between variables
117
Exploratory Factor Analysis
Employed to estimate and extract factors, and decide how many to retain
118
Factor Loading
An outcome of factor analysis, explains correlations between observed variables using a smaller number of factors
119
Selection Ratio
Proportion of those ested who can be allocated to the category of showing behaviour of interest (sum of valid positives and false negatives)
120
Instrumental Validity
Extent to which the instrument used is correct for that measurement
121
Content Validity Ratio
Formular to measure agreement among raters regarding how essential individual item is for inclusion
122
Criterion Contamination
```
When a criterion measure is influenced by something other than performance related to the construct
caused by:
low reliability
rater bias
cheating
```
123
Criterion Deficiency
Criterion fails to measure important aspects related to the construct
124
Test-retest Reliability
The degree to which test results are consistent over time
| greater than 6 months
125
Confidence Interval
Range/band of test scores that is likley to contain trues scores
