Psychometrics and Assessment Skills Flashcards
What is psychological assessment?
Can include testing, but is more comprehensive than just giving someone a test. Clinical assessment typically includes;
An interviews/session where a therapist asks about current problems
Observing the person in a particular setting
A test to measure symptoms
A checklist to ensure therapist has gathered all relevant details.
Reviewing collateral info.
WHERE AND WHY
In clinical psych.?
Diagnosing clinical disorders and mental health issues
Assessing the risk and severity of symptoms
Measuring efficacy of treatment interventions
WHERE AND WHY
In neuropsych.?
Assessment of behaviours and functional changes associated with neurological illness or injury.
Measuring efficacy of treatment interventions
Measuring specific cognitive functions
WHERE AND WHY
In organisational/HR?
Assessing suitability for particular role (e.g. aptitude tests)
Measuring performance.
WHERE AND WHY
In educational/developmental?
Assessing learning abilities (giftedness, learning disorders)
Observing and managing classroom behaviour
Measuring developmental stages.
WHERE AND WHY
In Health?
Assessment of motivation/ability to change health related behaviours Chronic pain management Pre surgery (e.g. before plastic surgery is there a body image disorder)
WHERE AND WHY
In Forensic?
Assessing risk
Behavioural issues
Impact reports.
What are important things to look for when interviewing?
Bodily behaviour (slouched/fidgety)
Eye behaviour (avoiding eye contact)
Emotional tone
Physiological responses (blushing/shaking)
General appearance (hair brushed, clothing style)
- Not just about what people say, it is also what they don’t say*
What are some important things to consider about how you conduct yourself?
Facial expression
Voice-related behaviour (tone, pitch, volume, speed)
Space (physical distance between C & T)
General presentation
Important to present yourself in a way that makes them feel comfortable and that they can trust you
What is bodily behaviour unpacked?
S = face the client squarely
O = adopt and open posture
L = remember that it Is possible to lean towards the other person
E = Maintain good eye contact
R = try to be relatively relaxed or natural in these behaviours.
What is the clinical treatment process?
- Initial assessment
- Case formualtion
- Treatment planning
- Implement plan
- Monitoring progress
- prepare for closure
- closure
What does initial assessment involve?
Why has this person come to see you? gather relevant information from various sources (e.g. school, family) but this is not always necessary.
What does case formulation involve?
Makes initial diagnosis, think about what factors might be contributing to current concerns. As you go through, reevaluate as new information continues to be added.
What does treatment planning involve?
Develop an evidence based treatment plan that is concrete, with specific goals and clear steps describing how to get to those goals.
It is important to always explain to the client what is happening.
What does implementing a plan involve?
Talking through the plan and initiate it.
What does monitoring progress involve?
This helps both you and the client to be able to look back and see how things have or have not improved.
This can boost peoples confidence.
What does preparing for closure involve?
What might happen when therapy ends? How might we reduce the risk of relapse?
What does closure involve?
Provide advice on how to ensure continued wellbeing.
What should be covered in initial assessment?
Identifying info about the client. General appearance and demeanour History related to the presenting problems Past psychiatric/psychological/counselling history Educational and job history Health (medical) history Social/developmental history Family history.
It is important to begin with the least threatening topics
What is Psychometrics?
It is a branch os psychology that deals with the design, administration, and interpretation of quantitative tests for the measurement of psychological variable.
What is psychological testing?
administering some form of test (pen and paper, computerised) often as part of a broader assessment.
How can interpretation of scores occur?
A test may have a cut off (passing the cut off score may indicate pathology)
A person might be tested repeatedly to see if there is a change in scores.
An individuals test score might be compared to how a large number of other have scored to see if the person is above average, below average, or average (known as the norm referenced test)
What might be an example of a self-report test?
Personality, emotional state, aptitude testing, Pain
What might be an example of a performance test?
Memory recall, block test (spatial awareness)
What are some things to think about when testing?
Is this test appropriate for the person
Is this a good test?
Are you using the test appropriately?
Is this person administering it appropriately qualified/train?
Is this test process ethical?
Have all details been appropriately recorder?
Have the results been clearly explained to the person/other agencies?
How do you know if the test is appropriate for the person?
Age, gender, physical ability, norms and diagnostic all need to be taken into account when delivering a test.
How do we know if a test is a good test?
Tests need to be valid and reliable
Validity: is the test measuring what it should be measuring? Validity reflects the test’s ability to assess the construct it was designed to measure
Reliability: is it consistent. Reliability concerns measurement consistency or the ability of a test to produce consistent results.
How do we know if we are using a test appropriately?
Practice effects: will people learn from previous exposure to a test and will that inflate scores?
Fatigue: how long are you testing for?
Setting: is there any distractions that could impact on performance
How is a test/assessment developed?
Normally it starts with a theoretical construct
A test can then be designed that should measure the construct of interest, then we can use statistics to check that the test does measure what is is designed to measure.
We then can check how accurate the test is, and even compare it to other tests that measure the same thing.
What are the step in developing a test/questionnaire?
- Define the test.
- Select a Scaling method
- Construct the items
- Test the items
- Revise the test
- Publish the test.
DEFINING A TEST
What is a test/questionnaire?
A set of items that allows measurement of some attribute of an individual
DEFINING A TEST
What is an item?
Generic word for the various forms of content in a psychological test or questionnaire (e.g. a question)
DEFINING A TEST
What is it that you are seeking to measure?
Develop a clear idea or specification of the attribute
Use exisiting theory as a guide where possible
Write a document containing specification for the development of items.
DEFINING A TEST
How do we ensure that a test/questionnaire has not already been developed?
Check the literature and mental measurement handbook which contain the information about tests that have come out.
DEFINING A TEST
What is the Kaufman & Kaufman Model of the test definition process?
- Measure attribute/construct from a strong theoretical and research basis.
- Must have capacity to distinguish between different attributes
- Yield score that are translatable to an intervention
- Include novel tasks or questions
- Be east to administer and objective to score
- Be sensitive to the diverse needs of groups being assessed.
SELECTING A SCALING METHOD
What is nominal data?
A group you put someone in, it is categorical. Numerical value assigned will be meaningless, it is the most basic level.
SELECTING A SCALING METHOD
What is ordinal data?
It is still categorical but in ranking order e.g. 1st place, 2nd place.
The order can be told but the distance between each point cannot be told.
SELECTING A SCALING METHOD
What is interval data?
This is where continuous data can be obtained e.g. temperature in Celsius.
It is equal distance between points but there is no true 0
SELECTING A SCALING METHOD
What is ratio data?
This is a continuous measurement with a starting point of zero and differences between the points on the Cale are meaningful
SELECTING A SCALING METHOD
List the types of data from most basic to most complex.
Nominal
Ordinal
Interval
Ratio
CONSTRUCTING THE ITEMS
What is the item format?
Related to scaling method of choice with dozens of choices available.
CONSTRUCTING THE ITEMS
What are the types of item formats?
Multiple choice
True false
Force choice
Likert scale
CONSTRUCTING THE ITEMS
What is a multiple choice format?
Common for student exams. There is usually one correct answer and permits quick scoring.
The problems with this format is it is difficult to construct items and provides cues for correct responses.
CONSTRUCTING THE ITEMS
What are true/false questions?
Often used in personality test and easy to understand/answer.
The problem with it is it may reflect social desirability more than personality traits. It doesn’t permit much variability.
CONSTRUCTING THE ITEMS
What is forced-choice methodology?
Often used in personality tests and overcomes the problems of true/false questions in social desirability.
The problems with it are that people don’t always fit in either category (same problem as true/false)
CONSTRUCTING THE ITEMS
What are Likest-Type Scales?
One of the most used types of response formats, as it can better account for individual differences. It is good for assessing attitudes and perceptions and reduces desirability bias.
The problems with it is it consistently measuring the construct Q.? Are all the items appropriate and contributing to the overall interpretation of the test? It assumes strength/intensity is linear. People do not always fit into specified option. Social desirability can still occur.
How are items tested?
- Conduct a pilot study to first ensure the items are clear and can be easily answered.
- Administer questionnaire to a large participant sample
- Do some number crunching in special statistical software.
- Investigate psychometric properties for the individual items
- Determine reliability and validity for sub scales.
What are the key concepts in testing?
Standardisation and norms
Validity
Reliability
What are norms?
Refer to the performances by define groups on particular tests.
What is standardisation and norming?
The process of administering a test to a representative sample for the purpose of establishing norms is referred to standardising a test.
What are percentiles?
When we have a group of scores, we can also work out where a score fits in a distribution.
What are the 3 types of validity?
Content validity
Construct validity
Criterion-related validity.
What is content validity?
Is determine by the degree to which items on the test are representative of the domain of behaviour the test purports to measure.
What is construct validity?
The appropriateness of the inferences about the underlying construct.
A construct is a theoretical, intangible quality or trait in which individuals offer.
What is correlation?
Statistical measure to indicate the extent to which 2 variables are related.
What is factor analysis?
A statistical technique to determine the pattern of correlations or variability amongst the items; Correlated items or items that share variance form factors or dimensions.
Factors represent underlying abilities. Factors in a test can be correlated with factors in other alternative tests.
What is Criterion Validity?
is the extent to which the test predicts or is related to an outcome.
What are the types of reliability?
Internal reliability
External reliability
What is internal reliability?
Concerns the extent to which a measure is consistent with itself.
What is external reliability?
Concerns the extent to which a measure varies from one use to another use
What is test-retest reliability?
Stability over time
What is Inter-rater reliability?
The degree to which different rater’s give consistent estimates of the same behaviour
What are the 3 sources of error?
Test construction
Test administration
Test scoring and interpretation
SOURCES OF ERROR
What is test construction?
item sampling or content sampling
SOURCES OF ERROR
What is test administration?
External factors that affect the test takers attention or motivation; factors specific to the test taker; examiner related variable.
SOURCES OF ERROR
What is test scoring and interpretation?
Rigidity and rigorousness of scoring criteria; meaning of test result in relation to psychological functioning.
