Assessments Flashcards
Assessment Competencies (KSAO Model)
- Knowledge of psychometrics and theoretical information
- Skills: proficiency in different methods of assessment - Test administration, scoring and interpretation, interviewing, observation and communication of assessment findings.
- Abilities: rapport building, critical and integrative thinking, and psychological mindedness.
- Other Characteristics: attitudes and values such as respect for the person of the client and appreciation of diversity, and using precision/accuracy/attn. to detail/and good communication skills.
- Assessment is broader than testing. So the testing is the smaller part of the assessment processes and results.
- Process starts with a question? The client has a difficulty. Start with the referral question or the presenting problem.
- Then decide how to collect info on how to understand and help the client. Interviews, observations, and let’s do some formal testing as well.
o You would consider formal testing if it is going to increase access to programs/services. You may refer the parents for an assessment by a psychologist. Don’t hesitate to do this. Ie) Autism and support – IEP’s. Perhaps a history of mental illness.
o The patient might be reporting some symptoms and maybe they need a more in depth assessment.
o Formal testing always gives us data/numbers to provide us objectivity and you can have counter-transference and we get our own opinions/thoughts in the way. It gives you a bit of a distance.
It is important to get #’s to compare those #’s with the rest of the population. The norm sample allows us to compare our client with others like him/her….same age/culture/gender/socio-economic status.
People are inter-changeable from a humanistic point of view and people share commonalities. We share genders….there are categories to compare our individual clients.
Find out how your client is doing. Ie) in terms of depression you are very depressed. 95% of the entire population and this is very serious. I am not dumb/stupid like I thought I was. Telling them how they are in the big picture might help them.
We trust #’s for good reason as they mean something. There is always value behind a #. #’s are infused with values. It is the beginning of a long process of interpreting and finding meanings.
What context/purpose are you using the test?
After the test and the scores mean decisions need to be made
Decisions mean that need to be made
Actions will come out of the decisions. Ie) a learning disability will lead to making an IEP plan for a student in school. There are consequences at multiple levels
There are benefits and harms.
Suffering brings meaning
Science helps us bring meaning.
Overview of Psychometrics
Reliability: The thing a test needs to have is reliability. Your tests need to yield the same results over time. You need to be able to count on the data to make decisions. Your client may not always tell you the truth. People might not always circle the most accurate # on the test. You can take the average of all of the tests. The true score from all of the scores is X (Raw/Actual Score) = T + E (Error). This means the true score is. X is the observed score. You see it on the test. The # on the test. It is also called the raw score. T is the True Score. You can’t ever get the true score. You can’t observe it. No one has ever seen it, touched it. It’s a ghost. This is because it is idea that it is there, but you are never going to get it because there is always an amount of error. True score is always unknown. The point of measuring is to get to the closest truth as possible. Observed score is close to the true score with some error. The observed score is gravitating around the true score. They are attracted to the true score. If you are willing to take lots of results (1 million +) and find the average then that is the true score. No-one would ever do this. So we just assume that there is a true score.
The error is the same across participants
They are interchangeable units.
The error is the same across clients for CTT Theory. The error is individual.
These interchangeable units create a score.
This is a criticism of the theory – how can everyone have the same error….but this is the assumption
Classical Test Theory:
Classical test theory is a body of related psychometric theory that predicts outcomes of psychological testing such as the difficulty of items or the ability of test-takers. Generally speaking, the aim of classical test theory is to understand and improve the reliability of psychological tests.
Classical test theory may be regarded as roughly synonymous with true score theory. The term “classical” refers not only to the chronology of these models but also contrasts with the more recent psychometric theories, generally referred to collectively as item response theory, which sometimes bear the appellation “modern” as in “modern latent trait theory”.
Item Response Theory
In psychometrics, item response theory (IRT) also known as latent trait theory, strong true score theory, or modern mental test theory, is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables.
IRT: Individual scores = probability of a correct answer/endorsed item depending on the latent ability.
People who have high intelligence will get the answer right. People with low intelligence will not.
Connects the intelligence (Trait) with the outcome.
Each individual has attributes/characteristics called latent variable and test scores reflect how much of that latent ability one needs to have in order to respond correctly on a test or in order to endorse test items
Reliability
a. Test-Retest
b. Alternate Forms
c. Alpha (split-half)
d. And also
i. Inter-rater reliability
ii. Generalizability
Validity
a. Content
b. Criterion
i. Concurrent - Construct Validity
ii. Predictive - Construct Validity
Standardization
- Standardization:
a. Uniform Procedures
b. Criteria and Norms
Test on the score is able to predict your GPA. GPA is the criterion.
The test predicts your performance outside of the test.
Convergent / Discriminate Evidence
Means that your measure will correlate with other measures of the same kind. Ie) anger is related to violent behaviours. It may correlate with # of offences that are committed. You have to figure that out. Or it won’t or should not correlate with something. Ie) violence doesn’t correlate with happiness. Convergent means that we expect things to correlate with some things.
Convergent / Discriminate Evidence
measures of constructs that theoretically should be related to each other are, in fact, observed to be related to each other (that is, you should be able to show a correspondence or convergence between similar constructs)
and
measures of constructs that theoretically should not be related to each other are, in fact, observed to not be related to each other (that is, you should be able to discriminate between dissimilar constructs)
Conceptual Model for Interpreting Assessment Data
Phase 1 – Intial Collection Phase
Phase -2 – Development of References
Phase 3 - Reject inferences – Modify Inferences - Accept Inferences
Phase 4 - Develop and Integrate Hypothesis
Phase 5 - Dynamic model of the person
Phase 6 - Situational Variables
Phase – 7 Prediction of Behaviour
Nomothetic Measurement
constructs being measured are assumed to be attributes describing people in general, so measurement is context free. The framework of measurement is universal.
