assessment_in_counseling_20151104230017 Flashcards
Biases are…
built into the instrument
We cannot ever have an instrument that’s…
culturally fair
In a non-standardized test, the bias is in…
the administrator of the test
Assessments are created with the assumption that…
all test-takers have had an equal chance to learn the material and will have an equal chance to do well
Why would someone not have an equal chance to do well on an assessment?
Educationally deprived & low educational support in families
Why is the Culture-Fair Test of Intelligence not used often (except in research)?
It doesn’t predict grades
What is on the Culture-Fair Test of Intelligence?
Mostly pictures, very few words
What items are usually over-diagnosed?
Women - personality disorders (borderline, histrionic…)Men - anti-social personality disorderSexual Minorities - panic disorderLess formal education - schizophrenia
What items are usually under-diagnosed?
African-Americans - depression
What items are usually mis-diagnosed?
Autism is often misdiagnosed as an intellectual disability in African-Americans
What are the aspects of test-sophistication (aka practice-effect)?
Faking a test (depends on level of crazy)Motivation to answer correctly/incorrectlyLanguage abilitiesInsight to self (children have a hard time)Acculturation (appropriate for culture group)
Accommodations for Disabilities
Sign languageLip readerSomeone to read for themScribeBrailleTime limit changesMore breaksComputer or paper testsAllowing food and drinks (those with medical issues like diabetes)
Anytime you change the test for a disability…
you change the test (scoring will be a little different)
What must be done with employment testing?
the test MUST accurately reflect what will be done on the job
Types of vocational assessment (besides tests):
Job SamplesOn the Job Assessment
A counselor attends closely to testing conditions.
Test administration
A counselor reviews and applies scoring procedures.
Test interpretation
A counselor presents a comprehensive picture using several assessment data sources.
Communication of findings
A client completes a satisfaction form at the end of a comprehensive session.
Outcome assessments
Type of assessment that allows information to be obtained from many people within a short period of time at relatively little cost.
Group assessments
Type of assessment that permits counsellors to adapt the test administration to the needs of the client.
Individual assessment
This type of assessment includes intelligence tests, ability tests, personality inventories, interest inventories and values inventories.
Standardized tests
This type of assessment includes rating scales, projective techniques, behavioral observations, and biographical measures.
Nonstandardized tests
This type of assessment involves informal and flexible procedure often used in individual and group counseling. Focused on increasing client self-awareness within a session. Ex: simulation exercises, projective techniques, and card sorts.
Qualitative assessment.
When raters show a tendency to generalize from one aspect of the client to all other aspects
Halo effect
The tendency to rate all people as “average” or near the middle of the rating scale
Error of central tendency
The tendency to rate the characteristics of people more favorably than they should be rated.
Leniency error
Semantic differential techniques
Requires raters to rate concepts (my job) by bipolar scales/rank-order scales.
Situational tests require
The person to perform a task in a situation that is similar to the situation for which the person is being evaluated.
These assessments use vague or ambiguous stimuli to which people must respond (e.g., inkblots)
Projective assessments
These observations refer to behaviors that can be observed and counted.
Behavioral observations
Precontemplation stage
Individuals are not especially state of their problems and gave no plans to change their behavior in the foreseeable future.
Contemplation stage
Individuals are aware of their problems but have not yet made a serious commitment to do anything about them.
Preparation stage
Individuals have begun to make small changes in their problematic behaviors, with the intention of making more complete changes within one month.
Action stage
When individuals successfully change their behavior for short periods of time
Maintenance stage
When individuals have maintained the behavioral and attitudinal changes that have occurred for 6+ months
The best general source of information about commercial tests
Mental Measurements Yearbook (MMY)
The notion that data sometimes can be affected by what the administrator expects to find.
Expectancy effect it Rosenthal effect
The notion that counsellors who seek to confirm negative stereotypes, intentionally or unintentionally, during test administration influence test performance.
Stereotype threat
A counselor and client consider various assessment methods
Test selection
The ____ was established in 1985 as a forum for counseling and education related associations to collaborate for the common good for fair, accessible, and appropriate use of tests. Goal was to improve test use through education, not to limit test access
Joint Committee on Testing Practices (JCPT)
What is reliability?
Looking to see if you get the same results from the same test from the same person during a short amount of time
Instrument assessments look for _____, while personality tests look for _____.
growth; stability
Why do we need to be careful when assessing children for personality disorders?
They are still developing their personality
What is measurement error?
The goal is to reduce the error, but there is going to be some error in your test (known as “error score”)
What are the two parts (scores) of each test?
True score & error score
(T/F) No score will be perfectly reliable or perfectly without error.
True
What are correlation coefficients (aka reliability coefficients)?
When assessing reliability, the statistical measures that determine degree of relationship b/w two factors
What is the range of values for correlation coefficients?
-1 to +1
(T/F) No relationship will be perfect (score of -1 or +1)
True
(T/F) The bigger the number, the stronger the relationship
True
(T/F) If the number is “-“, there is a negative effect; as one factor increases, the other factor decreases
True
What are the base correlation coefficients for achievement tests an personality tests?
Achievement Test: .85-.90Personality Test: .50-.60
What are the types of reliability?
Test-retestSplit-halfAlternate formInter-rater reliability
What is test-retest?
Give test on one occasion, give test again later to same group of people and correlation the scores
What is alternate form in reliability?
Measuring same construct same way, but with different questions. Give test to a group of people then give the alternate version and correlate scores
What is the best way to assess reliability?
Test-retest combined with Alternate form.
What is usually used to assess reliability?
Split-half and inter-item (cheap and simple)
What is inter-rater reliability?
Whether different raters will have same results on same assessment
What is validity?
Does the test measure what it says it’s going to measure? Is the test measuring a construct adequately?
Validity Notes
More difficult to assess than reliabilityAssessments are usually only good for one of two assessments at most
What are the different types of validity?
Validity coefficientsFace validityContent validityCriterion related validityConstruct validityTreatment validity
What are validity coefficients?
The statistical measures that determine degree of relationship b/w two factors. - Usually lower than reliability coefficients.
What is face validity?
Does it look like the test is measuring what it is supposed to measure?- Should increase motivation for test-taker to do well
What is content validity?
Is the test assessing the appropriate content for the body of study that a group of students has gone through- Applies mostly to achievement tests- A group of experts looks at a group of items to make sure they fit
What is criterion related validity?
Comparing scores with performance
What are two types of criterion related validity?
-Concurrent validity: Give a new test at the same time as an older one measuring the same thing, then correlating their scores. The results should be equal - Predictive validity: Prediction of how well a person will do with a certain construct (ex: pilot training, ASVAB)
Can we predict low base-rate behaviors (suicide, murder)?
Can be done (Suicide and homicide prediction can be done, but has false-positives because it’s a low base-rate behavior (doesn’t occur very often), so the assessments can’t be used)
What is a low base-rate behavior?
A behavior (suicide and murder) that does not occur often.
What is a false-positive?
Saying someone is a certain way when they aren’t.
What is a false-negative?
Saying someone is not a certain way when they are.
What is construct validity?
Are you really measuring what you’re trying to, or you measuring something else?
Can you have a reliable test that is not valid?
Yes
Can you have a valid test that is not reliable?
No
Reliability is _____ the concept of validity.
subsumed under
What is treatment validity?
Do these tests and their results make any difference in treatment?- If the person tested concurs and is motivated to take the test, and the results are shared: then it will aid in treatment
What is a response-set (aka response style)?
- When you’re giving an instrument, and the person responds to that instrument in a way that are not what is looked for (through distortion or deception)- We want people to be honest on answers, but some people unintentionally distort subconsciously (ex: checking all “no” at doctor checklist)