Chapter 5: Assessment (Overview)

Question 1

Evidence-Based Assessment

Answer 1

the use of research and theory to guide the variable assessed, the methods and measures, and the manner in which the assessment process unfolds

Question 2

Assessment-Focused Services

Answer 2

services conducted primarily to provide information on a person’s psychosocial functioning

Question 3

Intervention-Focused Assessment Services

Answer 3

assessments conducted in the context of intervention services

Question 4

Screening

Answer 4

a procedure to identify individuals who may have problems of a clinical magnitude or who may be at risk for developing such problems

Question 5

Case Formulation

Answer 5

a description of the patient that provides information on his or her life situation, current problems, and a set of hypotheses linking psychosocial factors with the patient’s clinical condition

Question 6

Prognosis

Answer 6

predictions made about the future course of a patient’s psychological functioning, based on the use of assessment data in combination with relevant empirical literature

Question 7

Base Rate

Answer 7

the frequency with which a problem or diagnosis occurs in the population

Question 8

Sensitivity

Answer 8

proportion of true positives identified by the assessment

Question 9

Specificity

Answer 9

proportion of true negatives identified by the assessment

Question 10

Standardization

Answer 10

consistency across clinicians and testing occasions in the procedure used to administer and score a test

Question 11

Internal Consistency

Answer 11

the extent to which all aspects of a test contribute in a similar way to the overall score

Question 12

Test-Retest Reliability

Answer 12

the extent to which similar results would be obtained if the person was retested at some point after the initial test

Question 13

Inter-Rater Reliability

Answer 13

the extent to which similar results would be obtained if the test was conducted by another evaluator

Question 14

Evidence of Content Validity

Answer 14

the extent to which the test samples the type of behavior that is relevant to the underlying psychological construct

Question 15

Evidence of Concurrent Validity

Answer 15

the extent to which scores on the test are correlated with scores on measures of similar constructs

Question 16

Evidence of Predictive Validity

Answer 16

the extent to which the test predicts a relevant outcome

Question 17

Evidence of Discriminant Validity

Answer 17

the extent to which the test provides a pure measure of the construct that is minimally contaminated by other psychological constructs

Question 18

Evidence of Incremental Validity

Answer 18

the extent to which a measure adds to the prediction of a criterion above what can be predicted by other sources of data

Question 19

What is assessment?

Answer 19

recall that classification requires a systematic collection of data

can take various forms, e.g., structured interviews, formal testing, observation, self-report, etc.

Question 20

In what way is psychological assessment not the only type of assessment?

Answer 20

part of day-to-day life

e.g., the typical undergraduate takes about five courses per semester x two semesters x with three exams/papers per course 30 units of assessment per year + lab assignments, etc.

work appraisals, job interviews, driving tests, auditions, and other less formal evaluative contacts are all assessments

Question 21

What types of specific questions are psychological assessments concerned with answering?

Answer 21

e.g., is this person suffering from a mental disorder?

are they eligible for special program and/or for some form of funding

are there any indications that treatment is improving their functioning?

do they pose a danger to themself or others?

Question 22

Why is there a lot at stake in assessment?

Answer 22

getting an assessment wrong can be pretty devastating, tremendous room for error

eugenic commission: sterilizing people without their consent in in-patient mental services

psychologists strive to use valid and reliable assessment procedures which have direct utility to the questions being asked

Question 23

Does all psychological assessment depend on testing?

Answer 23

no, a broad array of information is considered

clinical presentation is extremely important, and is considered in the course of the interview and mental status examination (MSE)

affect, sound, eye contact, mannerism, attire, posture

several informational appraisals of memory and concentration, apparent intellectual level, abstraction, and others

may be less emphasized in MSE if psychological testing is planned as it will provide more valid and reliable assessments of those dimensions

Question 24

What is a working formulation?

Answer 24

the psychological assessment is iterative; following the acquisition of some initial data, a working formulation is created

hypotheses are naturally generated

lead to further questions which then refine the formulation, and lead to additional hypotheses

presented along with other assessment findings and recommendations

Question 25

Why is psychological assessment expensive?

Answer 25

must balance between investing enough time in this process to arrive at an accurate diagnosis on one hand, and yet not drain time-resources or unnecessarily delay treatment

doing it systematically and scientifically is therefore of great importance

Question 26

Why should limitations of assessment methods be considered?

Answer 26

limitations (always present) of any assessment procedures must be frankly disclosed, and alternative diagnoses should be considered explicitly where more than one is reasonable

Question 27

What is the foundational knowledge that assessment is impossible without?

Answer 27

psychopathology, psychometrics, psychological testing, development, etc.

Question 28

Why should the process of arriving at conclusions as consultative?

Answer 28

if the client, or other concerned parties, say you’ve missed the mark or failed to answer the referral question, it is important to revisit the process

surprisingly, a number of psychologists focus almost exclusively on interventions and seemingly default on these skills

Question 29

What is the difference between assessment versus intervention-focused services?

Answer 29

clinical psychologists often undertake assessment primarily to inform their treatment practice

at other times, it is done with the explicit intention of establishing a diagnosis, eligibility, prognosis, or recommendations for treatment/management

must not lose sight of the fact that the assessment outcome may have far-reaching implications for the client/patient

in cases where the individual is not self-referred informed consent becomes especially crucial, e.g., risk assessment

Question 30

What is stand-alone assessment?

Answer 30

going to doctor with a specific question but not looking to get treatment; done with the intention of finding something out

screening: rule out things

diagnosis

prognosis: with or without treatment, one type of treatment is always nothing

treatment recommendations

Question 31

What is integrated assessment?

Answer 31

want to get treatment for something; done with intention of treating a condition

diagnosis and case formulation: looks the same as what was discussed earlier

prognosis
treatment planning
treatment monitoring
treatment evaluation

Question 32

What are the layers of evidence-based psychological assessment?

Answer 32

noticing –> gathering data using adequate tools –> generating hypotheses –> noticing

Question 33

What are the considerations of assessment of mandated populations?

Answer 33

always be aware that assessment questions are inherently accompanied by some sort of agenda; by default, we anticipate a clinical agenda

like being mandated to pressured by employer, partner, parent; because might not consent of their own free will

do not have to like it in order to ethically consent to assessment; nothing unethical about treating a mandated individual

examples: pursuing a giftedness diagnosis, presenting with questions about neuropsychological functioning (with a hidden agenda of suing a party in a motor vehicle accident), workers compensation claims, escaping legal responsibility, child custody issues

mature minors (16-17): take competence test to determine if they are capable of evaluating before you begin procedure there is an issue of consent or non-consent so you ask them questions to make sure they understand

Question 34

What are the considerations of contracting private assessments?

Answer 34

if contracting services privately, it is reasonable to specify that court appearances, additional documentation, or other services subsequently required will be reimbursed

the parties having responsibility for compensating the psychologist should be clearly identified and must be party to the contract

Question 35

What are intervention-focused assessment services?

Answer 35

recognize the importance of an adequate evaluation in the formulation of a treatment plan and commencement of those activities

sometimes results in a recommendation that no intervention be undertaken, or that a referral be made to a separate practitioner

in either case, the initial assessment should provide a string basis for gauging treatment gains

Question 36

What is screening?

Answer 36

serves the purpose of “ruling in” rather than “ruling out”

numerous screening measures are available for a variety of conditions to assist clinicians in determining whether or not more comprehensive assessment is warranted

these are not diagnostic tests per se.

diagnosis requires more in-depth, systematic comparison of signs and symptoms to established criteria

Question 37

What is the course of a disorder?

Answer 37

virtually every section in the DSM contains a section about the course of the disorder

a description of the manner in which disorder is likely to progress without effective intervention

hence, to provide a diagnosis is tantamount to attaching predictions regarding likely outcome

when making treatment recommendations, specific reference is made to outcome studies to provide realistic projections regarding treatment effectiveness

Question 38

What is prognosis?

Answer 38

differs from course in that it incorporates the effects of any anticipated intervention

also based on outcome studies

Question 39

What is base rate?

Answer 39

it is natural for patients to ask how rare or (conversely) common a condition is

this is a question about base rate: how many people in population have the condition, what is the prevalence of the condition

Question 40

In what way are prediction and prognosis inherently probabilistic exercises?

Answer 40

to say that a given percentage of individuals can anticipate a certain outcome is not necessarily meaningful with respect to a specific patient

important to carefully consider issues of the external validity of outcome studies

does your patient differ in any significant respects from those included in the reported search? Are there any other factors that would lead you to expect a more or less favorable outcome?

motivation, support form significant others, access to treatment services, etc.

Question 41

What is treatment planning?

Answer 41

where assessment data and evidence-based treatment literature come together

unique characteristics of each client, and their circumstances are utilized in selecting the best treatment approach

decisions regarding the frequency and number of contacts are made, decisions are made concerning treatment progress tracking, and discontinuation rules are established

Question 42

What are the three essential components of treatment planning according to Mariush (2002)?

Answer 42

problem identification
setting treatment goals
selecting treatment strategies

Question 43

Why is treatment monitoring important?

Answer 43

inherent problem when clinicans and patients rely on their subjective perceptions of treatment progress

can be colored by factors not centrally relevant to the original problem, and may therefore be highly inaccurate

particularly true when both parties have invested considerable time in the provess

some form of documenting specific goals and markers of progress is therefore necessary, and more than one may be appropriate

Question 44

What is psychological testing?

Answer 44

without doubt the most unique domain of clinical psychology in multidisciplinary settings

the psychometric properties of tests are based on careful construction and standardization: usually considers various forms of validity and reliability, which are established empirically

not simply a matter of intuitively throwing together items

rules about test use, disclosing results, end-user qualifications are so strict that they are addressed by the APA in Standards for Educational and Psychological testing

good psychological tests only available to qualified practitioners

owing to the quality of their development, good psychological tests may form a central part of any assessment, and are given a high weight by courts and other agencies responsible for critical decisions

psychological tests have to be administered according to very strict rules of standardization

must likewise be scored and interpreted using very strict guidelines: to deviate from any of these carries the potential to significantly compromise test validity

Question 45

How does the APA Standards for Educational and Psychological Testing (1999) define a psychological test?

Answer 45

an evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process

Question 46

What is standardization?

Answer 46

a process of devising administration, scoring, and interpretive rules that maximize result validity and reliability, and promote consistency across examiners

all of the available research on given test presumes standardization admin and scoring

bluntly, if you don’t stick to its rules, you aren’t actually using the test

Question 47

What is internal consistency in psychological testing?

Answer 47

how “unified” a domain the test taps

i.e., do its items correlate well with one another

Question 48

What is test-retest reliability in psychological testing?

Answer 48

stability of test scores over time

not the same as “sensitivity to change”

Question 49

What is inter-rater reliability in psychological testing?

Answer 49

similarity of test scores across examiners

all of these can be expressed as a correlation

Question 50

What is content validity in psychological testing?

Answer 50

whether or not the test content corresponds meaningfully to the construct being assessed

Question 51

What is concurrent validity in psychological testing?

Answer 51

whether or not the test correlates well with other measures purporting to tap the same construct

Question 52

What is predictive validity in psychological testing?

Answer 52

whether or not the test is useful in forecasting certain clinical outcomes

Question 53

What is discriminant validity in psychological testing?

Answer 53

the degree to which the test can avoid contamination from constructs outside the one it intends to measure

Question 54

What is incremental validity in psychological testing?

Answer 54

whether or not the test contributes anything novel, compared to other tests, regarding the construct under study

Question 55

What are subscales in psychological testing?

Answer 55

psychological tests often contain numerous subscales that reflect the various components of a broader construct

virtually every intelligence inventory produces a global score that best captures an individual’s overall functioning; however, intellectual ability is a multi-faceted construct and good measures produce more specific scores that reflect relatively narrower domains

e.g. verbal versus nonverbal skills

Question 56

What is cultural fairness in psychological testing?

Answer 56

tests are inherently biased toward individuals resembling members of the standardization sample

not merely a matter of language fluency

as obvious as this sounds, subtle forms of this problem are easily overlooked

some measures attempt to avoid language issues by utilizing non-verbal response options and instructions

this does not mean that other cultural factors don’t influence the results

Question 57

What are criterion based tests?

Answer 57

items are created and scored on the basis of requisite performance

e.g., drivers test: one must be able to demonstrate an ability to safely operate the vehicle and complete certain maneuvers under specified conditions

Question 58

What are norm based tests?

Answer 58

a (usually large) normative sample is drawn from a specified population

each individual is administered the test and their performance is scored according to predetermined criteria

individual standings can be expressed as raw scores, standard scores, T scores, percentiles, etc. provided that the distribution of scores is well understood for that population

subsequently, any individual’s performance can be compared to that of the standardization sample

Question 59

How do norm-based based tests allow for a fast description of a person’s performance?

Answer 59

may, or may not, answer specific questions about concrete capabilities

at what point is an individual score low enough that we would predict academic failure in the mainstream education program?

and what point is a person’s psychological processing speed inadequate for the purposes of safe driving?

Question 60

Where do projective test fit into assessment?

Answer 60

they don’t; their content and application rules do not lend themselves to the establishment of psychometric properties

subjects are able to respond in a relatively unstructured manner

can still look at predictive validity and correspondence between examiner interpretations, however

some frameworks have emerged attempting to fasten greater structure to some projective measures

Question 61

What is informed consent in the ethics of testing?

Answer 61

children and members of mandated populations may be under significant onus to participate in testing that is potentially not in their best interest

they may likewise have no control over the dissemination of their results

Question 62

What is understandability in the ethics of testing?

Answer 62

we may not provide clients with access to raw, uninterpreted data; this is proven to be a contentious area in court

courts have proven to be very reluctant to keep “medical records” from patients

Question 63

What is confidentiality in the ethics of testing?

Answer 63

third parties frequently request access to psychological tests, even when there is no good reason for doing so

when the clinician must comply with those requests, signed consent is obtained in advance, and we have an ethical responsibility to provide data in the context of a proper interpretation, including limitations on the certainty of any test conclusions

that notwithstanding, any information regarding threats to the safety of one’s patient or others must be disclosed to the appropriate authorities, irrespective of any expectations of confidentiality

Question 64

What is the security of test materials in the ethics of testing?

Answer 64

most psychological tests are protected by copyright; disclosing the patient’s results is not the same as providing blank copies of the test instruments

test are useless if content is already known

signing a confidentiality agreement invariably precedes purchase of any psychological tests: it is understood that unauthorized duplication or disclosure of test contents could result i legal action against the psychologist

this can create a problem in public health care settings where the individual ordering the tests, is not the owner of the tests

many test publishers now provide seperate protocols for recording client responses that do not contain the test items themselves

Question 65

What are user qualifications in the ethics of testing?

Answer 65

being a member of a self-governing profession is not to say that members can do as they wish

it is incumbent upon practitioners to restrict use of tests to those that we are competent with

research has shown that psychologists tend to depend heavily on tests they were exposed to in graduate school, despite the expectation of remaining current with newly-released tests

if the psychologist chooses to include all or part of an interpretive report generated by computer software, they do so with the understanding that they are still responsible for its accuracy