Selecting, Administering, Scoring, and Interpreting Assessment Results 2

Question 1

What is the 5th and the last step in Selecting assessment instruments and strategies?

Answer 1

5-Evaluate and select an assessment instrument or strategy

Question 2

When selecting which assessment instruments and strategies to use, counselors must?

Answer 2

• evaluate instruments on the basis of several factors (e.g., purpose, test scores, reliability, validity).
• read published test reviews as a primary evaluation before reading the test manual. The test manual is a primary source evaluative information, and published test reviews are the primary evaluation.

Question 3

In order to accurately evaluate an assessment instrument, the counselor should ask?

Answer 3

o What is the purpose of the instrument?
o Who is the intended population?
o What sample of the population was used in instrument development, validation, and norming?
o Are the results of the instrument reliable?
o Do the instrument’s results have evidence of validity?
o Does the instrument’s manual provide clear and detailed instructions about administration procedures?
o Does the manual provide sufficient information about scoring, interpreting and reporting results?
o Is the instrument biased?
o What level of competency is needed to use the instrument?
o What practical issues should be considered for this instrument?

Question 4

What the counselor should focus on and do to answer this question:
What is the purpose of the instrument? Who is the intended population?

Answer 4

Focus:

• (does it measure the intended behavior of interest?)
• (is it appropriate for the intended client?)

Do:
Look for the extent to which an instrument is appropriate for the individual(s) being assessed.

Look at the manual, and it should state the instrument’s recommended uses as well as describe the population for which it was intended.

Question 5

What the counselor should focus on and do to answer this question:
What is the makeup of the norm group? / What sample of the population was used in instrument development, validation, and norming?

Answer 5

Focus:
- is it representative of the population from which it is drawn?

Do:

• Look whether the sample of individuals (i.e., norm group) used during instrument development represents the population of potential examinees.
• Evaluate the suitability of the norm group in terms of representativeness, the year that the sample was gathered, and the size of the norm group.

Question 6

What the counselor should focus on and do to answer this question:
Are the results of the instrument reliable?

Answer 6

Focus:
- Are the results consistent and stable over time?(Reliability addresses the degree to which scores are consistent, dependable, and stable.)

Do:

• Check if there are measurement errors, which is any fluctuation in scores that results from factors irrelevant to what is being measured.
• Estimate the reliability applicable to the various sources of measurement error, suing methods like test-retest, alternate forms, internal consistency, and interrater reliability.

Question 7

What the counselor should focus on and do to answer this question:
Do the instrument’s results have evidence of validity?

Answer 7

Focus:
- does the instrument measure what it is supposed to measure?
(Validity refers to whether the claims and decisions that are made on the basis of assessment results are sound or appropriate. )

Do:
- Look for evidence of validity by
 systematically examining the content of the instrument
 considering how the instrument’s scores relate to other similar instruments
 considering the association between scores and other variables related to the construct being measured.
- Look at the manual or published research studies, as they considered sources of validity information.

Question 8

What the counselor should focus on and do to answer this question:
Does the instrument’s manual provide clear and detailed instructions about administration procedures?

Answer 8

Focus:
- (is there information about time limits, testing sequence, use of extra material?)
Do:
- Make sure that all administration specifications should fully describe in the manual, including instructions and time limits.

Question 9

What the counselor should do to answer this question:

Does the manual provide sufficient information about scoring, interpreting, and reporting results?

Answer 9

1Look if the manual:

• present information about the materials and resources available to aid in scoring the instrument, such as scoring software, mail-in scoring services, or scoring keys and templates.
• provide information about the methods used to interpret and report results.

2Check if the instrument developers provide computer-generated profiles and narrative reports based on test results.

3Determine if profiles are clear and easy to understand and if narrative reports provide accurate and comprehensive information.

Question 10

What the counselor should focus on and do to answer this question:
Is the instrument biased?

Answer 10

Focus:
- are differences in results attributed to demographic variables such as age, language, and gender or to the construct being measured?

Do:
Two things to consider when looking for bias in instrument:
- An instrument is considered biased if differences in results are attributable to demographic variables (e.g., gender, race, ethnicity, culture, age, language, geographic region, etc.) rather than to the construct being measured.
- Instrument developers are expected to exhibit sensitivity to the demographic characteristics of examinees and document appropriate steps taken to minimize bias.

Question 11

What the counselor should do to answer this:

What level of competency is needed to use the instrument?

Answer 11

Do:

• Check the level of education, training, and supervised experience needed for the instrument.
• What test user Qualifications they have

Question 12

What are the Test User Qualifications levels ? give examples

Answer 12

A-level instruments:
To purchase an A-level instrument, an indi-vidual should have a bachelor’s degree in psychology, human services, education, or related disciplines; training or certification relevant to assessment; or practical experience in the use of assessment instruments. Example: attitude and career exploration tests.

B-level instruments:
master’s degree in counseling, psychology, education or related area, or have completed specialized training or expertise in assessment. Or completed specialized training or coursework in assessment; or have licensure or certification documenting training and experience in assessment. E.g., general intelligence tests interest inventories

C-level instruments:
doctorate in psychology, education, or a related field, or licensure. E.g., intelligence tests, personality tests, projective measures

Question 13

What the counselor should consider to answer this question:

What practical issues should be considered for this instrument?

Answer 13

o When evaluating a particular assessment instrument, counselors should consider issues of practicality.
o These include the time required for administration, cost of the instrument, format, readability, administration procedures, scoring procedures, and interpretation.

Question 14

How the Time required for administering the instrument might cause a practical issues ? Give example

Answer 14

• We know that the longer the test, the more reliable the results; but the question might concern how much reliability is necessary for a particular purpose if it significantly extends the time of testing.
• E.g. can it be administered during a regular class period? Or counselling session?

Question 15

How the Ease of Administration might cause a practical issues ? Give example

Answer 15

 there are different procedures for test administration. Some require extensive training to administer and score; others do not.
 Some tests are more difficult to administer because they have a number of individually timed subtests and elaborate instructions for the test taker.
 The test users should read through the test manual and evaluate the difficulty of test administration.
 (e.g. difference between administering the WISC and the TAT)

Question 16

How the Ease of Scoring might cause a practical issues ? Give example

Answer 16

 How an instrument is scored is an important issue because it is possible for scoring to take more time than administering a test.
 hand-scored instruments are more time consuming, if the instrument provides scoring templates or answer keys to make the scoring process quicker.

Question 17

How the Ease of Interpretation might cause a practical issues ? Give example

Answer 17

 Results are not useful unless they are interpretable, and both test developers and examiners are expected to provide explanations.
 (are there computer-generated profiles and interpretations? Or sample case studies? Or detailed sections in the manual providing info on interpretation?)

Question 18

How the Format might cause a practical issues ? Give example

Answer 18

 (is the format attractively designed with a clear format, a variety of colors, and large print?)
 The test user should think of the test taker when evaluating such characteristics.
 An attractive format may provide more valid results.

Question 19

How the Readability might cause a practical issues ? Give example

Answer 19

reading level should be simple and easily understood

Question 20

How the Cost of the instrument might cause a practical issues ? Give example

Answer 20

 an important point since most school and agencies have limited budget
 Cost is an important feature when considering a particular instrument because most schools and agencies have limited budgets.
 Purchasing a commercially available instrument can become quite expensive and may require buy-ing the manual, test booklets, answer sheets, scoring templates, computer software, or ancillary materials.