Assessment and Testing Flashcards
What is appraisal?
Appraisal refers to the process of assessing or estimating attributes. It could include a survey, observations, or even clinical interviews. A test is simply an instrument which measures a given sample of behavior. Measure means that it connotes that a number or a score has been assigned to the person’s attribute or performance.
What is the study of psychometrics?
Psychometrics is the study of psychological measurement. Someone who primarily administers and interprets tests has the job title of a psychometrician. It is critical that counselors inform clients about the limitations of any tests they administer.
What is a test and what is test format?
A test is a systmatic method of measuring a sample of behavior. Test format refers to the manner in which test items are presented. Test formats can be subjective (a paradigm that relies mainly on the scorers opinion, can be impacted by personal bias). In an objective test, the rater’s judgement plays little or no part in the scoring process.
What is a free-choice test?
In a free-choice or free-response test or question, the person taking the test can respond in any manner he or she chooses. Although free choice responses can yield more information, they often take more time to score and increase subjectivity.
What is a forced-choice test?
Forced choice items can also be known as recognition items – I.e. multiple choice. On some tests, this format is used to control for the social desirability phenomenon (when people puts the answers he thinks is socially desirably). The MMPI uses forced choices to create a “lie scale” composed of human frailties we all posses, so the scale ferrets out people who try to make themselves look good vs. answering honestly.
What is the difficulty index?
TH isis the percentage of individuals who answered each item correctly. The higher the number of people who answered a question correctly, the easier it is – and vice versa. A .5 difficulty index would suggest that 50% of those tested answered the question correctly. Most theorists agree that a “good measure” will provide a wide range of items that even a poor performer can answer correctly.
What are recognition items?
Recognition items are response-types, like multiple choice, that gives the examinee two or more alternatives. A true/false test has dichotomous recognition items. If a test has 3 or more forced choices, psychometrician call it a multipoint item.
What is a normative test format?
Normative tests are used to compare someone to other people who took the same test. Can be used to assess a quality, trait, etc.
A client who takes a normative test can be compared to others who have taken the test. A normative interpretation is one in which the individual’s score is evaluated by comparing it to others who took the same test – a percentile rank is an excellent example.
What is an Ipsative test format?
Ipsative measures compare traits within the same individual - they do not compare a person to other persons who took the instrument (I.e. NOT the MMPI).
You cannot legitimately compare two or more people who have taken an ipsative test because the ipsative measure does not reveal absolute strengths. The person is measured in response to his or her own standard of behavior. (i.e. PHQ9, GAD7)
So when someone says, “Mr. Johnson’s anxiety is improving” she has given an ipsative description – and has nothing to do with comparing Mr. Johnson’s anxiety to another person’s. The ipsative approach yields a within-person analysis.
What is a speed test?
A speed test is a timed test that is really intended to be fairly easy – the difficulty is induced by the time limitations and not the difficulty of the tasks or questions themselves. A good timed speed test is purposely set up so that no one finishes it. A timed test is really a type of speed test, but a high percentage of the test takers completed it and it is usually more difficult and has a time limit (I.e. the NCE).
What is a power test?
A power test is designed to evaluate the level of mastery without a time limit. Like a speed test, this is ideally designed so that no one receives a perfect score.
How does an achievement test (also called an attainment test) differ from a personality test or interest inventory?
Achievement/attainment tests measure maximum levels of skill or present performance of skill. A personality test or interest inventory measures typical performance. Interest inventories are popular with career counselors because they measure what the client likes or dislike.
What is the Q-Sort?
This is a design often used to investigate personality traits. It involves a procedure in which an individual is given cards with statements and asked to place them in piles of “most like me” and “least like me”. Then the subject compiles them to create the “ideal self”. The ideal self can then be compared to his or her current self perception in order to assess self esteem.
What is a spiral test?
In a spiral test, the items get progressively more difficult.
What is a cyclical test?
A cyclical test has several items that are spiral (items get progressively more difficult) in nature. So in each section, the questions go from easy to more difficult.
What is a test battery?
In.a test battery, several measures are used to produce results that could be more accurate than those derived from a single source. This is considered a horizontal test. A horizontal test measures various factors (I.e. math and science) during the same testing procedure.
What are parallel forms of a test?
When a test has two versions or forms that are interchangeable, they are termed parallel forms or equivalent forms of the same test. From a statistical/psychometric standpoint, each form must have the same mean, standard error, and other statistical components.
What are the most critical factors in test selection?
Validity and reliability. Validity refers to whether the test measure what it says it measures. Reliability tells how consistently a test measures an attribute.
Which is more important - validity or reliability?
Experts nearly always consider validity to be the number one factor in the construction of a test. A test must measure what it purports to measure. Reliability is then the second most important concern. A scale, for example, needs to actually measure body weight to be valid. and to be reliable, it will need to give repeated readings that are the same if the same person keeps stepping on the scale.
What are the 5 types of validity?
Validity is a measure of whether a test really measures what it purports to measure – though note that a test that is valid for one population is not necessarily valid for another one. There are 5 basic types of validity:
- content validity - does the test examine or sample the behavior under scrutiny in a comprehensive way (I.e. an IQ test that only looks at memory can’t say it has examined the entire range of intelligence).
- Construct validity - this refers to a test’s ability to measure a theoretical concept like intelligence, self-esteem, artistic talent, etc. Any trait you cannot “directly’ measure or observe can be considered a construct.
- concurrent validity - this deals with how well the test compares to other instruments that are intended for the same purpose
- predictive validity - also known as empirical validity which reflects the test’s ability to predict future behavior based on established criteria. Sometimes concurrent validity and predictive validity are lumped together under the title of “criterion validity”
- Consequential validity - tries to ascertain the social implication of using tests
Can you have reliability without validity?
Yes. A test can be reliable but not valid – I.e. a scale that consistently reads 109lbs for someone who weighs 140lbs. So a test can have a high reliability coefficient but still have a low validity coefficient. Reliability places a ceiling on validity but validity doesn’t set limits on reliability.
What is face validity?
Face validity mere y tests you whether your test looks like it measures the intended test. I.e. does the Wechsler appear to be an IQ test?
What is incremental validity?
This has been used to describe a number of testing phenomena: it has been used to describe the process by which a test is refined and becomes more valid as contradictory items are dropped. Incremental validity also refers to a test’s ability to improve predictions when compared to existing measures that purport to facilitate selection in business or educational settings. When a test has incremental validity, it provides you with additional valid information that was not attainable via other procedures.
What is synthetic validity?
Synthetic validity is derived from the word “synthesized”.
This is technique for inferring the validity of a selection test or other predictor of job performance from a job analysis. It involves systematically analyzing a job into its elements, estimating the validity of the test or predictor in predicting performance on each of these elements, and then combining the validities for each element to form an estimate of the validity of the test or predictor for the job as a whole.
What is convergent validity?
This is a method used to assess a test’s construct/criterion validity by correlating test sores with an outside source (I.e. seeing if someone with a known phobia has test results that indicate the phobia).
What is discriminate validity?
This means that the test will not reflect unrelated variables. So if phobias are unrelated to IQ, there should not be a correlation between someone’s phobia and IQ tests. When a researcher is engaged in test validation, both convergent and discriminatory validity should be examined.
Is a valid test always reliable?
Yes, a valid test is always reliable. BUT a reliable test is not always valid.
What is test-retest reliability?
This is a method for testing reliability in which you give the same test to the same group of people two times and then correlate scores. This method tests for stability, which is the ability for a test score to remain stable or fluctuate if the client takes the test again. This method is generally only valid for traits like IQ that remain stable over time.
What is equivalent or alternate forms reliability?
This is when a single group of examinees takes parallel forms of a test and the researcher figures out a reliability coefficient based on the two sets of scores. (I.e. one group takes two tests that are designed to be equivalent and then scores are compared). Doing this well requires counterbalancing which means you split the group and one half gets Test A first and the other half gets Test B first – this controls for things like fatigue, practice, and motivation.
What is the split-half correlation method?
In this situation, the individual takes the entire test as a whole and then the test is divided in half. The correlation between the half scores yields.a reliability coefficient. But this only works if the researcher splits it using random numbers or even/odd numbers (vs. first and second half of the test) because it must account for practice and fatigue.
What is inter-rater/inter-observer reliability testing?
This is when several raters assess the same performance. This method is also called scorer reliability and is utilized with subjective tests like projectiles to ascertain whether the scoring criteria are such that two people who grade or assess the responses will get roughly the same score.
What does a reliability coefficient of 1.00 indicate?
This would indicate a perfect score and generally only occurs in physical measurement. An excellent psychological or counseling test would have a reliability coefficient of .90 which indicates that 90% of the score measured the attribute in question and 10% of the score is indicative of error. A personality test typically has a reliability coefficient around .70 (70% of the score is accurate and 30% is inaccurate).
Although .70 is generally acceptable for psychological attributes, admissions for jobs, schools, etc should be at least .80 and some experts will not settle for less than .90
What is the coefficient of determination?
This is when you have to determine the variance of one factor accounted for by another. To do that, you merely square the correlation (I.e. reliability coefficient). So if the correlation between two instances of a test to the same population (test-retest) is .70, you would square that to get .49 which would be the coefficient of determination.
What does IQ mean?
Intelligence quotient. The early ratio formula for the Binet IQ score was Mental Age/chronological age x 100. The score indicated how you compared to others in your age group. IQ testing has been the subject of heated debate.
What did Francis Galton conclude about IQ?
Sir Francis Galton of England has been recognized as one of the major pioneers in the study of individual differences. He believed that exceptional mental abilities were genetic and ran in families. He did research and concluded that intelligence was normally distributed like height and weight and that it was primarily genetic. He felt that intelligence was a single or so-called unitary factor.
Who is Charles Spearman?
In 1904, he postulated two factors that were thought to be applicable to any mental task: a general ability G and a specific ability S which were thought to be applicable to any mental task.
What is fluid vs. crystallized intelligence?
Fluid intelligence is flexible, culture-free, and adjusts to the situation. Crystallized intelligence is rigid and doesn’t change or adapt.
What were JP Guilford’s contributions to the field of understanding intelligence?
JP Guilford isolated 120 factors which added up to intelligence. Two of the dimensions – convergent and divergent thinking - are still popular terms today. Convergent thinking occurs when divergent thoughts and ideas are combined into a singular concept. Divergent thinking is the ability to generate a novel idea.
Who worked to create the first standardized IQ test?
Alfred Binet and Theodore Simon.
In 1904, the French government appointed a committee to distinguish between normal and “feeble-minded” Parisian children so that the kids with an intellectual disability could be taught separately. Binet let the committee and, by 1905, he and his coworker, Theodore Simon, had created a 30-item test with school-related items of increased difficulty. Binet used his own daughters as test subjects and is also cited as a pioneer in projective testing based on his work with inkblots.
In 1916, after testing nearly 3,000 children, Lewis M Terman of Stanford published an American version of the Binet that was translated into English – they added “Stanford” into the name and it became the Stanford-Binet IQ test.
What is the Kuder-Richardson coefficients of equivalence?
This is a method used to establish inter-item consistency, whether each item on the test is measuring the same thing as every other item. The Kuder-Richardson reliability/item consistency estimates, which are often denoted as KR-20 or KR-21, measure exactly this. Though the split-half method measures internal consistency reliability, it does not do it on a per-item basis.
What is inter-item consistency?
Inter-item consistency, also known as internal consistency or homogeneity of items means that someone wants to find out if each item on a test is measuring the same thing as every other item. Is performance on one item truly related to performance on another? This can be done by using the Kuder -Richardson reliability/item consistency estimates ( also known as KR-20 or KR-21 formulas)
What is cross-validation?
Cross-validation takes place when a researcher further examines the criterion validity (and in rare cases, the construct validity) of a test by administering the test to a new sample. This procedure is necessary to ensure that the original validity coefficient is a guard agains error factors which are likely to be present if the original sample size is small. In most cases, a cross-validation coefficient is indeed smaller than the initial validity coefficient. This phenomenon is called shrinkage.
What is a standardized measure?
A standardized measure is one in which the scoring and administration procedures are formal and well delineated. Measures that are not standardized lack procedural guidelines for scoring or administration and do not include quantitative information relating to standards of performance. The Stanford-Binet IQ test is an example of a standardized test.
IQ stands for intelligence quotient which is expressed by:
MA (Mental age)/CA (chronological age) X 100
The test is Binet’s, but the famous formula was created by German psychologist William Louis Stern. The formula produced what is known as a ratio IQ. Today a “deviation IQ” is used which compares the individual to a norm (I.e. to others in their age group). Although we still use the term IQ, the Binet today actually relies on a standard age score (SAS) with a mean of 100 and a standard deviation of 16. So IQ today is no longer really a quotient.
According to Binet’s age-related tasks, how would he define a “9 year old task”?
That would be something that 50% of 9 year olds tested could answer correctly.
What is the best test for measuring genius?
Some experts believe that the Stanford-Binet is a more accurate test for assessing extremes of intellect (high and low) while the Weschler is a better test for those who fall in the average range.
What is entropy?
This is a popular family therapy/systems theory term that means that dysfunctional families are either too open or too closed (I.e. letting too much information in or not enough information in). The healthy family is said to be in a balanced state known as negative entropy.
Why did the Weschler IQ test gain popularity over the Binet?
The Binet ultimately didn’t seem to be the best test for adults. Wechsler felt that the Binet was slanted towards verbal skills and thus, on Wechsler’s test, he added “performance skills” to ascertain certain attributes which might have been cultivated in a background that did not stress verbal proficiency.
What are the different Wechsler tests?
- The WPPSI - This is the Wechsler Preschool and Primary Scale of intelligence that is suitable for kids from 2.5 years = 7 years, 7 months.
- The WISC-IV - This is the Wechsler Intelligence Scale for Children and is appropriate for kids afters 6-16 and 11 months
- The WAIS-IV - This is the Wechsler Adult Intelligence Scale and is intended for ages 16 - 90.
The WAIS-IV and the WISC-IV no longer provide verbal and performance IQ scores.
What is the WAIS-IV?
This is the Wechsler Adult Intelligence Scale, intended for people ages 16-90.
- It is based on neurocognitive research and the Cattell-Horn-Carroll leading theory of human intelligence.
- It can be administered and scored online and takes 60-90 minutes to complete.
- The test used to have object assembly and picture arrangement sections, but those have been dropped.
- It is made up of 10 subject areas also called subtests which make up 4 index scores: verbal comprehensive index, perceptual reasoning index, working memory index, and processing speed index
- FISQ stands for full scale IQ. It has a mean of 100 with a standard deviation of 15
- There is less emphasis than before on crystallized intellience
- This test measures IQ from 40-160. The Stanford-Binet, however, has a wider range so is a better measure for extremely low IQs or giftedness
What do the terms “floor” and “ceiling” mean on a test?
On any given test, the floor refers to the lowest possible score and the ceiling is the highest possible score.
What is the Myers-Briggs Type Indicator (MBTI)?
This is a personality inventory based on Carl Jung’s analytic psychology. The MBTI uses dichotomous types (extraversion vs. introversion, sensing vs. intuition, thinking vs. feeling and judging vs. perceiving). The test results in a 4 letter score like INTJ.
When a test is guided by theory like this one is, it is known as a theory-based test or inventory.
What is the benefit of group IQ tests like the Otis-Lennon, the Lorge-Thorndike, and the California Test of Mental Abilities?
Group tests are quicker to administer. School districts, government, and industry often prefer tests which can be administered to many people simultaneously. The catch, however, is that group tests are less accurate and have lower reliability.
When did the group IQ test movement begin?
With the Army Alpha and the Army Beta during World War II
What is a culture-fair test?
This is a test in which items are known to the subject regardless of his or her culture. The culture-fair test attempts to expunge items which would be known only to an individual due to his or her background. Ethics now consider it unethical to administer a test to a client from a given population unless that particular test or inventory has been normed on that specific population. For example, if you gave a Black client a test that had not been normed for Black people, this would be considered to be a violation of ethics.
What is the Black vs. White IQ controversy?
Arthur Jensen sparked tremendous controversy when he suggested in a 1969 article that the closer people are genetically, the more alike their IQ scores will be. Jensen then leveled the charge that white people sore 11-15 IQ points higher than African Americans regardless of social class. His theory stated that due to slavery, it was possible that African Americans were bred for strength rather than intelligence. Urie Bronfenbrenner claimed that Jensen relied on twin studies with poor internal validity. Others felt that genetic influences contributed less than 50% to IQ.
Who is John Ertl?
John Ertl claimed he invented an electronic machine to analyze neural efficiency and take the place of a paper and pencil IQ test. The device relies on a computer, an EEG, a strobe light, and an electrical helmet. The theory is the faster one processes the perception, the more intelligence he has. Most counselors don’t buy this idea.
Who is Raymond B. Cattle?
He is related for the concepts of fluid intelligence (inherited neurological intelligence that decreases with age and isn’t dependent on culture) and crystallized intelligence (intelligence from experiential, cultural, and educational interaction). Crystallized intelligence is measured by tests that focus on content. Fluid intelligence has been called “content-free reasoning” such as a block design or analogy problem.
Who is Robert Williams?
Robert Williams is an African American psychologist who created the Black Intelligence Test of Cultural Homogeneity (BITCH) to demonstrate that Black people often excelled when given test laden with questions that would be familiar to Black people. Williams charged that tests like the Binet and the Wechsler were part of scientific racism. Wiliams, a victim of the system himself, scored an 82 on an IQ test when he was 15 years old and a counselor suggested bricklaying because he was good with his hands. Williams rejected the advice and went on to get a PhD. IQ tests, though controversial, are often excellent predictors of school success since schools emphasize values that have been heavily influenced by European culture.
What is the 1979 Larry P. vs. Wilson Riles, Superintent of Public Instruction, State of California court case?
In this now oft-quoted court battle, it was initially ruled that IQ tests were racially biased against African American children who were overly represented in EMR (educable mentally retarted) classes (proper terminology at the time) based on IQ scores.
What is the MMPI-2?
The MMPI-2 is a standardized personality test, known as a “self-report” personality inventory. The client can respond true or false to 567 questions. The new MMPI-2 is intended to help clinicians diagnose and treat patients – and is an updated version of the MMPI that also attempted to eliminate sexist language.
The MMPI is suitable for people over 18 and requires a 6th grade reading level.
What is the client shown in a projective test?
In a projective test, the client is shown neural stimuli. The idea here is that the client will project his or her personality if given an unstructured task. More specifically there are several acceptable formats for projective tests:
- association - I.e. “what comes to mind when you look at this inkblot”
- completion - “complete these sentences with real feelings”
- construction - I.e. drawing a person
The theory is that self-reports like the MMPI do not reveal hidden unconcsciou impulses. In order to accomplish this, the client is shown vague, ambiguous stimuli like an inkblot. Some believe that by using projective measures a client will have more difficulty faking his or her responses and that he or she will be able to expand on answers. Examiner bias is common when using projective – a therapist using projective measures needs more training than someone who only works with self-report tests.
What is the 16PF?
The 16PF is the 16 Personality Factor Questionnaire and was developed by Raymond B. Cattell. The test is suitable for people over 16 and measures key personality factors like assertiveness, emotional maturity, and shrewdness. A couple can even decide that they will each take the 16PF and them both individual and joint profiles can be compiled for use in marital counseling. Tests and inventories like the 16PF which analyze data outside of a given theory, are called factor-analytic tests or inventories rather than theory-based tests.
What was Oscar K. Buros’s contribution to the world of testing?
He created the Mental Measurements Yearbook which was the first publication to ever review available tests. The University of Nebraska then set up a center to continue to produce MMY books to help counselors pick appropriate tests.
What kind of counselor would most favor a projective measurement?
Psychodynamic. Projective measures try to access the unconscious mind and unconscious impulses.
What is a particularly important type of validity when choosing an aptitude test?
Predictive validity – because the test is supposed to measure someone’s potential.
What does the TAT (thematic apperception tests) consist of?
The TAT consists of 31 cards with pictures of them. The test can be give to people 4 and up and uses up to 20 of those cards (19 selected to fit the age and sex of client + one blank card). The pictures on each card are intentionally ambiguous and the client is asked to make up a story for each of them.
What is the Rotter Incomplete Sentence Blank (RISB) test?
This is a projective test in which the subject completes an incomplete sentence with a real feeling.
What does test bias primarily result from?
Test bias often comes from the test being normed solely on white middle-class clients.
What is the Bender Gestalt II test?
This is an expressive progressive measure which first and foremost is known for its ability to discern whether or not brain damage is evident. It is suitable for 4+ and asks the client to copy 16 geometric figures which they can look at while constructing their drawing.
Who do interest inventories work best for?
Interest inventories work best with individuals who are of high school age or older, as interests are not always that stable before then. Interests become quite stable around age 25.
What are some major criticism of interest inventories?
- They emphasize professional positions and minimize blue collar jobs
- interests and abilities are not actually highly correlated. I.e. a client could have a lot of interest in music but could really dislike being a musician.
- The person often tries to answer in a social acceptable manner (social desirability)
What is a positive aspect of interest inventories?
They are reliable and are generally non-threatening to the test taker.
What is the AARC (Association for Assessment and Research in Counseling)?
This is one of 202 ACA divisions and is an organization for counselors who are primarily interested in testing.
What is social desirability?
This is the tendency for people to try to answer a questioning a socially acceptable manner.
What is deviation in terms of test-taking behavior?
This is when someone purposely gives unusual responses. This is the opposite of social desirability
What is acquiescence?
When a client always agrees with something.
What is an aptitude-achievement test?
This is a test like the GRE that attempts to BOTH test your knowledge and predict some kind of performance. Aptitude-achievement tests are like GET, MAT, MCAT, SAT, etc.
What does the standard error of measurement tell you?
How accurate or inaccurate a test score is. If a client decided to take the same test over and over again, you could plot a distribution of the scores. This would be the standard error of measurement for the instrument. The lower the standard error of measurement the better – a low standard of measurement means high reliability.
What is social loafing?
Social loafing describes a phenomenon in which a person in a group puts forth less effort than if he or she were attempting to accomplish the same goal individually.
How does increasing a test’s length impact the reliability?
Increasing a test’s length raises reliability. Decreasing a test’s length shortens reliability (so the reliability coefficient would go down)
What are CPT codes?
CPT (Current Procedural Terminology) codes are used to let insurance companies, managed care firms, etc know which service you provided (I.e. individual therapy, family therapy, etc)
What are informal assessments?
These are things like self-reports, case notes, checklists, sociograms of groups, interviews, journals, etc that are considered informal assessments.
What is a good way to tell if a test is intended for a certain population?
Reading the test manual should indicate the target population for the test.
What is one major testing trend?
Computer-assisted testing and computer interpretations
What is one contradictory future trend?
Some want to rely on tests more while others want to rely on them less. Some counselors would like to see future tests that assess creative and motivational factors.
What kind of test would a word association test be?
A projective test
Are infant IQ tests more or less reliable than those given later in life?
Infant IQ tests (“toddler tests”() are generally more unreliable than those given later in life – though they are sometimes capable of picking up on gross abnormalities like severe intellectual disabilities.
What is a good practice for counselors in regard to testing?
To never generalize on the basis of a single test score.
What is Public Law 93-380, also known as the Buckley Amendment
This law states that people over 18 can inspect their own records and those of their children. The Family Educational Rights and Privacy Act also stipulates that information cannot be released without adult consent.
Who is Lewis TErman?
Terman was associated with Stanford University and Americanized the Binet. The test later came to be known as the Stanford-Binet.
