Biais, Échelles de validité Méthodes de construction d’instrument de mesure Types d’items et échelles de réponses Traduction / adaptation transculturelle

Question 1

Biais des tests

Answer 1

Une chose très importante est de ne pas confondre différence de moyenne entre des groups et biais

-The public sometimes has the impression that all assessment instruments are biased (e.g., by age, by sex/gender, by ethnic group, by clinical group, etc.).
-This is sometimes the case and it is the duty of the test user to be aware of it. devoir de l’utilisateur.trice
Reminder:
Bias = systematic error, is not random
Biais = de l’erreur systématique, n’est pas aléatoire

One very important thing is not to confuse difference in means between groups with bias

Question 2

Biais des tests

Answer 2

Differences in means between certain groups are not a priori a bias since some are theoretically/conceptually expected
e.g., In adolescence, few or no differences in means between ethnic groups for behavior problems, but differences by sex/gender
e.g., In adulthood, presence of sex differences in some personality traits, but few or none in adolescence
normative: compared to mass population

Question 3

biase de test - Un instrument d’évaluation est biaisé

Answer 3

Un instrument d’évaluation est biaisé «si les differences entre les membres de différents groupes sont identifiées sur la base de caractéristiques autres que celles que l’instrument prétend évaluer» (Merrell, 2008; Whitcomb, 2017)
Autrement dit, il y a présence de biais pour un instrument si le contenu, la procédure ou l’utilisation favorise ou défavorise systématiquement les membres d’un groupe plutôt qu’un autre et si cette différenciation est non pertinente à l’objectif de l’instrument

An assessment instrument is biased “if differences between members of different groups are identified on the basis of characteristics other than those the instrument purports to assess” (Merrell, 2008; Whitcomb, 2017)

In other words, bias is present for an instrument if the content, procedure, or use systematically favors or disfavors members of one group over another and if this differentiation is irrelevant to the purpose of the instrument

Question 4

how is reliability affected directly

Answer 4

As we have seen, the fidelity of scores on an assessment instrument can be compromised by various sources of measurement error
-We have also seen that the inferences and interpretations permitted with scores provided by an assessment instrument are dependent on the degree of validity of those scores.
-Validity can be affected directly by (a) response bias on individual items or
(b) scale score biasThe presence of bias is a critical issue for both test developers and test users
(a) biais de réponse aux items individuels ou par des
(b) biais des scores à une echelle

Question 5

Biais de réponse: heuristiques ou biais cognitifs

Answer 5

People who are being assessed and asked questions, whether about themselves or as an informant for a third party, are always at risk of being partially biasedFor example, in a job interview where a person has to answer a personality questionnaire, would they want to look their best? Or even better than their best?Even at a basic level, it is now recognized that the human cognitive system is “victimized” by several heuristics or cognitive biases (Kahneman, 2011; Kahneman, Slovic & Tversky, 1982)

Question 6

Heuristiques associés aux styles de réponses

Answer 6

Heuristiques: Stratégies cognitives utilisées pour simplifier et accélérer une décision en situation d’incertitude (Kahneman, 2011).
Heuristics: Cognitive strategies used to simplify and speed up a decision under uncertainty (Kahneman, 2011)Sometimes referred to as “mental shortcuts.”
Apply to behavioral evaluation/estimationVery useful when one does not know a person to be evaluated well enough.
Can also lead to misjudgment and “stereotyping” of people.

Question 7

Quatre exemples connus d’heuristiques

Answer 7

1. Heuristique de la représentativité
   Representativeness heuristicEvaluating a specific characteristic in terms of how well it matches a prototype (e.g., evaluating a child’s attention based on our ADHD prototype)
2. Heuristique de la disponibilité
   Availability HeuristicRating that is influenced by the things that come most easily (or frequently) to mind for the rater (e.g., children’s aggressive behaviors)Those things that come to mind more easily are considered more frequent and more representative of reality
3. Heuristiques de primauté / de récence
   Primacy / recency heuristics
   Evaluation that is influenced by the individual’s first vs. last impression
4. Heuristique de l’affect
   Affect heuristics
   Assessment colored by current emotional and affective state (e.g., bad mood leads to estimation of more behavior problems)

Question 8

influencent directement la validité

Answer 8

Response biases may seem trivial, but they can be very serious as they directly influence the validity of test scores
Diminished” validity can in turn compromise the quality of inferences and clinical decisions that are made about an individual (or group) being assessed

Question 9

Huit grands types de biais de réponse (see pictures )

Answer 9

1.Extrémité: responds are extreme
2. Indécision: neutral response
3. Acquiescement: say yes to everything
4.Objection: always say no
5. Désirabilité sociale: socially acceptable exaggerate the positive
6. Gestion défavorable des impressions (malingering): answer exageratevily negative
7. Réponse aléotoire ou negligent:random
8.Deviner (guessing):
9. halo

Question 10

Que faire pour prévenir ou minimiserles biais de réponse ?

Answer 10

Three things to do:
1. Manage the assessment situation
Anonymity, minimize frustration, give warnings (i.e., warn that there are validity scales)
2. Manage the content of the tests
Simple items (language level), content-neutral items (i.e., non-suggestive), conceptually clear response options
3. Specialized validity tests or scales

Question 11

Quelques exemples d’échelles de validité

Answer 11

Toutes ces échelles sont basées sur le même principe : des scores très élevés ou extrêmes suggèrent un problème potential
All these scales are based on the same principle: very high or extreme scores suggest a potential problem

Indeterminacy scale (e.g., the MMPI-2; Ben-Porath & Tellegen, 2008)
The full MMPI-2 questionnaire has over 567 items
Unanswered items, or items with multiple responses on the same item, are summed

Question 12

Échelles de validité- échelles de désirabilité sociale

Answer 12

1. Échelles de désirabilité sociale
   Échelle de désirabilité sociale de Marlowe-Crowne
   Marlowe-Crowne Social Desirability Scale (Crowne & Marlow, 1960)e.g., “I never lie”; “I like everyone I know”; “I have never been angry”.
2. Inventaire balance de style de réponse socialement desirable
   Self-Deception: generally honest, but overly positive responsesImpression management: dishonest responses, positive bias is used to (a) please others or (b) gain advantage

Question 13

Échelles de validité - Échelle de gestion dévavorable des impressions

Answer 13

Échelle de gestion dévavorable des impressions
Unfavorable impression management scale (e.g., the MMPI-2; Ben-Porath & Tellegen, 2008)
Tendency to respond positively to unlikely negative items (e.g., “I’m no good at anything”; “I have no talent”)
Difficult to distinguish effect with severe clinical cases (e.g., major depression or depressive personality disorder, etc.)

Question 14

Échelles de validité -
- Échelle de style de réponse extreme
-Échelle d’indécision

Answer 14

3. Extreme Response Style Scale
   Criteria proposed by the EDC (Parent et al., 2006)
   i.e., choosing the 1st or 7th choice of items an abnormally high number of times
4. Indecision scale
   Criteria proposed by the EDC (Parent et al., 2006)
   i.e., choosing an abnormally high number of times the central category, i.e., the 4th choice (the one in the middle) of the items

Question 15

Échelles de validité -
Incohérence variable des réponses (VRIN)

Answer 15

5. Variable response inconsistency (VRIN)Sum of the number of item pairs that were answered inconsistentlySimilar: “I don’t think before I act” - “I act without thinking about the consequences”Different: “I don’t think before I act” - “I think carefully before I make decisionsWe give 1 pt for each inconsistent pair and calculate a sumUsed to detect random responses réponses aléatoires (intentional or not) or confusion in a questionnaire

Question 16

Échelles de validité -
Incohérence vraie des réponses (TRIN)

Answer 16

True response inconsistency (TRIN)
In this one, only pairs of items that are conceptually different are used
Calculates a sum of the inconsistently true response item pairs minus the sum of the inconsistently false response item pairs
Used to detect inconsistent responses that indicate acquiescence l’acquiescement (very high score) or objection (very low score, possibly negative)

Question 17

Biais des items et tests

Answer 17

une fois que le niveau du trait est contrôlé
Aussi appelé «fonctionnement différentiel des items»
Item (or indicator) biasNot differences in scores on the trait, but systematic differences in the probability of responding in a given way for each item individually, once the trait level is controlled forAlso called “differential item functioning.”Compares the probability of endorsing items on a scale of individuals in different groups who have the same score/level on the traitSame principle as control variables in predictive studies (e.g., when “controlling for SES”)

Question 18

Biais des items et tests - Biais structurel

Answer 18

Biais structurel
Pour un instrument unidimensionnel, il peut s’agir de différences significatives des saturations factorielles entre deux groups
Pas banal puisque ceci signifie que le trait n’est pas mesuré de la même façon dans différents groupes
Pour un instrument multidimensionnel, (a) différences des saturations et (b) la structure factorielle n’est pas la même dans différents groupes
e.g., analyse factorielle révèle 3 facteurs pour les hommes, mais seulement deux pour les femmes

Question 19

Biais des items et tests -Biais critériel (ou critérié)

Answer 19

Criterion (or criterion-referenced) biasApplies to both concurrent criterion validity (independent criteria and contrasting groups) and predictive validitye.g., A temperamental trait that predicts later adjustment for one group of children, but not for anothere.g., an IQ test predicts success for one cultural group, but not for anotherCaution: the observation of differences between groups for predictive relationships can be expected because this is theoretically justified… it is not a bias then

Question 20

Biais des items et tests -Fidelity bias

Answer 20

Fidelity bias
Fidelity estimates are significantly different in different groups
Can be potentially important for interpretation
if bias is present, the level of confidence one can have in the scale scores varies across groups
observed group differences in means can then be partly explained by error

Question 21

Biais des items et tests

Answer 21

Although testing by comparing groups by sex/gender, ethnicity, cultural background, clinical group, etc., can be informative for many researchers, it often results in “over-generalization”«sur-généralisation»
Variation between individuals in the same group (intragroup variance) can be enormous (see figure distributions)As a psychoeducator, one must never lose sight of the fact that the purpose of a psychoeducational assessment is to interpret the scores and make recommendations for ONE particular individualpour UN individu particulier

Question 22

Méthodes de construction des tests

Answer 22

There are a wide variety of tests useful in psychoeducation (Hogan et al., 2017)
Tests of intellectual ability/cognitive skills
Achievement tests
Neuropsychological tests
Measures of personality/temperament
Measures of interests, attitudes, and values
Measures of psychopathology
One major category is often overlooked in psychology books
Measures of environmental constructs

Question 23

CONSTRUCTION OF TESTS

Answer 23

In general, professional organizations expect authors to have constructed their instrument in accordance with the criteria listed in the Standards for Testing in Education and Psychology (AERA, APA, & NCME, 2014)
Test construction and validation is a long-term process
Requires revisions before it is fully satisfactory
Can take place over several years, even a few decades

Question 24

Deux grandes méthodes de construction des tests

Answer 24

Deductive (or rational)
“conclude from propositions taken as premises”.
Inductive (or empirical)
“conclude by going from the facts to the law

Question 25

Deux grandes méthodes de construction des tests–Méthode deductive (rationnelle)

Answer 25

From a theoretical framework
Scientific theory
Constructs, domains, indicators (the test designers determine them according to the theory)
Clinical theory
We want to answer a practical problem or need
e.g., how to measure PES intervention? How to measure motivation to change?
Advantage: Clear theoretical context, logical consistency (i.e., nomological network often known a priori)

Question 26

Deux grandes méthodes de construction des tests–Méthode inductive (empirique)

Answer 26

Based on an empirical (or factual, or pragmatic) 1.approachItem analysis / Factor analysis: items statistically related to the construct are selected (may also include internal consistency, criterion validity, etc.)
2. Criterion-referenced selection: e.g., only items that differentiate groups are selected (e.g., MMPI-2 Antisociality scale)
However, the approach is never completely empiricalto generate items, there is always an underlying theory, even if it is implicit

Question 27

Méthode inductive (empirique) ADVANTAGES AND DISADVANTAGES

Answer 27

Advantages: Greater objectivity and more representative of reality; we verify our understanding of a construct, explicitly supported by data
Disadvantages:You don’t necessarily get what you want, the data dictate the final outcome (e.g., factor structure, etc.)e.g., data suggest that anxiety and depression items are combined into one factorStatistics can sometimes distort concepts due to sampling biase.g., statistics suggest eliminating a clinically important aspect, while discordant results are mostly the result of poor sampling, or too small a sample

Question 28

Types de questions et choix de réponse

Answer 28

There are a host of different types of items and even more different response choices, making it challenging to present and categorize them (Urbina, 2014)
Different items and possible response choices, depending on:
Type of construct being assessed
Specific uses of an instrument
Personal preferences of the authors
Questions can also be presented in several ways
verbally in an interview
visually in a paper and pencil version
visually in a computerized version (on a fixed computer, or with an application on a smart phone, a tablet)
etc.
The most basic distinction is the type of response that is asked of the person being assessed: type de réponse qui est demandé à la personne évaluée
(a) constructed response items and (b) selected response items (Urbina, 2014)

Question 29

Choix ou échelles de réponse - Items à réponses construites

Answer 29

Constructed response items
Also called “essay” or “open-ended” or “free-response” questions
A premise is presented to the test taker, but there is no constraint on a fixed answer choice
There are, however, some rules, so there are (a) long-answer and (b) short-answer open-ended questions

Question 30

Choix ou échelles de réponse–Items à réponses construites

Answer 30

Constructed response items
Also called “essay” or “open-ended” or “free-response” questions
A premise is presented to the test taker, but there is no constraint on a fixed answer choice
There are, however, some rules, so there are (a) long-answer and (b) short-answer open-ended questions

Il y a néanmoins certaines règles, ce qui fait qu’il existe des (a) questions ouvertes à réponse longue et (b) des questions ouvertes à réponse succincte

Question 31

Choix ou échelles de réponse –Items à réponses construites (suite)

Answer 31

Constructed response items (continued)
An example of a long answer would be:
“Describe your usual relationship with your child?”
An example of a short answer would be:
“Using no more than 4 or 5 words, complete the following sentence, “My usual relationship with my child is”: __________”
Constructed response questions are essential in interviews
questions à réponses construites sont essentielles en entrevue

Question 32

Choix ou échelles de réponse - Items à réponses sélectionnées

Answer 32

Selected response itemsAlso called “objective,” “forced-choice,” “multiple-choice,” “true or false” questionsA premise is presented to the respondent and he or she is placed under the cognitive constraint of a fixed response choiceThis is the most common type of item used in humanities, social sciences and psychological assessment instrumentsMore objective, easier to derive a numerical score, more reliable, shorter, etc.

Question 33

Choix ou échelles de réponse

Answer 33

When a person is asked to answer a selected-response question, he or she must perform four cognitive tasks (Tourangeau, Rips, & Rasinski, 2000)Comprehension: understanding the relevant contentRetrieval: retrieve the relevant information from memory that is needed to answerJudgment: making a judgment based on the retrieved informationResponding: reporting this judgment, based on the available response options

Question 34

Traduction / adaptation transculture

Answer 34

Principe d’un instrument standardisé suggère qu’il s’agit d’une étape qui doit être prise au sérieux
Crucial issue in QuebecVery often “home-made translations” are used, (1) without any study verifying their psychometric properties and/or (2) without collecting Quebec standardsSometimes a simple translation is insufficient, but often an adaptation is necessaryImportant: The understanding of the content, the meaning/significance of the items is more important than the exact translationPrinciple of a standardized instrument suggests that this is a step that must be taken seriously

Question 35

Six étapes de l’adaptation transculturelle

Answer 35

1. Translate and adapt items (minimum 2 people)
2. Method of choice: Back translation Traduction à rebours
3. Independent experts review the translation
4. Eliminate or adapt items according to their comments
5. Pilot study with targeted individuals
   Empirical validation (detailed evaluation of psychometric properties)
6. Standardization (establishing norms)

Question 36

Cinq façons d’établir l’équivalence transculturel

Answer 36

1. Semantic equivalence
   Do items veulent dire mean the same thing in both languages/cultures?
2. Content equivalence
   Is each item relevant (pertinent) in both languages/cultures to measure the construct?
3. Construct equivalence
   Are the factor loadings similar? Is the factor structure the same? Is the C/D validity similar?
4. Équivalence critériée
   Criterion-based equivalence
   Sometimes quite difficult to conclude that there is no criterion-based equivalence
   e.g., with a measure of parenting practices, a practice related to adjustment problems only in one version (or culture) is not necessarily a problem of the instrument
   5.Fidelity equivalence