lecture 4 - ecological validity / construct validity / confirmatory factor analysis and WAIS

Question 1

what is ecological validity

Answer 1

Ecological validity refers to the degree to which test performance corresponds to real-world performance

Very relevant to applied situations – what are the functional consequences of
stroke, brain injury, dementia etc.

Question 2

it is suggested that some neuropsychological tests have ______ validity

Answer 2

limited

Question 3

how can ecological validity be assesed

Answer 3

Test scores can be compared with rating of everyday behaviour for the domain of interest using

-Self and informant questionnaires
-Clinical rating scales
-Observation of simulated tasks

Question 4

executive function

Answer 4

Executive function refers to cognitive skills in problem solving, decision making, planning and completing tasks, and reflecting on activity

Question 5

dysexecutive problems

Answer 5

Dysexecutive Problems: Starting or finishing tasks, Planning ahead, Making decisions, Thinking through problems and forming solutions, Behaving appropriately and controlling emotions such as anger

Question 6

tests that measure dysexceutive problems

Answer 6

Range of tests that measure dysexecutive problems, e.g. Behavioural Assessment
of the Dysexecutive Syndrome (BADS: Wilson et al., 1996).

Dysexecutive Questionnaire (DEX; Burgess et al., 1998)

• 20 items measuring behavioural, cognitive, motivational and emotional changes from pre-morbid functioning generating.

-Neuropsychological tests of executive function can be compared with DEX scores

Question 7

ecological validity
- correlation
-what are test results affected by

Answer 7

Burgess reports modest correlations
Better correlations when using informant, clinician, parent/carer ratings

Test results will be affected by:
* Environment
* Limited behaviour observed
* Compensatory strategies
Ratings of everyday functioning are by no means perfect
* Relatives may be protective, or may be overly negative in their
appraisals
Taking these into consideration, ability to predict is improved

Question 8

burgess et al 1998- correlation between everyday executive problems (DEC proxy rating) and some putative EF tests

Answer 8

-phonemic fluency - eg you ha e 60 seconds to say words beginning with A - good test in EF -0.35

-modified card sorting test -set of cards with 3 concepts, colour,shape, number, individual has to sort by one element 0.37

• six elements -6 subtests, maths,dictation, picture (both have 2 )0.40
• cognitive estimates - like a pop quiz, more about how they think through the question and reason (how tall is a london bus, how many camels are there in the UK, questions that don’t necessarily have an answer) 0.29

Question 9

what is construct validity

Answer 9

Construct validity is the broadest form of validity: does a test measure what it’s meant
to test

Question 10

constructs are ___________. but the test measures them

Answer 10

Constructs are unobservable but the tests measure them – the WAIS measures intelligence

Question 11

Researchers can assess construct validity by making predictions on how scores may change in various ways
-what are the ways

Answer 11

Test Homogeneity
◦ Evidence from changes with age
◦ Evidence from pre-test and post-change changes
◦ Evidence from distinct groups
◦ Convergent and divergent evidence
◦ Factor Analysis

Question 12

what is factor analysis

Answer 12

Factor analysis determines the underlying relationships between sets of variables such as
test scores.

Relationships are called Factors (i.e. the construct such as intelligence or personality traits).

Question 13

Psychometric factor analysis is used as a data reduction technique. explain this

Answer 13

Here we are NOT interested in the statistical procedures!

In psychometrics FA is used as a data reduction technique – it take individual test and the correlations between them. Patterns of scores clustering together suggests they are measuring the same thing
◦ E.g. Do you like going to parties / socialising / are you the life and soul of a party etc., and similar scores may all be measuring – extraversion

So, what’s this got to do with CV?
Researchers can collect data and make specific predictions on how scores should correlate
based on their theory.

Question 14

Traditionally, factor analysis can be used to study _______ validity
-what is a more recent technique to study this

Answer 14

traditionally, factor analysis has been used to study construct validity

Confirmatory factor analysis is a relatively recent technique for evaluating the
construct validity of psychological tests

It is now very widely used in psychological research. Therefore you should become familiar with it if you want to make sense of the literature on psychological assessment and in other areas of psychology

As a starting point we will use the example of the WAIS-IV and ask whether intelligence is made up of one, two, or more constructs

Question 15

representation of CFA model - one factor model for WAIS-IV. (consists of )

Answer 15

latent variables - (underlying factors) are represented as ovals or circles

observed variables are represented as squares or oblongs

single headed arrow indicates causality

(error is also treated as a latent variable; represents the variance in observed variables that is not explained by the factor

Question 16

cfa model
-can allow for _________ between factors too

Answer 16

relationship
(slide 12) - here the two factors are allowed to correlate (i.e they are related)
slide 13 - Here the two factors are FIRST order factors; a further higher (SECOND) order factor accounts for the relationship between the two factors. It is models of this
type we will be looking at for the
WAIS-IV

Question 17

fit statistics

Answer 17

We get a number of statistics that tell us which model the data best fits – these are collectively called fit statistics.

For example - A large chi square value ( x2 ) tells you that a model has poor fit.

Question 18

Cumulative Fit Indices
-what do high values indicate

Answer 18

Cumulative Fit Indices (CFI): there are a variety of these. All have the
characteristic that they range from 0 to 1.

High values (i.e. at least above 0.9) for a CFI suggest that the model has good fit

There are other measures of fit but we will limit ourselves to these two

Question 19

Results of testing competing CFA models of the structure of the WAIS-IV in UK validation sample