Reliability and Validity

Question 1

what is psychometrics?

Answer 1

it is a branch of psychology with the design, administration, and interpretation of quantitative tests for the measurement of psychological variables

• involves the measurement of directly unobservable mental constructs
• what does something have to be to be considered a measured construct

Question 2

what is a mental construct?

Answer 2

theoretical, intangible quality or trait

• A psychological test aims to measure mental constructs by inferring an individual’s true score on the construct from their observed test score
• try to get as close a match of the observed score and the true score

true score: a latent construct most correct to the specific individual,

Question 3

what are the first two (2/5) assumptions of psychometrics?

Answer 3

1. it is possible to measure psychological states and traits
   - you must be able to measure, and assign values to the state
   - you must define the construct being studied (operationalisation)
2. various methods of measuring aspects of the same thing can be useful, including
   - format - multiple choice, interview, etc
   - administration - individual vs group, timed, self-reported or administered
   - scoring - is it categorical or dimensional

Question 4

what are the last three (3-5) assumptions of psychometrics?

Answer 4

3. assessment can provide answers to some of life’s most momentous questions
   - for example hsc means you are competent for a job
4. error is part of the assessment process
   - true score + measurement error = observed score
5. testing and assessment can be conducted in a fair and unbiased manner
   - cultural bias, use criteria that applies to the culture

Question 5

what is reliability?

Answer 5

consistency of measurement (does the test give consistent results when it’s given at different times, by different raters)

Question 6

what is validity?

Answer 6

accuracy of measurement (does it measure what it’s supposed to measure)

Question 7

what are the two types of measurement error?

Answer 7

random error: unpredictable influences that vary from measurement to measurement

• in two directions - May increase or decrease a score

systematic error: biases that influence scores in a similar way across multiple measurements

• in one direction - only increasing or decreasing a score

Question 8

what are the types of reliability?

Answer 8

temporal stability (test-retest reliability), inter-rater reliability, internal consistency reliability

Question 9

what is test-retest reliability?

Answer 9

does taking the assessment again yield the same results?

• potential limitations may include the effect of practice or fatigue and the fact that it can be expensive to undertake.

Question 10

what is internal consistency reliability?

Answer 10

are there positive correlations among items that are intended to measure the same construct?

• the extent to which test items that propose to measure the same construct actually show similar scores

it can be increased by increasing the number of items, or to remove the items that are uncorrelated with majority of the others

Question 11

what is inter-rater reliability?

Answer 11

can raters agree on the scores of an instrument? can they yield the same results?

• helps watch for rater bias etc

Question 12

what are the types of validity?

Answer 12

content validity, construct validity, predictive validity

Question 13

what is content validity?

Answer 13

the degree to which the items adequately sample the mental construct in question

• for example, when measuring intelligence if you used a scale that only measured vocabulary in comparison to a scale that measured vocabulary, arithmetic, etc then the latter would have better content validity

Question 14

what is construct/criterion validity?

Answer 14

it is seperated into two subtypes:

convergent validity - the extent to which a scale correlates with other tests that appear to measure the same construct

discriminant validity - the extent to which a scale does not correlate with other tests that do not appear to measure the same construct

Question 15

what is predictive validity?

Answer 15

the degree to which the test is able to predict present or future performance on a real-world outcome

it is seperated into two subtypes:

concurrent validity: present
predictive validity: future

Question 16

what is the definition of a mental disorder in the DSM?

Answer 16

a syndrome characterised by clinically significant disturbance in an individual’s cognition, emotional regulation or behaviour that reflects a dysfunction in the psychological, biological or developmental processes underlying mental functioning

Question 17

when is something not a mental disorder?

Answer 17

• expected or culturally approved response to a common stressor or loss is not a mental disorder
• socially deviant behaviour, and conflicts primarily between the individual and society are not mental disorders unless it results from a dysfunction

Question 18

what are some changes to the DSM since the DSM-5?

Answer 18

• homosexuality removed in 1974
• removal of multiaxial system in DSM IV-TR which split clinical disorders with personality disorders even though there was no scientific reason for this decision
• dsm-5 now pays more attention to contextual influences of gender and culture with respect to symptom presentation

Question 19

how is the DSM organised?

Answer 19

• developmental and lifespan considerations
• chapters and diagnostic categories whereby childhood disorders are outlined first and others later in life
• then grouped with similar types of disorders

Question 20

what is differential diagnosis?

Answer 20

other disorders that need to be ruled out for the disorder looked at to apply to the person

• E.G before diagnosing someone with GAD you must first rule out anxiety disorder to another medical condition, substance-induced anxiety disorder

Question 21

what are the advantages of the DSM 5?

Answer 21

• common language (shorthand communication, so that a client’s symptoms can be referred to quickly)
• guide treatment recommendations and access to funding
• benefits for research (research can be conducted on the same phenomena, identifying prevalence, morbidity and mortality rates)
• normalising and understanding the clients experience so that they can identify what they have

Question 22

what are the limitations to the DSM 5?

Answer 22

• arbitrary cut-offs to determine what is abnormal vs normal
• categorical vs dimensional
• diagnostic validity (challenges with specificity)
• influence of the medical model (reductionistic, simplifies complex human experience)
• lowering and broadening of diagnostic thresholds (some people abuse this)