Exam Revision Flashcards
What are the (5) major categories of psychological tests?
- mental ability tests: designed to assess cognitive function i.e. intelligence
- achievement tests: measures of previous learning i.e. NAPLAN
- personality tests: measures traits, temperament, disposition i.e MMPI
- interests and attitudes tests: i.e. high school vocational interests
- neuropsychological tests: yield information about brain function i.e. fMRI
Name four other ways to categorise tests
- paper and pencil tests vs. performance
- speed vs. power tests
- group vs. individual tests
- norm vs. criterion referenced tests
What is the difference between norm and criterion referenced tests?
Norm referenced tests compare an individual’s result to a bell curve of a representative sample to determine the level of performance i.e through a percentile, while criterion referenced tests can be thought of as pass/fail tests where an individual either meets a specific level of performance or they don’t
What are four settings where pscyhological tests might be used
- educational
- clinical
- employment
- research
What are the four fundamental assumptions in psychological testing?
- people differ in important traits
- we can measure these traits on a continuum
- these traits are relatively stable
- measures of the trait relate to actual behaviour
What is the differential perspective?
Assumes laws and theories may not apply to everyone, and instead that everyone is different - generally in psychology we are more interested in people’s differences than their similarities
What is the difference between psychological testing and psychological assessment?
Testing is just an element of psych assessment, and is a way to gather information about an individual, usually vie a standardised test. In addition to psychological testing, psych assessment requires piecing together of data to come to a conclusion and recommendation about an individual
What are the four main stages involved in psychological assessment?
- Initial referral
- Test selection
- Conducting the formal assessment
- Feedback and report writing
What are the 6 key elements of a psychological test?
- a procedure or device
- yields some kind of information
- this information is about behaviour or cognition
- this information is only a sample
- the procedure is standardised
- information should be delivered in a quantified form
What are the main sections of a psychological report?
- identifying data
- referral question
- background/history
- behavioural observations
- tests used
- results/findings
- summary
- recommendations
- signature
Name some of the major assessment methods used other than psychological testing
- case history data
- clinical interviews
- intellectual and cognitive tests
- behavioural observation
- self report
- psychophysiological measures
- projective tests
What is classical test theory?
Based on the true score model, it states that each person has an observed score on any give test that is made up of their true score, plus or minus any error from the test.
What is error (in statistics)?
Error is a component of the test score that is unrelated to the test-taker’s real ability or the trait being measured (we will therefore never be able to actually know the person’s true score)
What is reliability?
Reliability refers to the consistency of any given measure, i.e. measures with high reliability will consistently give the same result over and over again
What is systematic error?
Any source of error that is constant, proportionate, predictable - generally thought of more in relation to validity than reliability
What is random error?
Any source of error that is inconsistent, unrelated and unpredictable - because this sort of error changes each time and could cause a positive or negative change, on average people should score close to their true score
Name five sources of measurement error
- test construction i.e. different items on the same test
- test administration i.e. variations in testing environment
- test scoring or interpretation
- sampling error i.e. non-representative samples
- methodological errors i.e. poor training, unclear or biased questions
What is Item Response Theory (IRT)?
Provides a way to model the probability that a person with X ability level will correctly answer a question that is “tuned” to that ability level, incorporates considerations of item difficulty and discrimination
- Difficulty relates to an item not being easily accomplished, solved, or comprehended
- Discrimination refers to the degree to which an item differentiates among people with higher or lower levels of the trait, ability, or construct being measured
What are the four most common ways to estimate reliability?
- test-retest reliability
- parallel and alternative forms reliability
- internal consistency reliability
- inter-rater reliability
What are some of factors affecting choice of reliability estimates?
- Is the test homogenous (for measures of internal consistency)?
- Is the trait static or fluid (for test-retest)?
- Is the range of test results restricted (influence correlations)?
- Is the test qualitative or quantitative (for inter-rater)?
How are SEMs related to reliability?
Standard error of measurement is a standardised reliability estimate (that is also influenced by the SD) that tells us how confident we can be in a single score being reflective of the true score. From here, we can create a confidence interval to understand how broad the band is in which a score might sit.
What influence do SD and reliability have on the SEM?
- larger reliability means smaller SEM
- larger SD means larger SEM
Given this we generally want to try to have as good reliability as possible because the SD is hard to change.
What is the SED?
The standard error of the difference is a measure of how large a difference in test scores would be considered to be “statistically significant”
What is a standard score and how is it used to interpret test scores?
A standard score is a raw score that has been converted from one scale to one with a predefined mean and SD. This is useful because it allows different sets of scores to be compared, and this is particularly relevant when comparing an individual to a set of normal scores to determine where they fall compared to the rest of the population
What is validity?
A judgement or estimate of how well a test measures what it purports to measure i.e. scales validly measure weight, but not IQ
What are the four types of validity
- Construct validity: determined through a combination of results matching results from other related tests, and whether the idea behind the test is theoretically valid
- Content validity: based on the evaluation of content covered by the test
- Face validity: whether the test appears to cover the relevant content
- Criterion validity: obtained by evaluating the relationship between scores on your test and other tests/measures
How is face validity measured?
a judgement concerning how relevant the questions on a test appear to be
How is content validity measured?
- test blue print to ensure all relevant concepts are covered
- Lawshe’s content validity ratio (CVR) to determine whether each item is necessary
How is criterion validity measured?
- concurrent reliability: an index of the degree to which a test score is related to some criterion measured concurrently
- predictive reliability: an index of the degree to which a test score predicts some criterion, or outcome, measured in the future
- the validity coefficient: a correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion
How is construct validity measured?
- factor analysis
- convergent evidence
- divergent tests
- distinct groups show differences on the test
What is test utility?
Utility is the usefulness or practical value of testing
What are three factors that effect utility?
- Psychometric soundness: generally higher validity = greater utility but valid tests aren’t always useful i.e. ones that take hours and hours
- Costs: economic costs, time costs, ethical considerations can diminish the utility of the test even if the test is valid
- Benefits: if there are large number of benefits of a test, these might outweigh any potential costs i.e. better data, relevance, more reliable etc.
What is a cut score?
A cut score is a score in a range of measures that indicated a test-taker is at a certain level or category, and is used to differentiate people (i.e. score of 24 or above means depressed)
What are the five methods for setting cut scores?
- The Angoff Method: judgements of experts are averaged to produce a cut score
- The Known Groups Method: entail collection of data on the predictor of interest from groups known to possess a trait, and known not to possess a trait, then a cut score is selected that is best able to discriminate these two groups
- Item Response Theory (IRT) Methods: use the item difficulty parameter
- Discriminant Analysis: statistical techniques used to quantify how well a set of identified variables can predict memberships to groups of interest
- Receiver Operating Curves: derives the sensitivity and specificity associated with different cut points that classify individuals as having or not having a condition of interest
- Youden Index: cut point selected based on the maximal sensitivity and specificity
What is a base rate?
The true prevalence of the condition of interest in the population - must be considered when setting cut scores
What are ethics?
a set of principles to guide professional behaviour
Give two reasons why ethics in psychological testing are important
- Many members of the public first encounter psychology through psychological testing (need to uphold the name and standard of the profession
- Decisions made on the basis of test results may have lifelong implications for the person being tested
What are the 5 steps in making ethical decisions?
- Recognise that there is an ethical issue present
- Clarify the ethical issues
- Generate and examine available actions
- Choose and implement preferred option
- Reflect on and review the process
What are the 3 principles of the APS Code of Ethics?
- Respect for the rights and dignity of people and peoples
- Propriety
- Integrity
What does the first APS principle entail?
Respect for rights and dignity means valuing and respecting individuals’ rights, including the right to autonomy and justice. This has implications for principles such as informed consent, and the right to withdraw consent at any time, along with privacy and confidentiality
What does the second APS principle entail?
Propriety incorporates principles of beneficence, non-maleficence (including competence), and responsibility to the clients, the profession, and society
What does the third APS principle entail?
Integrity reflects the need for psychologists to have good character, and acknowledges the high level of trust intrinsic to their professional relationships and the impact of their conduct on the reputation of the profession
Which section of the APS Code of Ethics refers direct to ethics in research, and what does this section outline?
B.13; the need to use appropriate and reasonable tests, report results accurately, and keep classified tests available only to those qualified to use them
What is the concept of competence?
Refers to the ability of a psychologist to choose, conduct, and interpret tests when they are used, and understand the issues or limitations of any given test