Week two - Core Principles of Assessment Flashcards
What is the role of community and kinship when it comes to consent for Aboriginal and Torres Strait Islander?
Community and kinship play an important role in who is responsible for a child or someone who cannot give consent. There may be multiple people who are able to give consent, not just the parents.
What is Classical Test Theory?
Classical (Psychometric) Test Theory states that the observed score in a psychometric test is made up of the true score and error. O = T + error
Reducing error will allow tests to deliver observed scores that are more reflective of the true score.
Error can stem from multiple sources.
What is Generalisability Theory?
What is Item Response Theory?
What are some ways that allow tests to be administered in a standardised manner?
All participants get the same amount of time.
Questions are asked in the same way.
Tests are scored in the same way.
What are some of the sources of error for psychological tests?
Testing environment, e.g. is the room quiet? Does the participant know the test taker?
Tester characteristics, e.g. how the tester is behaving. Is their behaviour increasing the anxiety of the test taker?
Test taker characterists, e.g. are they tired? Have they eaten yet? Have they had prior negative experiences taking tests?
Has tester and test-taker rapport been shown to improve test scores?
Yes.
What is the Expectancy Effect?
Also known as the Rosenthal Effect from the study that looked at this conducted by Rosenthal et al. Teacher and students who were going to “excel”.
AKA The Pygmalion Effect.
Tester beliefs and expectations can influence how a person performs on a test.
In an experiment that was looking at Expectancy Effect, what did they find when they randomly labelled rats as ‘maze bright’ or ‘maze dull’ ?
They found that ‘maze bright’ rats completed the mazes, on average, quicker than the maze dull rats. It was hypothesised this occurred due to the different treatment researches may have inadvertently given the rats.
What are some of the sources for test score error?
Examiner Bias - e.g. Halo effect - higher marks awarded to children who are warmer or nicer.
Familiarity with examiner - knowing the examiner leads to improved test scores.
Reinforcement by examiner - e.g. encouragement or discouragement. Approval and disapproval.
Experience of the examiner. Examiners who are not well-accustomed to the test are not as likely to score or interpret scores as correctly.
Examinee (Test-taker) factors - e.g. if the test is in their second language. If they are very anxious this may impact their abilities.
Stereotype threat. e.g. women are stereotyped to not be as good at math as men. An example of this is some researched that looked at test scores from math test completed by men and women. One condition involved telling the women that the test was looking at gender differences in math competency. Women performed worse in the this condition. The other condition did not state this and was a just a math test. In this condition women did not perform worse than men.
What is one of the theories discussed in lecture about why negative stereotypes can negatively impact test performance?
If a test taker feels that there is a negative stereoptype about one of their characteristics, such as being a woman, then they may be more focused on the self when taking a test, decreasing their working memory capacity when taking the test. This can lead to reduced test scores.
What are Norm Referenced Scores?
What are Criterion-Referenced Scores?
Norm referenced scores refers to scores being compared to a reference group. E.g. intelligence test scores are compared to the population scores.
Criterion-referenced scores refer to scores that are compared to a benchmark or cut-off. E.g. if a score of 50/100 is required to pass, then the test scores will be compared to this benchmark and not the population scores or a reference group. e.g. tests that examine whether people meet the required criterion to live on their own.
Most psychological tests use norm referenced scores.
What is one of the pitfalls of using norm referenced scores?
In order for norm referenced scores to be used effectively, the reference group needs to be appropriate. E.g. if you are comparing the IQ test scores of an 8 year old to an adult reference group then you are not going to getting a good understanding of the intelligence level of the child at that developmental stage.
What are “norm” tables?
They are tables that have test score data about different subgroups within the population. E.g. IQ scores for different subgroups of children, such as age, geographic location, gender.
When selecting a test, what is very important to consider? Hint - Norm tables.
The way most psychological tests are scored is by norm-reference scoring. This means that test-taker’s score is compared to the scores of a the reference sample. If the reference sample is not repressentative of the test-taker, then this means scores may be inappropriately interpreted. This can have profound practical and psychological effects on people.
What is the Flynn Effect?
The Flynn Effect refers to the trend that IQ appears to be increasing on average per year.
Why is it important to consider whether the normative sample is representative of your client when selecting a psychological test?
Tests can have normative groups that are outdated. The data is too old.
The normative sample may not be representative of your client’s demographics.
Is the sample size of the normative sample big enough.
It is important to consider whether you are administering a test in the same way it was administered to the normative sample. why?
Administering a test under different conditions, such as allowing the client more or less time to take the test than was intended, can mean that the test result cannot reliably be compared to the normative sample results. Interpretation can therefore be affected.
What are scaled scores?
Scaled scores are standardised scores that can be used to compare to mean and standard deviation of a given population a test is measuring.
e.g. WAIS-IV has a mean of 10 and sd of 3.
You cannot just consider the test scores
What are some examples of standard scores?
Z-scores, Wechsler, T-scores, IQ scores.
Is reliability necessary for validity?
Yes, but it is not sufficient.
What are some measures of reliability?
Test-retest reliability.
Interrater reliability e.g. Cronbach’s alpha.
What is a reliability coefficient?
Between 0 and 1. Measures reliability of a question. This is based on…??? Cronbach’s alpha?
What is the intraclass coefficient?
What is a problem with test-retest coefficients ?
They do account for learning effects.
Parallel forms coefficients can be used to help mitigate this, but not eliminate it.
What is a parallel forms coefficient?
What is the standard error of measurement (SEM)?
SEM is a confidence interval/error bar that is calculated around a test score, that takes into account the reliability of the test.
It is calculated as: SEM = sd of test x sqrt(1-reliability score)
Therefore, the higher the reliability of the test, the lower the SE.
What are some ways of calculating internal consistency?
Split half method
Kuder-Richardson’s Method
Cronbach’s Alpha.
What are the differences between content validity, criterion validity, construct validity?
Content validity - is the measurement being used a good measure of the concept being tested for? E.g. if the test is looking at anxiety, do the questions ask about or test for anxiety? Not a form of validity that is reported on.
Criterion validity - if a test uses cut-off criteria to categorise respondents/examinees, how well do these cut-offs correctly categorise individuals? e.g. how well does a test classify someone as having an eating disorder?
Concurrent validity - how well does one test classify individuals compared to another test, e.g. how well does a survey classify MDD compared to a interview with a psychiatrist?
Predictive validity - how well does a test score predict future outcomes? e.g. ATAR.
Construct validity - how well does the intrument (test) measure/s used actually measure the construct of interest? e.g. for a test on anxiety, is the measurement also picking up on depression symptoms and classifying them as anxiety?
Convergent validity -
Discriminant validity -
