Week 1 Flashcards
What does Psychology Measure?
- Human Behaviour
- Feelings
- Cognitions
Psychological constructs are often abstract and therefore can’t be measured directly
Assumptions of Psychological Testing
- Psychological Traits & States Exist
- Psychological Traits can be measured
- Test-Related behaviour predicts non test related behaviour
- Tests and other Measurement techniques have strengths and weaknesses
- Various Sources of Error are part of the assessment process
- Testing and assessment can be conducted in a Fair and Unbiased manner
- Testing and Assessment benefit society
Assumption 1: Psychological Traits and States Exist
Construct: Informed concept backed by Scientific Theory explains behaviour affect, and cognitions
Traits: And enduring characteristic within people that varies between them
What is measured
State: Similar to trait,s but less enduring
There are differences and variation amongst individuals
Assumption 2: Psychological Traits & States can be Quantified & Measured
- Before we measure something we need to agree what it is
- Important - Not only definition of something
- Also the operational definition of something
Operational Definitions
a description of something in terms of the procedures, actions, or processes by which it could be observed and measured.
Assumption 3: Test Related Behaviour Predicts Non Test Related Behaviour
- If we can’t measure a construct directly we can use proxy measures
- Trust that the behaviour indicates the construct it reflects
- Also trust that the construct connects with other construsts the way we predict they should
Prediction in Constructs
“Prediction is a central, indeed nearly ubiquitous, activity of psychologists. Many clinical decisions, such as treatment selection, depend on predictions. Psychologists, or at least applied psychologists, are, therefore, obliged to know as much as possible about how to make good predictions.” (Grove & Lloyd, 2006, p. 192)
Assumption 4: Tests and Other Measurement Techniques Have Strengths and Weaknesses
- We need to be aware of the strengths and weaknesses of the tools we use
- These are not always perfect representations of our constructs
- Most measures have limitations like acquiring resources
- Test takers should be informed of the details of any tests they are required to complete
Assumption 5: Various Sources of Error are Part of the Assessment Process
- We are only measuring proxies not the construct itself
- Other sources of validity will impact our results
- ‘Error’ does not mean something has gone wrong
- Errors can be indicator of how good our measure is working
- Other factors present that affect what we measure
Error in Psychological Measurement
Refers to the idea that there is variability that we can’t always account for.
Doesn’t necessarily mean our data is wrong
Assumption 6: Testing and Assessment can be conducted in a Fair and Unbiased Manner
- This is a controversial assumption
- Example: WEIRD samples in research
Assumption 7: Testing and Assessement Benefit Society
- Some think that tests do more harm than good
- Particularly standardised testing can be seen as hindrance
- Standardised testing are not always a true measure of an indiviudal
- But what would the world be like without them?
Survey Based Assessment
- The major way to measure and assess concepts
- How we operationalise the constructs we want to know about
- Constructs can’t be seen or observed so we infer them from behaviour using indicators
Self Report Measures
- Beck Depression Inventory
- Rosenberg Self Esteem Scale
- Intelligence Tests
- These indicate behaviours that can be inferred to predict future behaviour
Scientific Standards of Self Report Measures
- Unidimensional - Measures one thing
- Factor Analysis - Analyses relationships between factors and variables
- Reliability and validity - Gives us stable picture of what is measured
Measures Traits not States
e.g. self esteem is a stable characteristic
Should be the same if measured over time.
State: Mood like sadness could fluctuate across time and might be different if measured over time.
Test Retest Reliability
Measuring a construct trait at different intervals in time
Reliable if the traits are consistent over time
Internal Consistancy Reliability
When we measure a construct over time and it remains stable
Test Validity
- Is actually measuring what we expect it to measure
Psychometric Properties
- Factor Structure
- Reliabilty
- Validity
When we measure a construct we need to ensure that we use sound Psychometric Properties
Scale vs Questionnaire
- Questionnaire is the entire package
- Starts with title and instructions
- Contains scales that can be measured
- Needs supplies Fit
Multi Dimension Scale
Contains items that capture more than one construct
Constructs vs Indicators
- Constructs cannot be measured but we deal with them everyday e.g. Intelligence or personality
- We infer them from things we observe
- These observations can be measured and are called indicators
Reverse Coding
When rating states in a questionnaire are reversed
On a scale of one to 5 reflects different states with 5 being Strongly Agree
I am a talkative person - Disagree 1 2 3 4 5 - Agree
I am a shy person - Disagree 1 2 3 4 5 - Agree
I enjoy Social Situations - Disagree 1 2 3 4 5 - Agree
- Agreeing on Q2 would indicate extroversion so it needs to be flipped before anaylsis
Ovals and Squares
- When developing an operational definition of a questionnaire
- We use ovals to indicate theoretical construct
- We use Squares/Rectangles to indicate observable indicators
Building Blocks of Scales
- Writing Good Questions
- Can stand alone as single entities
Scales & Questionnaires as a Package
- Includes questionnaire & Information Sheet
- Information sheet comes before the questionnaire
- Information Sheet acts as informed consent to advise the participant what to expect
Ways to Administer Questionaires
- On Paper
- By phone
- Face to Face
Self Report
We ask people to give us answers about some aspect of themselves
* Attitudes
* Values
* Beliefs
* Behavious
* Personal Characteristics
* Knowledge
* Experiences
* Expectations
Psychometrics
- Branch of Psychology
- Uses statistical techniques and deals with design administration interpretation of variables
- Things like Correclation & Factor Analysis
- try to identify underlying constructs
Where to start with Psychometrics
- Start with a Theory or Construct that you want to observe
- Then design a test to match the theory
- Then use statistics to check if your measure works
Intelligence Tests
- Unusual in Scientific Discourse
- Measures of Intelligence were developed first
- Theories of intelligence evolved to fit the measures
Open Ended Question - Values & Uses
- Exploratory Research allows collection of broad range of data
- Generate new directions of study
- Provide freedom and scope to express themselves
- complex & nuanced
- Not limited to pre determined questions
- Allows subjects to use own words
- Provides different perspectives
Open Ended Questions - Drawbacks/Problems
- Time consuming to answer
- Time consuming to analyse
- Responses can be ambiguous
- Difficult to analyse
- Can be messy when subjects get off track - this can sometimes be good, leads to further discussion or research
Fixed Choice Questions - Value/Uses
- Limited to predetermined response options
- Generate quantitative data
- Suited to statistical analysis & numeric calculation
- Should not leave room for ambiguity
- Should not be too difficult to categorise
- Quick and easy to administer and analyse
- Keep respondents on topic
Fixed choice questions - Drawbacks/Problems
- Force respondents to choose from limited range of responses
- May not capture the full picture
- True attitudes may not be wholly captured
- Pre testing required to be sure responses are relevant
Types of Fixed Choice Questions
- Forced Choice
- Numerical or Likert Type
- Graphic or Semantic Differential
Fixed Choice Questions - Forced Choice
- Forces subjects to choose between discrete response options
- You can’t select both
- You can’t select neither
e.g. I-E Scale - Rotter 1966
I-E Scale
- Internal Locus of Control
- External Locus of Control
Internal Locus of Control
Tendency to feel personally responsible for behaviour consequences
e.g. I failed the exam because I didn’t study.
External Locus of Control
Tendency to give blame to external factos like luck, fate or others for consequences of behaviour
e.g I failed the exam because the lecturer hates me
Using I-E Scales
- Rotter beleived these Locus of Controls were stable characteristics
- Also could predict how people will behave in different circumstances
- Measured this with 23 paired statements
Fixed Choice Questions - Numerical
- Used in Practical Applications
- Give two response questions True or False - Code False as zero and True as one
- The higher your score, the more positive the attitude
Numerical Questions - 5 Point Scale
- Allows us to capture more vairability than the I-E Scales
- Can add nuance to responses
- Coding is implicit but can be made explicit with numbering
- Keep track of numbering so that we know what 0 actually means
Numerical Scales - Likert Scales
- Often used in online surveys
- Subject required to rate a series of statements
- Reversing should be done with software not by hand
- Then arrive at an overall score, average and Standard Deviation
Reversing Questions in Lickert Scales
- When the numerical scoring scale runs in the opposite direction.
- good idea to reverse code items that are negatively worded so that a high value indicates the same type of response on every item.
- Ensure full measurement of an answer
- Reduces the effect of answering carelessly
- Helps correct for agreement bias
Questionnaires - Things to Consider
- How many questions is enough
- What is the mid point
- Include negative and positive worded items
- Label every response or only the anchors?
Acquiescent Responses
- Respondents’ perceptions of how they think the researcher wants them to respond
- May lead to demand effects.
- Responding favorably towards and answer because they think it is the expected answer
Central Tendency Bias
- When questions are seen all in a row, people tend to lose interest in giving out an extreme score.
- Mixing the questions up helps them appear more interesting and may reduce this problem.
- Including positively and negatively worded questions to reduce this tendency
Mid Point
- Instead of asking subjects to agree or disagree we give a middle option.
- But this can reduce variablity making differences difficult to see differences
- Some researchers consider a neutral answer as the same as no answer
Negativ/Positive worded questions
- If someone answered two questions oppositely charged we could assume they are not attending to the survey and eliminate them from the data
- However could result in different sub scales and becom es making them multi dimensional in nature
Graphic or Semantic Differential Scales
Allow people to identify where they sit between polar opposites
Can cause scoring complications so we divide the scale into segments
Response Questions Format
All fixed choice questions must be:
* Accurate
* Exhaustive
* Mutually Exclusive
* Uni-dimensional
* Variable Responses
Response Options Formats: Accurate
Does the response match the question?
e.g. answers to the questions How satisfied are you with your meal?
The option of Very Satisfied is more appropriate that Strongly Agree
Response Options Formats: Exhaustive
Ensure all the options are covered.
e.g. What brand of phone do you have? a) Apple, b) Blackberry or c) Nokia
What if someone has another brand or if they have two phones?
Response Options Formats: Exhaustive
Ensure all the options are covered.
e.g. What brand of phone do you have? a) Apple, b) Blackberry or c) Nokia
What if someone has another brand or if they have two phones? or no phone at all?
Response Options Formats: Mutually Exclusive
- Two or more events that cannot happen simultaneously.
- Each choice is incompatible with the others
Response Options Formats: Uni-Dimensional
- First options are inconsistent
- Need to be a meausre of the same thing.
Poorly Written Quesitons
- Avoid slang/jargon/abbreviations.
- Avoid vague or ambiguous wording
- Avoid double-barrelled questions.
- Avoid leading questions.
- Avoid loaded questions.
- Avoid complex grammar and negative wording. Write for your intended audience.
Ways to Address Poor Questions
- Use correct terms for items like Australian College of Applied Professionals not ACAP
- Give set of Response answers taking care not to shape the outcome
- use open response format even if behaviour is ambiguous
- Only ask about one thing at a time without double barrelling or even give a numeric scale
- Led in with filtered questions
- Design question to give answer you want
What is a Questionnaire
- Document used for colleting information
- Must be clear and professional
- Contains title, sub-headings and instructions for each section.
- Look at examples before creating your own.
What is Not a Needed in a Questionnaire
- Not an Academic Submission
- Does not adhere to formatting like Times New Roman font or double spacing
- Does not need to contain references
What Does a Questionnaire need?
- Each Scale describes what to do to get usable data e.g. circle answer or assign a number
- Demographics are set and are questions placed at the beginning or the end
- Decide on environment survey will be done i.e. quiet, at home, online
- merge and mix sub scales to avoid agreement bias or demand effect
- Finish with thanking subjects and instructions to return paper to you i.e. submit button or letter in the mail
Ways to conduct Questionnaire
- Face to Face
- Telephone
- Postal Mail
- Paper Based
- Online
Honesty/Accuracy in Self Report
Data
- Honesty: Subjects choose the socially acceptable answer, not the tru answer
- Introspective ability: The subjects may not be able to assess themselves accurately.
- Interpretation of questions: The wording have different meanings to different subjects.
- Rating scales: Yes or no can be too restrictive, but numerical scales also can give an extreme or middle response
- Response bias: Biases of previous responses relate to experience and other factors.
- Sampling bias: Are the subjects representative of the population you wish to study?
Questionnaire Conclusion
- Efficient way to gather data
- Many factors need consideration in preparation
- Data is shaped by what you ask and how you ask it.
- Use a Multi-Modal approach wherever possible
- Plus methodological triangulation
Methodological Triangluation
- Involves using more than one kind of method to study a phenomenon
- Gives confirmation of findings
- Data is more comprehensive
- Increases validity and enhances understanding of studied phenomena.
- If multiple methods of collecting date lead to the same conclusion then findings are sound
