L5: Scale Construction

Question 1

What is assessment and why is it important?

Answer 1

The gathering and integration of data to evaluate a person’s behavior, abilities, and other characteristics,
One of the primary goals as a psychologist
Need to think about policies at an individual level & population
Needs and risks are interchangeable
Need to assess risk and needs in individuals and populations

Question 2

What did Landa et al., (2012) show by evaluating developmental trajectories of at risk infants?

Answer 2

Aimed to evaluate what the developmental trajectories of these at risk groups
Followed infants from birth
Used the Mullen scales are which are administered by a professional
The scales cover cognitive, motor & language development domains
Appropriate for a wide age range
Two groups decreased in developmental trajectories
Modelling allows researchers to identify & evaluate trajectories for at risk groups

Question 3

What are the Mullen scales?

Answer 3

Mullen Scales of Early Learning is a developmentally integrated system that assesses language, motor, and perceptual abilities, measures cognitive ability and motor development quickly and reliably

Question 4

What are features of assessment?

Answer 4

Including a range of content
Standardization
Flexibility e.g., Infants may not always perform the way you want them to
Concurrent validity
Predictive validity

Question 5

What is predictive validity?

Answer 5

Predictive validity refers to the ability of a test or other measurement to predict a future outcome

Question 6

What is concurrent validity?

Answer 6

Concurrent validity shows you the extent of the agreement between two measures or assessments taken at the same time
It compares a new assessment with one that has already been tested and proven to be valid
Subtype of criterion validity

Question 7

What is the Neonatal Behavioral Assessment (Brazelton, 1973)

Answer 7

Identifies babies at risk
Hold baby up to see if they do the stepping reflex

Question 8

What types of assessments are used in research?

Answer 8

Interviews
Observations’
Checklists
Inventories
(All rely on indirect observations from parents, teachers & children themselves)

Question 9

What are the benefits of parent report measures?

Answer 9

Parents are always with their kids in a range of contexts

Question 10

What are drawbacks of parent report measures?

Answer 10

Bias towards their children
Systematic error based on bias
May be unreliable observers

Question 11

Was there a correlation between teachers and parents of Murphy’s study (1999)?

Answer 11

Measured emotionality & regulation
Correlation between teacher & parent ratings

Question 12

What is shared method variance?

Answer 12

Variance is due to the tool rather than what the tool is set out to measure
This can happen when using 1 data tool for everything
This may have occurred in Murphy’s study as exclusive reliance of parent-report can cause this to occur

Question 13

What is a multi-informant approach?

Answer 13

Different informants give different reports
Able to build a bigger model
E.g., Parents, teachers
Informants may not always agree

Question 13

How did Murphy’s study address the issues of shared method variance?

Answer 13

By using multiple informants - teacher & parent reports

Question 14

How did multiple informants influence Murphy’s (1999) results?

Answer 14

Correspondence 0.2 to 0.6
Correspondence varied as a function of context, child age, observability,

Question 15

What should precede scale construction?

Answer 15

Defining other existing measures
Wherever possible we want to use existing measures

Question 16

What are the fundamental issues when constructing scales?

Answer 16

Are you measuring what you’re setting out to measure?
Limitations of your scale
Do relations between your question and/or variable reflect another causal factor?

Question 17

What is reliability?

Answer 17

The extent a measure is consistent and repeatable
Scale developers prefer assessing reliability as it’s easier to assess than validity

Question 18

What is validity?

Answer 18

The extent to which a method measures what it is intended to measure

Question 19

What is test-retest reliability? (Assessing reliability)

Answer 19

Administering the same test multiple times to a group of individuals to test the reliability (correlate those two scores to determine reliability)

Question 20

How does criterion validity assess validity?

Answer 20

By calculating the correlation between results of your measurement and the results of a criterion measurement

Question 21

What are the 7 item “dirty trick” Clifton (2000) claimed researchers use?

Answer 21

1. Administering items in unidimensional blocks
2. Administering items in a fixed order
3. Administering no or few opposite scored items
4. Using items w/ diverse means
5. Deleting items w/ diverse means
6. Deleting all items w/ less related aspect of the phenomenon (items that don’t cohere)
7. Ignoring response bias

Question 22

What are the eight steps in scale development?

Answer 22

1. Determine what you want to study
2. Generate item pool
3. Determine the format
4. Review the item pool
5. Consider inclusion of validation items
6. Administer pilot scale
7. Evaluate items
8. Produce final score

Question 23

What does the Baby Scale Questionnaire measure? (Winstanely & Gattis., 2013)

Answer 23

Interested in measuring parent principles during infancy
Measures of infant care
Infant feeding styles questionnaire: measured parent’s beliefs and views about feeding their child
For this study: aimed to find out how parent’s beliefs about caring for their baby would effect the outcome for infants cognitive development later in life

Question 24

Why is it important to define what you want to study?

Answer 24

Scale construction requires clear formulation of constructs and corresponding measurement tool

Question 25

How did Gattis., (2013) use the BCQ to define what they wanted to study?

Answer 25

Sample: Infants born pre-term (early risk poses challenges for caregiving)
Measuring a culmination of the parent and child

Question 26

What should you consider when generating your item pool for your scale?

Answer 26

Clearly defined constructs
Large pool so you are able to drop items (redundancy)
Choose items strongly related to our construct
Include both negative & positive items so researchers can evaluate bias
Negative items express exactly the opposite of what positive items express

Question 27

What type of response methods can you use?

Answer 27

Likert scales: 0-5 scale, ordinal data (difficult for children to use

Visual Analog Scale: Allows ppts to answer on a scaled line; Offers greater accuracy of measurement of continuous constructs

Question 28

What is a cognitive interview?

Answer 28

Valuable tool
Method for evaluating surveys
Conduct concurrent think aloud procedures or may be retrospective report
Allow researchers to examine the interpretation of survey
Especially recommended when the study uses self-report

Question 29

What did Chambers et al., 2002 find with the use of Likert scales and children?

Answer 29

Limited data on response formats
Found giving out Likert scales to young children is not a good idea
Subjective vs. Objective terms in the questionnaire
Ratings should not fluctuate but there’s higher response rates for social subjective categories
Is it fair to ask a child to make these judgements

Question 30

What is a factor analysis?

Answer 30

An analytic tool which can tell us about the properties of a scale
By collecting data from a pilot scale in and inputting it we can determine how many factors or variables underlie a set of items & condense information
We can also determine the meaning of factors or latent variables (variable that can’t be observed)
It’s a way of condensing information and a way of making judgements of what to include in a final scale

Question 31

What is a PCA analysis?

Answer 31

Statistical procedure that allows you to summarize the information content in large data tables by means of a smaller set of “summary indices” that can be more easily visualized and analyzed

PCA analysis is good for strong correlations it doesn’t do well to reduce weak correlations
Used this for the Baby Care Questionnaire to identify if it measures what it’s said to measure