PSYC - Ch 3 + 6.2

Question 1

Why measure?

Answer 1

comparison
classification
prediction
program evaluation
decision-making
diagnosis

Question 2

Variables can be

Answer 2

Directly observable or inferred states

Question 3

Constructs

Answer 3

A hypothetical mechanism that helps explain and predict behaviour.

External stimuli -> Construct -> External behaviours

Presumed unobservable internal mechanisms that account for externally observed behaviour.

Rewards -> Motivation -> Performance
Exam -> Anxiety -> Affects behaviour

The construct cannot be studied directly, but the stimuli and behaviours which are influenced by the construct can be studied.

Change in environment leads to observable behaviours

ie: anxiety, self-esteem, motivation, aggression, intelligence.

Brings from Abstract to Concrete
Unobservable (stress) to observable (HR, persipration,)

A variable that is a hypothetical entity created from theory and speculation. They are influenced by external stimuli and influence external behaviour

Question 4

Step 3 of the Research Process

Answer 4

• determining a method for defining and measuring the variables that are being studied

Question 5

Operational definition

Answer 5

Precise description of what you will measure, how you will measure it, and when you will measure it (testing protocol). Defines how you will measure the construct. (Ie. sores on an anxiety test)

Defines the “operations” allowing us to link unobservable with observable

Converts abstract to concrete/measured

Needs to be - clear + precise - allows for replication

EX: the operational definition of anxiety could be in terms of a test score, withdrawal from a situation, or activation of the sympathetic nervous system.

ODs must be VALID and RELIABLE

Question 6

Validity

Answer 6

Accuracy of the measure
Will the test measure what it is supposed to measure?

Question 7

Reliability

Answer 7

Consistency of the measure over repeated applications in same conditions. (Reproducability)

Measurement always varries a little but should be as little as possible.

(Dots close to each other or close to the line on a line graph)

Question 8

Types of Validity - in regards to a measurement

Answer 8

Face validity
Predictive validity
Concurrent validity
Construct validity
Convergent & divergent

Question 9

Types of Validity - in regards to the study as a whole

Answer 9

Internal validity
External validity

Question 10

Face Validity

Answer 10

Extent to which a measurement appears to be a plausible measure of the variable at first glance.

Does it make sense? Would others agree?

Ex: Measuring anxiety through observation at a party or through # of toes.

Question 11

Predictive Validity

Answer 11

The strength of the relationship between 2 variables (correlation)

Can you use one to predict the other?

Intelligence - GPA
Neuroticism - Stress

Question 12

Concurrent Validity

Answer 12

Extent to which a measure relates to existing measures of the same thing (construct).

The new (yours) and old test (IQ test) have to produce similar results/have a high correlation or else the measure may have low validity

Question 13

Construct Validity

Answer 13

Extent to which scores obtained from a measure behave exactly the same as the variable itself (e.g., aggression & temperature).

The extent to which your test or measure accurately assesses what it’s supposed to

It is based on many studies that use the same measurement procedure. It grows as each new study contributes more evidence (same measurement with similar results).

Use multiple measures in your study, to help establish construct validity.

Question 14

Convergent & Divergent Validity

Answer 14

The scores of some of the measure will converge - measuring similar or related constructs

The scores of some of the measures will diverge - measuring unrelated constructs

Ex: Aggression is children - is it high energy or aggression

Measure of aggression should converge (correlate highly).

Behavioral measure of energy and the above measures should diverge (have a low correlation).

Question 15

Internal Validity

Answer 15

Can you safely say the changes in X have caused the changes in Y.

Depends on the control of other variables/confounding variables.

4 week yoga retreat decreases stress. Was it the yoga? The lack of work? The sun and beach?

Question 16

External Validity

Answer 16

Extent to which your results can generalize to other settings and populations. (Generalizable)

If effects remain with different groups = good external validity

Question 17

Types of Reliability

Answer 17

Test-retest reliability
Inter-rater reliability
Split-half reliability

Question 18

Test-Retest Reliability

Answer 18

Repeat same measurement twice with the same individuals and calculate the correlation between scores - reliability coefficient

If the scores from Time 1 and Time 2 have a high correlation you have test-retest reliability.

Question 19

Inter-Rater Reliability

Answer 19

Compare the scores from the two raters and calculate correlation.

If results from rater 1 are similar to rater 2 - you have inter-rater reliability.

Question 20

Split-Half Reliability

Answer 20

Typically, for clinical scales and questionnaires.

Take the scores from half the items and correlate them with the scores
from the remaining half - they should correlate highly.

Good internal consistency if high correlation.

Question 21

Measurement Error

Answer 21

Observed scores may not be a true reflection of the variable or construct being measured - There is always some degree of error present.

Observed score = measurement + error

Question 22

Sources of Error

Answer 22

The participant - mood, motivation, fatigue, health, memory, knowledge, ability

The instrumentation / apparatus - sensitivity, clarity of instructions, appropriateness, length, vocabulary

The testing environment. - comfort (heat), presence of others (social facilitation), distractions (noise, interruptions)

Scoring guidelines - clarity, complexity, experience requires, individual differences

Question 23

Reducing Error

Answer 23

Try to minimize the effects of possible confounds.

STANDARDIZATION

Question 24

Standardization

Answer 24

Participants - what will your inclusion and exclusion criteria will be?
ie: age, gender, educ level, health, ethnicity

Test protocol - no variation, must be consistent
ie: instructions, treatment of participants, order of tests

Environment - best environment conducive to testing and replicate is as best possible
ie: time of day/year, temperature, noise level

Scoring procedure - be as objective as possible, marking criteria should be as clear as possible for both participants and rater.
ie: allow practice rounds for both rater and participants

Question 25

Reliability coefficient

Answer 25

It reflects the degree to which the measurement is free of error variance.

You can never completely eliminate error, therefore you must account for it.

Reliability coefficient - ratio of true score variance to observed score variance

r = Score true/Score observed
= (score observed - score error)/score observed

Question 26

Relationship Between Validity & Reliability

Answer 26

A measurement procedure must be reliable (consistent) in order to be valid.

A measurement procedure can be reliable & not necessarily valid.

Can be reliable but not valid.
Cannot be valid if not reliable.

Question 27

Types of Measurement

Answer 27

Quantitative
Qualitative

Question 28

Quantitative Measures

Answer 28

Assigning a numerical value to a variable

Nominal Scale
Ordinal Scale
Interval Scale
Ratio Scale

Question 29

Nominal Scale

Answer 29

Used fo categorization
No inherent order

Gender, Education

Question 30

Ordinal Scale

Answer 30

Used for ranking
Has order

Height - short/tall
Strength - weak/strong

Question 31

Interval scale

Answer 31

Has order + magnitude + equal intervals between values but NO TRUE ZERO

Temperature

Question 32

Ratio Scale

Answer 32

Has order + magnitude + equal intervals between values WITH A TRUE ZERO

Distance, length, reaction time, HR

Question 33

Modalities of Measurement

Answer 33

Self-report - direct, subjective, social desirability bias (not honest)

Physiological - objective, invasive, costly, time consuming

Behavioural - interpretation, clusters better

Question 34

Multiple measures

Answer 34

Good - more confidence in the validity

Bad - can require complex statistical procedures & interpretation can be challenging.

Question 35

Sensitivity & Range Effects

Answer 35

Range effect - measurement is not sensitive enough to detect a difference.

Ceiling effect - clustering of scores at high end of scale; little possibility of increases in scores.

Floor effect - clustering of scores at low end of scale; little possibility of decreases in scores.

Question 36

Artifacts

Answer 36

An external factor introduced to the study accidentally that may influence or distort measurements.

Question 37

Experimenter bias

Answer 37

Measurements influenced by experimenter’s expectations regarding the outcome of the study.

Question 38

Participant reactivity

Answer 38

Participants changing their natural behaviour because they are participating in a research study.

Good subject role: support the hypothesis.

Negativistic subject role: acts contrary to the hypothesis (sabotage).

Apprehensive subject role: act/answer in socially desirable manner (fake good).

Faithful subject role: follow instructions to the letter (highly desirable).
*You want these subjects

Question 39

How to limit Experimenter Bias

Answer 39

Standardize or automate the experiment

Single-blind - the researcher is not aware of the expected results.

Double-blind - neither the participant nor the researcher know the expected results.

Question 40

How to limit Participant Reactivity

Answer 40

Disguise or conceal the measurement process (low face validity, deception, blind)

Reassure participants (performance/responses are confidential and anonymous)

Question 41

Demand Characteristics

Answer 41

Any cues or feature of the experiment that:

• suggest the purpose and hypothesis of the study.
• influence the participants to respond or behave in a certain way.

Can lead to reactivity, whereby participants modify their natural behavior knowing they are in a study.