MEASuREMENT IN PSYCHOLOGY

Question 1

VARIABLES
A variable is anything that varies
Variables are observable or hypothetical events that can change and whose changes can be measured in some way (height, time, the political party people vote for, feelings towards your partner or parent, extroversion, attitude towards vandals, anxiety)

Question 2

The value of a variable is not always a number: nationality, profession…

Try to write down your own definition of:
Intelligence
Anxiety
Superstition
Give some examples of people displaying these characteristics.

Question 3

The DEPENDENT VARIABLE (DV) is the variable in the study whose changes depend on the manipulation of the independent variables.
We do not know the values of the DV until after we have manipulated the IV.

Answer 3

The INDEPENDENT VARIABLE (IV) is the variable which is manipulated by the experimenter.
We know the values of the IV before we start the experiment.
Independent variable are defined in terms of LEVELS.

Question 4

2. PSYCHOLOGICAL CONSTRUCTS
   Hypothetical constructs
   Concepts that are not directly observable

Can you observe anxiety?
It is reference to an inner state that is assumed to play its part among all the effects on human behavior
Constructs can have relationship with observable behavious

Question 5

2. PSYCHOLOGICAL CONSTRUCTS
   Hypothetical constructs
   Concepts that are not directly observable

Can you observe anxiety?
It is reference to an inner state that is assumed to play its part among all the effects on human behavior
Constructs can have relationship with observable behavious

Question 6

Operational definition of psychological constructs

Constructs should be carefully explained.
The variables that we use in psychological research are observable measures of often unobservable constructs.
Our measurement must be precise and clear.
An operational definition gives us a more or less valid method for measuring some part of a hypothetical construct.

Discuss with a classmate the terms shown below. How could any of these be measured?

-Identity
-Reinforcement
-Attitude
-Instinct
-Unconscious
-Attention
-Egocentricity
-Neuroticism
-Conformity
-Conscience

Question 7

An operational definition of construct X gives us the set of activities required to measure X.

Answer 7

Depression - Construct
Clinical interview, depression inventory, teachers observations - operationalization

Question 8

Conceptual variable: Employee satisfaction Operational definitions: number of days per month that the employee shows up to work on time. Rating of job satisfaction from 1 to 9
“”: Aggresion: number of presser of a button that administers shock to another student “”: time taken to honk the horn at the car ahead after a stop-light turns green
“”: deppression: number of words used in a creative story “”: number of appointments with a psychotherapist
“”: Decision-making skills “”: number of people correctly solving a group performance task. Speed at which a task is solved.

Question 9

Scales of measurement

Nominal (nombres únicamente): categories only
Ordinal (ya empiezan a tener un orden, pero éste no está concretado): numerals represent a rank order, distance between subsequent numerals muy not be equal
Interval (el intervalo existente entre las posiciones ya está definido): subsequent numerals (las posiciones posteriores) represent equal distances, differences but no natural zero point
Ratio (ya hay un cero, que es una medida del círculo): numerals represent equal distances, differences and a natural zero point

Question 10

Measurement errors

Measurement errors may affect the reproducibility (reproducibilidad) of outcomes across studies
Two types: systematic and random
Systematic measurement error: systematic error is an error which, in the course of a number of measurements carried out under the same conditions of a given value and quantity, either remains constant in absolute value and sign, or varies according to definite law changing conditions. An error is considered systematic if it consistently changes in the same direction. For example, this could happen with blood pressure measurement i, just before the measurements were to be made, something always or often caused the blood pressure to go up.

Systematic measurement errors are known as bias, which function as extraneous (of external origin) variables
Random errors do not contribute to systematic differences between groups

Question 11

In psychological research, measurement errors refer to the discrepancies between the actual value of the variable being measured and the value obtained through a measurement tool or procedure.

Answer 11

Random errors (coding errors, participants inattention to and misperception of question, etc)
Other conceptual variables (self-esteem, mood, self promotion, etc) Systematic errors

Question 12

Scores on a measured variable, such as a likert scale measure of anxiety, will be caused not only by the conceptual variable of interest (anxiety), but also by random measurement error as well as other conceptual variables that are unrelated to anxiety. Reliability is increased to the extent that random error has been eliminated as a cause of the measured variable. Construct validity is increased to the extent that the influencie of systematic error has been eliminated

Question 13

Reliability (confiabilidad) refers to how consistent the results of a measure are

Answer 13

Consistency of measurement: if you repeat a study or a measurement, you should get the same results.

Test-restest: refers to the stability of a measure over time. Ej., questionnaires should produce the same results when re-tested on the same people at different times (so long as nothing significant has happened to them between testing) refers to the stability of a measure over time
Interrater: examines the consistency between different individuals (raters) who are evaluating or observing the same behavior or phenomenon. Ej. different observers counting aggressive acts made by children should come up with similar tallies
Internal: looks at the consistency within the measurement itself. It ensures that all parts of a test or survey measure the same concept and produce similar results. Ej.: answers in a questionnaire should be consistent.

Reliability: do you get consistent scores every time?
Interrater reliability: two coders ratings of a set of targets are consistent with each other.
Test-retest reliability: people get consistent scores every time they take the test
Internal reliability: people give consistent scores on every item of a questionnaire

Question 14

The test-retest statistic is used to assess the realiability or stability of a measurement tool over time. It determines whether a test produces similar results when administered to the same group of people at two different points in time

Answer 14

Interrater reliability
Consistent scores no matter who is rating

Internal reliability
Consistent scores no matter how you ask

Question 15

Internal reliability
Internal reliability: all parts of a test or survey measure the same concept and produce similar results (not to be confused with internal validity!)
The extent (proporción) to which multiple measures, or items, are all answered the same by the same set of people.
Relevant for measures that use more than one item to get at the same construct

Question 16

VALIDITY
Are we measuring what we are intended to measure?
Internal validity
External validity
Construct validity
External validity
Statistical conclusion validity

Question 17

Internal validity
Did the manipulation of the independent variable cause the observed changes in the dependent variable?
Whether there really is a casual link between the manipulation of the IV and the DV.

Answer 17

Threats:

Selection: two experimental groups have systematically different kinds of participants in them
Attrition: participant drop-out rate which could be different in each condition
Maturation: physiological processes occurring within the participants that could account for any changes in their behavior
History: extraneous events occurring during the course of the experiment that may affect the participants responses on the dependent measure
Testing: when participants are repeatedly tested, changes in test scores may be more due to practice or knowledge about the test procedure gained from earlier experiences rather than any treatment effects
Instrumentation: changes in the measurement procedures may result in differences between the comparison groups that are confused with the treatment effects
Regression to the mean: the tendency that participants who receive extreme scores when testes, tend to have less extreme scores on subsequent retesting even in the absence of any treatment effects

Question 18

Internal validity
Did the manipulation of the independent variable cause the observed changes in the dependent variable?
Whether there really is a casual link between the manipulation of the IV and the DV.

Answer 18

Threats:

Selection: two experimental groups have systematically different kinds of participants in them
Attrition: participant drop-out rate which could be different in each condition
Maturation: physiological processes occurring within the participants that could account for any changes in their behavior
History: extraneous events occurring during the course of the experiment that may affect the participants responses on the dependent measure
Testing: when participants are repeatedly tested, changes in test scores may be more due to practice or knowledge about the test procedure gained from earlier experiences rather than any treatment effects
Instrumentation: changes in the measurement procedures may result in differences between the comparison groups that are confused with the treatment effects
Regression to the mean: the tendency that participants who receive extreme scores when testes, tend to have less extreme scores on subsequent retesting even in the absence of any treatment effects

Question 19

Construct validity
Are we really measuring the construct we want to measure?
How closely our interpretation of a construct are related to the real thing.
If an affect really does occur when the independent variable is manipulated, but not for the reasons the researcher thinks; that is, the independent variable itself is not the variable causing the effect, that is a threat to construct validity:
Cofounding (factor de confusión): we make an assumption about the wrong psychological construct.
E.g.: a questionnaire intended to measure extroversion (an outcome measure) in fact only measures sociability.
E.g.: recall for concrete and for abstract words when abstract words selected are more uncommon.
E.g.: placebo effect

Answer 19

Eliminate possible confounding variables as alternative explanation are really the heart of scientific activity
a. Experimental group: therapeutic treatment (includes attention)
b. Placebo group (attention only)
c. Control group no therapy, no attention (often a wait list group)
————Measures after treatment, on outcome variables, e.g., anxiety, self-esteem, number of compulsive responses, time spent on previous obsessions etc.

If groups a and b improve similarly over c then placebo (attention only) appears to be affective

Question 20

a) Expectancy effects
Researcher’s expectancies:
Experimenters can affect participants’ responses through facial or verbal cues, and that certain participant are more likely to pick up experimenters influence than others. Pigmalion effect.
Participants expectancies:
It refers in general to the ways in which research participants interact with the researcher and research context
b)Demand characteristics
They refer only to the cues that can inform a participant about what is expected.

Question 21

Statistical conclusion validity
Inappropriate statistical procedures, or other statistical errors, may be responsible for the appearance of a difference or correlation that does not represent reality:
-We may simply have produced a ‘fluke(an unlikely chance occurrence, especially a surprising piece of luck)’ large difference between samples
We may have entered data incorrectly
We may have used the wrong king of statistical analysis

Question 22

Statistical conclusion validity Null hypothesis and alternative hypothesis
Reject the null hypothesis — the result is statistically significant — the probability of getting a result this extreme, or more extreme, by chance, if the null hypothesis is true, is less or equal than 5% — the difference is significantly larger than zero. The association is significantly stronger than zero.
Retain the null hypothesis — the result is not statistically significant — the probability of getting a result this extreme, or more extreme, by chance, if the null hypothesis is true, is greater to 5% — we cannot conclude that the difference is larger than zero. We conclude that the relationship is stronger than zero.

Answer 22

In statistical hypothesis testing, the null hypothesis of a test always predicts no effect or no relationship between variables

Question 23

Reject the null hypothesis (conclude that there is an effect) when there is really no effect in the population: type I error
Retain the null hypothesis (conclude there is not enough evidence of an effect) when there is really an effect in the population: type II error

Question 24

External validity
It asks whether the apparent effects demonstrated in an investigation can be generalized beyond the exact experimental context. In particular, can effects be generalised from:
A: The specific sample tested to other people — population validity
B: The research setting to other settings — ecological validity
C: The period of testing to other periods — historical validity

Question 25

Population validity
The extent to which an effect can be generalised from the sample studied to the population from which they were selected and also to other populations

Ecological validity
Generalisation of an effect to other settings.
Representative design

Historical validity
Whether an effect would stand the test of time and work today as it dit some years ago

Question 26

Samples and population
POPULATION: all the existing members of the group of study
SAMPLE: a representative group of the population

P: all countries of the world S: countries with published data available on birth rates since 2000

Sampling bias: weighting of a sample with an over-representation of one particular category of people
Participants variables: variations between persons acting as participants, and which are relevant to the study at hand — equivalent groups

Answer 26

Does it matter that individuals are not all the same?
It does matter, of course.
How do we ensure that the individuals we select will not introduce sampling bias?
-The simple truth is that a truly representative sample is an abstract ideal probably unachievable in practice
-The practical goal we can set ourselves is to remove as much sampling bias as possible. We need to ensure that no particular sub-groups of the member of the target population are under- or over-represented

Question 27

Equal probability selection and ransom sampling
EQUAL PROBABILITY SELECTION METHOD (‘epsem’)
‘An equal probability selection method is a procedure for producing a sample into which every case in the target population has an equal probability of being selected. There are several ways of getting as close as possible to this ideal and all of these involve some form of random selection.
RANDOM: the strict meaning of random sequencing is that no event is ever predictable from any of the preceding sequence.

Question 28

Probability-based sampling methods

Simple random sample: every case in the target population has an equal chance of selection and so does every possible combination of cases
Systematic random sample: we select every nth case from the population, where n is any number chosen started at random and picking every nth element in sucession.
Stratified sample: we sample randomly from the various strata (estratos) we have identified.
Clusters samples: instead of selecting strata, you select ‘clusters’ that represent sub-categories.
Random sample of clusters is selected from a simple random design.

Question 29

Quota sampling: in consists of obtaining people from categories, in proportion to their occurrence in the general population. Not equal chance of selection. Ex: gender and income level. Those who look most helpful. Those who look most helpful. Short time and budget is limited.
Self-selecting sample: volunteers for an experiment, observing people
Convenience sample: when participants are simply the most convenient ones available
Snowball sampling: a researcher might select several key people for an interview and these people in turn may lead the interviewer to further relevant people who could also be contacted for interview.
Critical cases: participant are selected due to a special characteristics, case studies
Focus groups and panels: panel of experts, a selection of people who are fairly representative of the general population

Question 30

Sample size
The larger the sample the less likely it is that serious sampling bias will occur so long as the selection method is truly random.
Size matters: larger sample sizes increase the ‘power’ of statistical test.
They make it more likely that we will detect effect if it exists.
Small samples may lead us to conclude that is no real difference between groups or conditions

Question 31

Statistically vs. Practical significant differences

Significant differences (i.e., reject the null hypothesis) means that differences in group means are not likely due to sampling error.
The problem is that statistically significant differences can be found even with very small differences if the sample size is large enough.
Practical (or clinical) significance asks the larger question about differences: “Are the differences between samples big enough to have real meaning?
Generally assessed with some measure of effect size:
Effect size is the actual difference you find.

Answer 31

Cohen’s d
How far apart the two means are, in standard deviations units.
The larger the effect size, the less overlap between the two experimental groups; the smaller the effect size, the more overlap.

At an effect size of 0.63, about 73% of the red group scored less than the average member of the green group.

Question 32

Variables apre phenomena whose changes can be measured.
Variables can be explanatory concepts. In psichology such concepts may not be directly observable but can be treated as hypothetical constructs, as in other sciences.
Under the conventional research paradigm (the ‘scientific model’),
Variables to be measured need precise operational definitions (the steps taken to measure the phenomenom) so that researches can communicate effectively about their findings.
The concepts of reliability of measures (consistency) and of validity (whether the instrument measures what is intended) are important aspects of a research design.

Answer 32

Statistical conclusion validity concerns whether statistical errors have been made that either lead to a false conclusion that an effect exists when it does not, or to a false conclusion that it does not exist when it does.
Internal validity refers to the issue of whether an apparent effect would have occurred anyway, without the application of the experimental ‘treatment’
Construct validity concerns generalisation from the particular operational measure of a construct to the construct itself.
A major task in experiments is to avoid confounding, which can occur through lack of control in varibles associated with the independent variables. These include: expectancies, participant reactivity effects, demand characteristics and variables systematically changing with the independent variable.
External validity concerns whether an effect generalises from the specific people, setting and time involved when it was demonstrated to the whole population, other population, other times and other settings.

Question 33

Samples should be representative of the populations ton which results may be generalised.
Equal probability selection provides representative samples if they are large enough.
Various non-random selection techniques (quota sampling, snowball sampling, critical cases, focus groups, panels) aim to provide representative, or at least useful, small samples.)
Opportunity and self-selecting samples may well be unacceptably biased.
Size of samples for experiments is a subject of much debate; large is not always best.
The concepts of effect size (estimated size of found difference) and statistical power (likelihood of conclusively demonstrating and effect if one exists) are briefly introduced.