Quantitative Flashcards

Question

Reliability and validity of measurement

Answer 1

Not to be confused with internal or external validity of a study However, reliability and validity of measurement affects internal validity of a study Measured score = “true” score (real score on the variable) + measurement error

Answer 2

Consistency or stability of a measure of behavior We expect measures to give us the same result each time Should not fluctuate much

Answer 3

Practice effects – literally more practiced 2nd time Maturation – simply that subjects change because time has passed

Answer 4

Individual takes 2 different forms of the same test Also at 2 different times

Answer 5

Generally measures whether several items that propose to measure the same general construct produce similar scores Assessed using responses at only one point in time In general, the more the number of questions, the higher the reliability 3 common measures of internal consistency: Item total Split –half reliability Cronbach’s alpha (α)

Answer 6

Correlation between an individual item and the total score without that item For example, if you had a test that had 20 items, there would be 20-item total correlations. For item 1, it would be the correlation between item 1 and the sum of the other 19 items Helpful in identifying items to remove Or in creating a short-from

Answer 7

Correlation of the total score on one half of the test, with the total score on the other half Randomly divided items Spearman-Brown split-half reliability coefficient We want > .80 for adequate reliability However, for exploratory research, a cutoff as low as .60 is not uncommon

Answer 8

How closely related a set of items are as a group How well all the items “hold together” Simply put: Average of all possible split-half reliability coefficients Expressed as a number between 0 and 1 Generally want > .80, (in practice > .70 considered ok) By far the most common measure you will see reported

Answer 9

Correlation between the observations of 2 different raters on a measure Measured by: Cohen’s Kappa By convention > .70 is considered acceptable

Answer 10

To what extent does the operational definition of a variable actually reflect the true theoretical meaning of the variable? Does the measure reflect the operational definition? Ex. Depression DSM-5 criteria BDI sxs

Answer 11

Content of the measure appears to reflect the construct being measured Very subjective Easy for a participant to “fake”

Answer 12

Extent to which a measure represents all facets of a given construct Subject matter experts may be part of the process

Answer 13

Measures how well one measure predicts an outcome for another measure

Answer 14

Predictive Validity Scores on the measure predict behavior on a criterion measured at a future time Ex: GRE -> grad school success Concurrent Validity Relationship between measure and a criterion behavior at the same time

Answer 15

Convergent Validity Scores on the measure are related to other measures of the same construct i.e., if we test two measures that are supposed to be measuring the same construct and show that they are related The scores should “converge” Ex: BDI & CES-D

Answer 16

Scores on the measure are not related to other measures that are theoretically different i.e., if we test two measures that are not supposed to be related, and show that in fact, they are unrelated The scores should “discriminate” between constructs Ex: Narcissism and Self-esteem

Answer 17

Measurement is reactive if awareness of being measured changes an individual’s behavior e.g., self-monitoring, wearing a fitbit Hawthorne effect / observer effect Productivity and working conditions

Answer 18

Ability of a test to correctly identify those with the condition (true positive rate) Proportion of people with the condition who will have a positive result

Answer 19

Ability of the test to correctly identify those without the condition (true negative rate) Proportion of people without the condition who will have a negative result

Answer 20

Categories with no numeric scales

Answer 21

Rank ordering Numeric values limited

Answer 22

Numeric properties are literal Assume equal interval between values

Answer 23

Zero indicates absence of variable measured

Answer 24

Consist of indivisible categories i.e., don’t do math on them, only frequency or percent Answers “how many” questions Usually naturally occurring groups or categories (not always)

Answer 25

Infinitely divisible into whatever units Ex: time or weight Scores that provide information about the magnitude of differences between participants in terms of the amount of some characteristic

Answer 26

A continuous variable can be converted into a categorical one Ex: level of anxiety symptoms (0-100)

Answer 27

Key to experimental method Want least ambiguous interpretations of results Manipulate IV and hold all other variables constant, either by experimental control or randomization

Answer 28

Varies along with the independent variable Cannot determine which variable is responsible for the effect Ex: exercise vs. video’s effect on mood Windows vs. window-less room

Answer 29

Reminder: Internal validity is ability to draw conclusions about causal relationships from the data Results can be attributed to the effect of the independent variable (IV) and not confounding variables More we control confounding variables, the more we strengthen internal validity

Answer 30

Obtain two equivalent groups of participants Introduce the independent variable Measure the dependent variable Assume the effect is of the IV on the DV

Answer 31

When people selected to conditions differ in an important way This is why we prefer to recruit all our participants first, and then randomize them! Basically anything other than randomization may lead to bias in some way Even when we do randomize, we usually want to make sure the groups are equivalent on important variables…

Answer 32

Pretest is given to each group Help assure that groups are equivalent at the beginning of the experiment

Answer 33

Can look at change in DV from pre to post Can assure the randomization worked i.e., we started with “equivalent groups” Compare the pre measures between the groups Especially helpful if have a small sample size Sometimes used to select participants for the experiment Like screening for people who score above a cutoff on depressive symptoms Then they get randomized to groups

Answer 34

May have started with equivalent groups, but then people start dropping out…. Drop out may be random… but what if it is due to something non-random, i.e. systematic? Like patients with “worse” symptoms dropped out Pretest allows us to see if those who dropped out were different in some way from those who remained in the study

Answer 35

Time consuming Might sensitizes participants to what is being studied Affects the way participants react to manipulation/intervention Practice effects When same pretest is used to post-test (measure DV) Disguise pretest if possible Different form Embed in other questions

Answer 36

A design that tests for effects of a pretest Half receive posttest only Other half receive pretest and posttest

Answer 37

>= 2 groups/conditions Participants participate in only one group Comparisons are made between different group/condition of participants

Answer 38

Participants experience all groups/conditions Comparisons are made within the same group of participants

Answer 39

Time Pre-post i.e. how did the group of people do from pre to post? T1 T2 T3 T4 Pre, post, 6-month follow-up, 1-year follow-up Condition All participants go through all conditions Ex. Taste test challenge Condition: 1) Pepsi, 2) Coke, 3) RC Cola Participant tastes all three.

Answer 40

Fewer participants - Each participant serves as their own “control” - Extremely sensitive to statistical differences (between-group tends to have more random error) - When you use the same participant, you automatically are controlling for a large amount of potential confounding factors: such as any demographic/historical differences

Answer 41

Order effects: Order of presenting the conditions/treatments affects the dependent variable Practice (learning) effects: Performance improves because of the practice gained from previous tasks, i.e., repeated measurement of the DV

Answer 42

All possible orders of intervention/condition are included Can help with order and practice effects (depending on measurement)

Answer 43

Fatigue effect: Performance deteriorates because the participant becomes tired, bored, or distracted from previous tasks Carryover effect: Effect of the previous treatment carry over to influence the response of the next treatment

Answer 44

Longer time interval between measures Sometimes called “washout” Helps with fatigue and potential carryover effects However longer intervals can lead to more attrition….

Answer 45

Method of assigning participants to conditions in an experiment based on a participant characteristic Goal is to achieve the same equivalency of groups Participants are grouped together into pairs based on some variable they “match” on e.g., age, gender, SES Then, within each pair, participants are randomly assigned to different treatment groups

Answer 46

All possible individuals of interest

Answer 47

The group of people you actually study that are drawn from the population

Answer 48

How you select your sample affects external validity In general, the larger the sample, the closer it is to estimating the population Sample size matters Must consider the cost/benefit of increasing sample size

Answer 49

inferences and predictions about a population based on a sample of data taken from the population in question Because a sample is typically only a part of the whole population, sample data provide only limited information about the population. This is why sampling is so important!

Answer 50

utilizes some form of random selection assures that a member of your population has equal probability of being chosen

Answer 51

Just about all other methods Also known as a “convenience sample” A sample that is not drawn randomly Usually consists of participants who are readily available to the researcher

Answer 52

Stratified sampling Used to ensure that the proportional representation of groups in the sample is the same as in the population. Example: Population of psychology grad students may be 88% female,10% male, 2% non-binary. How would you sample?

Answer 53

Systematic differences between the characteristics of a sample and a population Leads to underrepresentation of many types of people Limits the potential generalizability of results

Answer 54

The discrepancy between a sample statistic and its population parameter

Answer 55

Ex: Assignment to groups/conditions Treatment vs. control: therapy or pill 2 different signs in the bathroom Different instructions to participants

Answer 56

Might be when trying to create some psychological state (like anger, anxiety) Frequently use a confederate or accomplice Frequently uses deception Ex: Milgram obedience study Sneezing confederate

Answer 57

Maximizes the difference between groups and increases the chances that the IV will have an effect that is statistically significant Early stages of research Want to show that a relationship exists External validity? Might create situations which never naturally occur

Answer 58

Simple ask them! Used to measure aspects of human thought and behavior Attitudes, judgements, emotional states, intended behaviors, etc. Ex: How often do you exercise? How often? How long?

Answer 59

Direct observations of behavior Ex: Observe how much exercise one does FitBit – accelerometer/pedometer data the fitbit observes for you!

Answer 60

Recordings of responses of the body Ex: FitBit – Heart rate monitor data blood lactate concentration

Answer 61

Recordings of responses of the body Ex: FitBit – Heart rate monitor data blood lactate concentration

Answer 62

Maximum level is quickly reached Task too easy? Scale of 1-5 -> most responses are 4 or 5 Maybe would have seen a better spread if scale was 1-10

Answer 63

Reduces variability in data and difficult to detect any differences – just not captured by the DV measure

Answer 64

This also happens when there isn’t much room for improvement in your sample Ex: Low depression scores to start, no where to go A1C Stress management 1550 SAT score

Answer 65

Most people use “Likert-like” or “Likert-type” scales “True” Likert scale contains several items response levels are arranged horizontally and anchored with consecutive integers and verbal labels connote more-or-less evenly-spaced gradations, bivalent and symmetrical about a neutral middle

Answer 66

Participants respond/behave how they think is expected Especially problematic if they know the hypothesis Control by deception, filler items, asking about perception of purpose of research, blinding!

Answer 67

Used to control placebo effect i.e., just taking a pill makes a difference Compare placebo pill with active pill effects – needs to above and beyond placebo response

Answer 68

When specifically looking for effect of expectations

Answer 69

A nocebo response occurs when participant’s symptoms are worsened by the administration of an inert, sham, or dummy (simulator) treatment due to negative expectations of treatment or prognosis

Answer 70

Subtle biases of how experimenter interprets and records behaviors, or how they interact with participants

Answer 71

Train experimenters Run all participants at some time so experimenter’s behavior is the same Take the human out of the equation - Automate procedures print, video directions

Answer 72

Single and double blind