PSYC-523 Statistics and Research Methods

Question 1

ANOVA

Answer 1

analysis of variance; Statistical test used to determine whether significant differences exist between 2 or more conditions

statistical technique used to compare more than two random samples or groups at a time; determines whether there is a significant difference between groups but does not tell where that difference lies - must do further tests to determine; goes around need for doing multiple t-tests which would increase error significantly

Question 2

Clinical vs. Statistical significance

Answer 2

Clinical significance: is the obtained result important or meaningful?; Evidence Based Treatment; Will it be meaninigful in the real world? Looks from a therapeutic standpoint, looks at bigger picture. Clinical significance looks at symptom levels, remissions, client functioning, and quality of life. A high clinical significance suggests the post treatment symptom scores are lower than the pre treatment symptom scores

Statistical significance: The degree to which a result is not reasonably attributable to chance

is the obtained result likely to be attributable to chance factors?; Empirically Supported Treatment; Looks from an experimental standpoint, data driven
usually based on p

Question 3

Construct validity

Answer 3

measures qualities or constructs; degree to which a test measures what it claims to be measuring; assessed by evidence of it being convergent (correlated highly with tests measuring the same thing) or divergent (not correlated with tests measuring different constructs)

ex. if a researcher develops a new questionnaire to evaluate respondents’ levels of aggression, the construct validity of the instrument would be the extent to which it actually assesses aggression as opposed to assertiveness, social dominance, and so forth.

Question 4

Content validity

Answer 4

The degree to which the test measures content appropriate to the subject

the extent to which the items on a test are fairly representative of the entire domain the test seeks to measure.

For example, if a test is designed to survey math skills at a third-grade level, content validity indicates how well it represents the range of arithmetic operations possible at that level.

Question 5

Correlation vs. Causation

Answer 5

Correlation refers to the relationship between two variables (not causation)

Correlation: refers to whether 2 variables are related or the extent to which change in 1 variable corresponds with change in the 2nd variable. this also can include whether association is greater than expected by chance strength of association.
These changes may be due to a 3rd variable or external influences on the other 2 variables. uses data/variables that currently exist - NOT manipulated;
used to determine:
-whether 2 variables co-vary-association - weak, moderate, strong

Causation: relationship between cause and effect; causality usually determined via controlled study - when you can isolate variables you want to examine and control for extraneous variables.

Correlation does not imply causation

Question 6

Correlational research

Answer 6

Correlational research refers to a non-experimental research method which studies the relationship between two variables with the help of statistical analysis. Does not in any way establish causal factors

The goal is to describe the strength of the relationship between two or more events or characteristics. the more strongly the two events are correlated the more effectively we can predict one event from the other. Yields a correlation coefficient to show the relationship statistically; from -1 to +1. negative indicates inverse relationship. The closer the coefficient is to 1, the stronger the relationship is.

Ex: intelligence and height - people with higher IQs are taller; diet and weight

Question 7

Cross-sectional design

Answer 7

Research design that examines a group of similar people who differ in one key characteristic

a research design in which individuals, typically of different ages or developmental levels, are compared at a single point in time. An example is a study that involves a direct comparison of 5-year-olds with 8-year-olds.

Study several groups of differing ages (or other variable of interest), but similar characteristics (SES, gender, ethnicity, education) at one point in time; very common; Groups can be compared across a variety of dependent variables.
Advantages are the researcher does not have to wait for the individuals to grow up or become older, large amounts of data in a short amount of time, and are inexpensive.
Drawbacks include not giving information about how individuals change or about the stability of their characteristics. cannot disentangle cohort and developmental changes. less complete picture of individual development. used to gather information only - cannot infer causation (because it is just a snapshot); quasi-experimental design (participants are not selected randomly - selected based on age)

Ex: look at developmental differences between 5, 10, and 15-year-olds

Question 8

Dependent t-test

Answer 8

The dependent t-test (also called the paired t-test or paired-samples t-test)

compares the means of two related groups to determine whether there is a statistically significant difference between these means.
statistical analysis used when we want to know whether there is a difference between populations when the data are “linked” or “dependent”; also called the paired t-test or paired-samples t-test;

compares the means of two related groups/samples to detect whether there are any statistically significant differences between these means

Question 9

Descriptive vs. Inferential

Answer 9

Descriptive statistics describe what is going on in a population or data set. Inferential statistics, by contrast, allow scientists to take findings from a sample group and generalize them to a larger population.

Descriptive statistics summarize the characteristics of a data set. Inferential statistics allow you to test a hypothesis or assess whether your data is generalizable to the broader population.

Descriptive: statistics that describe; do not explain why; can describe relationships, but not causality

Ex: Are women waiting until later to get married? Has the average ago of marriage for women increased?

Mean, median, mode, range, variance…

Inferential:
examine the relationships between variables within a sample and then make generalizations or predictions about how those variables will relate to a larger population.

When conducting research using inferential statistics, scientists conduct a test of significance to determine whether they can generalize their results to a larger population. Common tests of significance include the chi-square and t-test. These tell scientists the probability that the results of their analysis of the sample are representative of the population as a whole.

include linear regression analyses, logistic regression analyses, ANOVA, correlation analyses

Question 10

Double-blind study

Answer 10

A type of clinical trial in which neither the participants nor the researcher knows which treatment or intervention participants are receiving until the clinical trial is over.
Guards against experimenter bias and placebo effects

Question 11

Ecological validity

Answer 11

The extent to which a study is realistic or representative of real life

a measure of how test performance predicts behaviours in real-world settings.; generalized to real-life settings/situations; more control in experiment, typically less ecological validity - conditions are different than those found in real-life setting

Question 12

Effect size

Answer 12

The magnitude of a relationship between variables

Effect size is a quantitative measure of the magnitude of the experimental effect. The larger the effect size the stronger the relationship between two variables.

For example, we might want to know the effect of a therapy on treating depression. The effect size value will show us if the therapy as had a small, medium or large effect on depression.

Question 13

Experimental research

Answer 13

Research designed to discover causal relationships between various factors (manipulation)

Experimental research is a study that strictly adheres to a scientific research design. It includes a hypothesis, a variable that can be manipulated by the researcher, and variables that can be measured, calculated and compared. Most importantly, experimental research is completed in a controlled environment. The researcher collects data and results will either support or reject the hypothesis.

Includes independent and dependent variables, pretesting and post testing, and experimental and control groups.

Question 14

Hypothesis

Answer 14

A testable prediction

A precise, testable statement of what the researchers predict will be the outcome of the study. In the scientific method, the hypothesis is constructed before any applicable research has been done, apart from a basic background review.

Question 15

Independent t-test

Answer 15

Statistical procedure which determines differences between the means of independent variables

statistical technique that involves selecting two random samples; compares the means of two independent groups in order to determine whether there is statistical evidence that the associated population means are significantly different

an inferential statistical test that determines whether there is a statistically significant difference between the means in two unrelated groups.

Question 16

Internal consistency

Answer 16

A measure of reliability; the degree to which a test yields similar scores across its different parts, such as on odd versus even items

The higher the internal consistency, the more confident you can be that your survey is reliable.

measures whether several items that propose to measure the same general construct produce similar scores; usually measured with Cronbach’s alpha

Question 17

Internal validity

Answer 17

The degree to which the effects observed in an experiment are due to the relationship between variables (not outside factors)

Internal validity is the extent to which you can be confident that a cause-and-effect relationship established in a study cannot be explained by other factors. In other words, can you reasonably draw a causal link between your treatment and the response in an experiment?

whether an experimental treatment was the only cause of changes in a dependent variable;
control for confounding variables can increase internal validity - random selection of participants

Question 18

Interrater reliability

Answer 18

The extent to which different raters agree on observations

used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon;

the extent to which two or more raters (or observers, coders, examiners) agree.

Question 19

Measures of central tendency

Answer 19

provides statistical description of the center of the distribution; three measures of central tendency are mean, median, and mode. Each of these measures describes a different indication of the typical or central value in the distribution.

Question 20

Measures of variability

Answer 20

The ways of measuring score distribution (range, variance, standard deviation)

provide descriptive information about the dispersion of scores within data. Measures of variability provide summary statistics to understand the variety of scores in relation to the midpoint of the data.

spread of the distribution - standard deviation, range, and variance; gives information on data set to perform statistical analysis

Question 21

Nominal/Ordinal/Interval/Ratio measurements

Answer 21

Nominal: scale in which labels are assigned for identification but cannot be counted or categorical data where there may be more than two categories

Ex: male/female; Republican, Democrat, Independent

Ordinal: data (numbers) that indicate order only, but may not indicate what measurement was used to determine the order or the magnitude of the differences within the order

Ex. How satisfied are you with our services?

1- Very Unsatisfied
2- Unsatisfied
3- Neural
4- Satisfied
5- Very Satisfied

Interval: true score data where you know the score a person made and you can tell the actual distance between individuals based on their respective scores, but the measure used to generate the score has not true zero

Ex: most psychological measures, IQ, SAT, GRE

Ratio: same as interval but also with an absolute zero

Ex. height, weight

Question 22

Normal curve

Answer 22

aka normal distribution; refers to “bell-shaped” curve formed on histogram when data has a normal distribution; symmetry; most data focused toward mean/average with less toward extremes; random sampling tends to follow normal curve

Ex: frequency distribution of peoples’ height; most people would be of average height with extremes occurring on either side

Question 23

Probability

Answer 23

likelihood that something will happen; based on hard data (unlike chance); p is between 0 and 1, 0 indicates impossibility of the event and 1 indicates certainty

Ex: utilized with suicide assessments; scale; if person falls within lower limits or lower end of the scale, the probability of them committing suicide is lower

Question 24

Parametric vs. nonparametric statistical analyses

Answer 24

Parametric relies on assumptions about the shape of the distribution

Nonparametric do not rely on this assumption

Parametric: statistics based on symmetrical distributions or distributions that come close to symmetry; focus on 1 variable or relationship; robust procedures with negligible amounts of error; random sampling.
based on assumptions about the distribution of population from which the sample was taken.

Nonparametric: data that do not form sufficient symmetry in the distribution; skewed data.
not based on assumptions

Question 25

Quasi-experimental research

Answer 25

A type of research design in which the independent variable has already been selected and cannot be manipulated

experimental design that involves selecting groups, upon which a variable is tested, without any random pre-selection processes; the researcher controls the assignment to the treatment condition; lacks the element of random assignment to treatment or control

Question 26

Random sampling

Answer 26

a simple random sample is a subset of individuals chosen from a larger set in which a subset of individuals are chosen randomly, all with the same probability.

process of selecting a sample randomly from the population to better represent the entire group as a whole; controls for extraneous and confounding variables

Question 27

Sample vs. Population

Answer 27

Population: entire subject group

Sample: subset of population

Question 28

Scientific methodology

Answer 28

Empirically based systematic way of conducting research

The scientific method is a standardized way of making observations, gathering data, forming theories, testing predictions, and interpreting results.

basic guidelines/empirical process that support all legitimate research; it starts primarily three tenets/assumptions, (1) nature is lawful - not random (2) laws of nature can be identified and understood through research (3) behavior is deterministic - influenced in a cause-effect manner,

then goes on to describe the relevant characteristics of data and researchers;
observe - question - formulate hypothesis - develop predictions - gather data - refine/alter/expand/reject hypothesis - develop general theories; controlling for errors/variables for internal validity; reach a consensus; creating and answering questions about nature

Question 29

Standard error of estimate

Answer 29

Standard deviation in a regression indicating the amount that the actual scores differ from the predicted scores

is a measure of the accuracy of predictions made with a regression line (line of best fit);

used to calculate confidence interval to give range of prediction; the smaller standard error of estimate, the more accurate the prediction

Question 30

Standard error of measurement

Answer 30

Hypothetical estimate of variation in scores if testing were repeated

SEM or Sm; provides estimate of how much individual’s score would be expected to change of re-testing with same/equivalent form of test; reminds there is error;
The SEM is a standard deviation of a set of observations for the same test - in practice the standard deviation of observed score and the reliability of the test are used to estimate SEM.
the larger the SEM the less certain we can be about the accuracy with which an attribute measured. creates confidence interval (range of where true score would fall).
MAJOR assumption - errors are obtained randomly and normally distributed;
estimates how repeated measures of a person on the same instrument tend to be distributed around his or her “true” score. The true score is always an unknown because no measure can be constructed that provides a perfect reflection of the true score. Error can occur systematically due to instrument in construction or the user may have made an error in use of the instrument

Question 31

Standard error of the difference (2 sample t-test)

Answer 31

Estimate of standard deviation of the difference between the means of independent samples in a two-samples experiment

in regression the difference between the means of two randomly selected samples;
the error between groups’ means; first step in t-test; differences in sample distribution

Question 32

Standard error of the mean (single sample z-test)

Answer 32

How far the sample mean of the data is likely to be from the true population mean

Question 33

Standard error of the mean, estimated (single sample t-test)

Answer 33

How far the sample mean of the data is likely to be from the true population mean; used when population SD is not known

Question 34

Standardization sample

Answer 34

Representative group of people who take the test and establish the norms

representative sample of typical population; norm group. A comparison group consisting of individuals who have been administered a test under standard conditions - that is, with the instructions, format, and general procedures outlined in the test manual for administering the test. Creates norms for testing. May not be representative, but should be. frame of reference for test score interpretation for norm-referenced test

Ex: need sample of cadets from The Citadel - the sample of cadets must be representative (reflect or mimic) the entire core of cadets

Question 35

Statistical significance

Answer 35

The degree to which a result is not reasonably attributable to chance

Statistical significance is a determination that a relationship between two or more variables is caused by something other than chance.

is the obtained result likely to be attributable to chance factors?; experimental standpoint, data driven; usually based on p

Question 36

Type I and Type II error

Answer 36

Type I and Type II error: in statistical hypothesis testing…mutually exclusive; helps - random sampling and increased sample size

Type I error: the incorrect rejection of a true null hypothesis (a “false positive”, i.e., accepting a false hypothesis as correct); detecting an effect that is not present

Type II error: the failure to reject a false null hypothesis (a “false negative”, i.e., rejecting a true hypothesis as incorrect); failing to detect an effect that is present

Question 37

Regression

Answer 37

Statistical technique in which one variable is used to predict or estimate the score of another variable

analysis one step beyond correlation; essentially means prediction and is based on correlated data; statistical technique developed by Sir Francis Galton;
reasoning is if two variables are significantly correlated, then we should be able to predict one from another; can also help identify factors important in a particular prediction; “line of best fit”; use relationship to predict behaviors; strength of the relationship determines the amount of error in predictions; more significant/strong the correlation = better prediction