Exam 3

Question 1

_______ _____–The population the researchers would like to sample from because this is the group they are interested in generalizing to.

Answer 1

Theoretical population

Question 2

______ _______–The population that is accessible to the researchers.•Example –A researcher wants to generalize to urban homeless older adults (i.e., 60+ years old) in the USA (i.e., theoretical population). It would be impossible to find an accurate listing of this population. Even if alist were accessible, the researcher would probably not have the resources to conduct research on a national sample. The accessible population might be homeless older adults (60+ years old) in five cities inthe USA.

Answer 2

Accessible population

Question 3

______ is the group of people who are selected to be in the study.
•The _____ is more than just the group of people who are actually in the study because:

• The researchers may not be able to contact or recruit all of the people they want to sample.
• Some participants may drop out during the study.§Some participants may not have complete information.

Answer 3

Sample

Question 4

______ ____–A listing ofthe accessible population from which the researchers will draw the sample. It is important to explain in detail how the sample will be contacted to assure representativeness.

Answer 4

Sampling frame

Question 5

_______ is the group that actually completes the study and is used in the data analyses.

Answer 5

Subsample

Question 6

Another option–The researcher could identify the area code and all three-digit prefixes within that area code and then draw a sample by randomly dialing numbers (i.e., __________). Thus, the sampling frame is not a list per se, but a procedure that isfollowed.

Answer 6

random-digit-dialing

Question 7

______ _____ Any method of sampling that utilizes some form of random selection.

Answer 7

PROBABILITY SAMPLING

Question 8

____ ____ ___
-Any form of sampling not based on random selection.
-It is most appropriate when the researcher wants to study sensitive,
relatively low frequency events. It is also useful in qualitative
research and/or when generating hypothesis or theories.

Answer 8

NON PROBABILITY SAMPLING

Question 9

______ ______

Advantages
§ Simple to accomplish
§ Easy to explain to others
§ Because it is a fair way to select a sample, it is reasonable to generalize the results from the sample back to
the population.
Disadvantages
§ Not the most statistically efficient method of sampling
§ May not get good representation of subgroups in a population

Answer 9

Probability sampling – W (advantages and disadvantages)

Question 10

____ ____ – Every individual in the population has the same chance of being selected.

Answer 10

Random selection

Question 11

_____ _____ ____ – Each member of the population has an equal and
independent chance of being selected to be part of the sample.
• Procedure: The researcher uses a table of random numbers, a computer random
number generator, or a mechanical device (e.g., bingo ball, power ball) to select the
sample.
§ Example – Researchers want to study all seniors at CSUN and UCLA (i.e., accessible population) and
generalize to all public university seniors in Los Angeles (i.e., theoretical population). First, the sampling frame is identified (computer generated list of CSUN and UCLA seniors). Next, the size of the sample is determined (e.g., 500). Next, the sample must be selected.
o A list of seniors could be printed, cut into strips, put into a hat, and mixed up. Then 500 names are
drawn from the hat. This procedure would be tedious, and the quality of the sample would depend on how thoroughly the names were mixed up and how randomly the person reached into the hat. A better option would be to put the list of seniors into an Excel spreadsheet, have Excel randomly assign numbers to the seniors, sort the column by number, and then select the first 1000 seniors.

Answer 11

Simple random sampling

Question 12

____ _____ _____ (also called proportional or quota random sampling) – The profile of the sample matches the profile of the population on some specific characteristic (e.g., gender, ethnicity, age). Specifically, the population is divided into homogeneous subgroups (i.e., stratums), and then a
simple random sample of each subgroup is taken.

o Example – The population of students at a school in the South consists of 4 primary groups: Caucasian
(80%), African-American (10%), Asian (5%), and Latino (5%). If 100 students are chosen using simple random sample method, then it is likely that around 80 Caucasians, 10 African Americans, 5 Asians, and 5 Latinos will be randomly selected (+/-). If the researchers want to compare the groups, larger numbers within the three minority groups will be needed. So, the researchers randomly select 25 from within each group (i.e., 25 Caucasians, 25 African Americans, 25 Asians, and 25 Latinos).

Answer 12

Stratified random sample

Question 13

Stratified random sample (also called……………………. )

Answer 13

proportional or quota random sampling

Question 14

________________ – Sometimes the population is disbursed across a wide geographic region; which could make it time consuming and expensive to sample participants. Hence,
the population could be divided into clusters (i.e., geographic regions), and then a
random selection of clusters could be conducted and each unit with the cluster
could be sampled.
• Example – Researchers want to survey municipal employees in Texas. Instead of
randomly selecting municipal workers across the whole state, the researchers randomly select seven municipalities in the state, and then interview each employee at the seven municipalities.

Answer 14

Cluster (area) random sampling .

Question 15

___________ – A combination of the aforementioned sampling methods. In many applied social research studies, more complex sampling methods are used.
• Two-stage sampling example – Seven municipalities are selected in Texas (cluster sampling), and then
employees at each of the seven municipalities are randomly selected to be interviewed (simple random
sampling).
• Three-stage sampling process example – Randomly select 20 school districts from a national sample of
school districts. Within selected districts, a stratified random sample of schools is taken (one high SES, one middle SES, one low SES). Within schools, a simple random sample of classes is taken, and then each student in those classes is surveyed.

Answer 15

Multi-stage sampling

Question 16

NON PROBABILITY SAMPLING
-Advantages –Low cost and time and convenient
-Disadvantages – Biased and difficult to generalize to the overall
population.

Answer 16

..

Question 17

_____ _____ is conducted with a purpose in mind (e.g., a researcher wants to study one or more predefined groups). In other words, the sample consists of subjects chosen because the researcher believes they are uniquely able to provide specific information and/or meet certain selection criteria.
• Example – Market researchers or pollsters in public setting who are stopping people and asking if they can
interview them. They examine people passing by and try to recruit those who they believe fit specific criteria
(e.g., Caucasian women who are 40-50 years of age).

Answer 17

Purposive sampling..

Question 18

_______________ refers to sampling the most frequent case or the typical case.
§ It is difficult to know what the typical or ____l case is. Hence, modal instance sampling is only reasonable for informal sampling contexts.
§ E.g., Many informal public opinion polls interview the typical voter; although the criteria to select the “typical” voter may not be accurate.

Answer 18

Modal instance sampling ..

Question 19

_________ involves surveying a group of individuals with known or demonstrable experience and expertise in some area.

§ Expert sampling can be used to gain insight into a specific topic (e.g., policy, cultural fad), to evaluate
measures to be used in a study, to provide evidence for the validity of another sampling approach (e.g., modal sampling), etc.

Answer 19

Expert sampling..

Question 20

________– Participants with certain characteristics are nonrandomly selected until a ____ is filled. One problem is deciding on the specific characteristic(s) on which the ____ will be based (e.g., gender, age, education, race, religion).

Answer 20

Quota sampling ..

Question 21

_______________ is used to include as many opinions or views as possible by recruiting respondents (especially outliers) with many different backgrounds, interests, views, ideas, etc. It is the opposite of modal instance sampling.

Answer 21

Heterogeneity (diversity) sampling

Question 22

___________ begins by identifying participants who meets the criteria for inclusion in the study. Then the researcher asks the participant to recommend others who also meet the criteria.
§__________ is especially useful when the researcher is trying to reach populations that are
inaccessible or hard to find (e.g., homeless).

Answer 22

Snowballing sampling…

Question 23

___________ (accidental or haphazard sampling)
• Participants are included because they are convenient (e.g., recruited from classes, advertisements,
announcements)
• It is one of the most common sampling methods.
• Traditional “person on the street” interviews.
• College student research pools or surveying students in a classroom.
• In clinical practice, using clients who volunteer to participate.

Answer 23

Convenience sampling..

Question 24

Convenience sampling (____ or ____ sampling)

Answer 24

accidental ; haphazard

Question 25

_____ OF THE STUDY refers to (1) the extent to which the interpretation of the results follows from the study and (2) the extent to which the results may be generalized to other situations with other people (i.e., generalize to the population).

Answer 25

VALIDITY

Question 26

____ ____ ___
-The extent to which the outcomes result from the variables that were manipulated, measured, or selected in the study rather than from other variables not systematically treated.

Answer 26

INTERNAL VALIDITY (CONTROL)

Question 27

____ ____ arises when events occurring over time, outside the researcher’s influence, impact the outcome. In other words, an event in history affects the subjects in a way that influences the observed outcome (rather than treatment being the cause).

o Example – The researcher is evaluating a parent education class, and right after it begins, the community
starts broadcasting public service announcements (PSAs) to increase positive parenting behaviors and attitudes. Now the researcher does not know if the class or the PSAs caused the changes in parenting.

Answer 27

History threat

Question 28

___ ____ occurs when normative events over the course of time influence the outcome and confound the results. In other words, changes due to biological or psychological forces impact the outcome.
o Example – Comprehensive sex education program testing youth in 6th

Answer 28

Maturation threat

Question 29

\_\_\_ \_\_\_\_ (also called ‘practice effect’) occurs when the act of taking a pretest affects how the subjects do on the posttest. Or, when the pretest affects performance on later measures.
o Example – In a study of class participation, the researcher measured class participation prior to
implementing a new attendance policy. If the students/participants become forewarned that there was about to be an emphasis on participation, they may increase participating simply as a result of involvement in the pretest measure. Hence, the outcome could be a result of a testing threat, not the treatment (i.e., new attendance policy).

Answer 29

Testing threat

Question 30

_____ ____t occurs when the outcome is influenced by inconsistencies with the testing instrument or procedures. In other words, a change in score is due to the testing or scoring procedure.
o Calibration of a mechanical measuring device – For example, researchers examining body mass index
(BMI) were weighing subjects, but the scale was slightly off when weighing. A researcher sees the
problem and adjusts the scale, but too late for some of the subjects.
o Proficiency of a human observer or interviewer – For example, researchers examining body fat
composition are using skinfold (i.e., body fat) calipers, but research assistants (RAs) using the calipers were not trained adequately. Hence, some RAs are not using the calipers correctly, while others are using them correctly.

Answer 30

Instrumentation threa

Question 31

_____ _____ occurs when subjects are not available for later measurement, which can
lead to an inflated measure of the effect.
o Mortality could be due to subjects dropping out of the study, subjects die before the
study ends, unable to locate subjects for follow-up studies.
o Example – In a study of class participation, a stricter attendance policy goes into effect.
Hence, most students drop out of a class, leaving only those more serious students in the class (those who would participate at a high level naturally). This could mean the effect is overestimated and suffering from a mortality threat.

Answer 31

Mortality threat

Question 32

______ _____ refers to the tendency of the sample to score close to the average (or mean) of a larger
population.
o ______ artifacts occur because of the tendency of very high scores to become lower over time and
for very low scores to become higher over time. If a researcher selected participants with extremely high scores on the GRE, it is likely some of these high scorers probably did a little better than they would normally do, and their scores will be a little lower when they take the GRE again. This regression artifact would be confounded with any treatment effect.

Answer 32

Regression threat

Question 33

\_\_\_\_ \_\_\_- occurs when the treatment/control groups are different from each other due to how the
groups were selected (e.g., non random sampling).
o Example – Dividing the class in half (i.e., students on one side of class become the control group, and
students on the other side of class become the treatment group). These two groups are different since friends with similar interests/values sit together.

Answer 33

Selection threat

Question 34

____ ____ ____ refer to social pressures in the research context that can impact the results.

Answer 34

Social interaction threats

Question 35

________________ occurs when subjects in the control group learn about the treatment group, and then they imitate the subjects in the treatment. Imitation of treatment can lead to equalization between the groups on the outcome variable; which makes it difficult to see the effect of the treatment (if there is one).
o Example – A treatment group learns about a new way to combat cancer through a special
diet (i.e., the treatment). The control group finds out about the special diet. So, they start the diet also. Hence, the comparison of the treatment and control groups is no longer valid.

Answer 35

Diffusion or imitation of treatment

Question 36

_____ ______ is when the control group develops a competitive attitude towards the treatment group. This competitiveness makes it harder to detect a treatment effect.
o Example – The fat and muscle mass of two groups are measured. The treatment group
participates in an exercise program using a new exercise machine. The control group figures out that they are being compared to the treatment group so they start eating healthy and exercising.

Answer 36

Compensatory rivalry

Question 37

_____ ______ is when the control group learns about the treatment group and gets discouraged and gives up. This threat can exaggerate the posttest differences between two groups.
o Example – A treatment group is given a new drug to fight cancer. The control group
becomes resentful and demoralized because they did not get the new drug that could save
their lives. Hence, their physical and mental state deteriorates.

Answer 37

Resentful demoralization

Question 38

_____ _____ _____ ____ occurs when the project staff compensates the control group to
be ‘fair’ and because they feel bad for the control group. Helping the control group leads to an equalization between the groups and also makes it more difficult to detect a treatment effect. It is the only threat that results from actions of the staff.
o Example – Researchers are testing a prenatal parent education program for low-income mothers. The
treatment group gets the prenatal parent education program which is supposed to lead to better outcomes for the baby. The staff feels bad that the control group does not get the advantage of the parenting program, so they give the members in the control group some of the parenting materials.

Answer 38

Compensatory equalization of treatment

Question 39

Methods to counteract threats to _________
• Control group
• Random assignment
• Pretests – used to examine the possibility of prior existing differences between groups and to statistically
adjust for these differences.
§ Several different pretests should be used.
§ One of the pretests should be the same as or parallel to the posttest.
• Eliminate confounding variables

Answer 39

internal validity..

Question 40

\_\_\_\_ \_\_\_\_ (GENERALIZABILITY)
-The extent to which the findings of a particular study can be generalized to people or situations other than those observed in the study.

Answer 40

EXTERNAL VALIDITY

Question 41

Threats to \_\_\_\_\_\_
• Interaction effects of selection bias and the experimental variable
-Example – A convenience sample of parents who are court-ordered to attend the parenting class are used in the study examining how effective a parenting class is on increasing parenting effectiveness.

Answer 41

external validity..

Question 42

Threats to ________

Pretest sensitization – The pretest modifies the subject in such a way that s/he responds to the treatment differently than un-pretested persons in the same population.

Answer 42

external validity..

Question 43

Threats to __________

Experimenter effects refer to effects arising from the experimental settings that will not occur in non- experimental
-Example – Observational data are gathered with a video camera.

Answer 43

external validity.

Question 44

Threats to _______
Multiple-treatment interference refers to effects due to multiple treatments applied to the same subjects where prior treatment influences subsequent treatments in the series because their effects are not erasable. Or, the subjects get an unintended treatment in addition to the intended treatment.
-Example – Evaluating parenting education class, but only after many of the parents read a parenting book
they received from another organization.

Answer 44

external validity

…

Question 45

Threats to ________
• Inadequate preoperational explication of constructs simply means the researchers did not define the
concepts very well before s/he measured them or implemented the treatment.
-The solution – Define the concepts well before proceeding to the measurement phase of the study.

Answer 45

construct validity…

Question 46

Threats to _________

• Mono-operation bias means that the researcher only used one version of the independent variable (e.g., program/treatment) in a single place or single time, and hence, limited the breadth of the study’s results.
• The solution – When possible, implement multiple versions of the treatment/variable to increase the study’s efficacy.
• Example – Stating a financial literacy program is effective after demonstrating it is effective with one group of businessmen in downtown Los Angeles. This same program may or may not be effective with a group of blue-collar workers in a low SES neighborhood.

Answer 46

construct validity..

Question 47

___________ means that the researcher only used one measure or observation of an important concept, which in the end reduced the evidence that the measure is a valid one.
-The solution – Implement multiple measures of key concepts and do pilot studies to try to demonstrate that
the measures are valid.
o Example – Use multiple measures of self-esteem and depression in a study examining the relationship
between self-esteem and depression.
oExample – Using multiple measures of academic achievement (e.g., GPA, standardized test scores)

Answer 47

Mono-method bias..

Question 48

___________________ occurs when the testing, in combination with the treatment, produces an effect. Hence, the researcher has inadequately defined the “treatment,” because testing becomes part of it due to its influence on the outcome.
§ The solution – Label the treatment accurately.

Answer 48

Interaction of testing and treatment…

Question 49

Interaction of different treatments means that it was a combination of the treatment and other things that brought about the effect.

Whether the researcher conducts a randomized experiment or a non-random study, the groups must be as equivalent as possible! This is the best way to strengthen the internal validity of the research study.

-Example, if a researcher was studying the ability of Tylenol to reduce headaches, but in actuality it was a combination of Tylenol and Advil or Tylenol and exercise that reduced headaches, the researcher would have an interaction of different treatments threatening the construct validity.

Answer 49

…

Question 50

Social threats to ________
• Hypotheses guessing – When participants base their behavior on what they think the study is about. Thus,
the outcome is not due just to the program, but also to the participants’ reaction to the researcher and study. § A participant in a treatment study believes the pill he is asked to take each day is designed to decrease his
anxiety. So, he feels less anxiety wholly are partly because he thinks the pill will work.

Answer 50

construct validity…

Question 51

_____________ – When participants’ fear of the study influences the treatment effect.
-A participant in a study on the impact of walnuts on ‘bad’ cholesterol fears getting her blood drawn so much that she has difficulty completing the study.

Answer 51

Evaluation apprehension…

Question 52

_____________________ – When researcher reaction shapes the participant’s responses. Hence, the researcher mislabels the treatment effect s/he sees as due to the program when it is more likely due to the researcher’s behavior.

-A researcher measures a subject’s level of shoulder mobility immediately after physical therapy treatment,
but the researcher looks disappointed when the subject can’t move the arm due to pain. So, the subject moves his shoulder even more to please the researcher.

Answer 52

Experimenter/Researcher expectancies…