Guiding Questions Deck 2

Question 1

Define unit of analysis. How does this guide study design?

Answer 1

• The unit of analysis in research, the group of people, things, or entities that are being investigated or studied.
• For example, in organizational contexts, data can be collected from students, who in turn are part of schools, which in turn are part of districts, which may have multiple sites in several counties.
• The unit of analysis helps define the scope of the research question.

Question 2

Define operational definition of a variable. How does this differ from a conceptual definition?

Answer 2

• A variable is a description of something in terms of the operations (procedures, actions, or processes) by which it could be observed and measured.
• For example, the operational definition of anxiety could be in terms of a test score, withdrawal from a situation, or activation of the sympathetic nervous system. The process of creating an operational definition is known as operationalization.
• An operational definition differs from a conceptual definition in that it provides a precise, measurable way to observe or manipulate a variable, specifying the exact procedures used in the research. It translates abstract concepts into concrete terms by outlining how a variable will be measured or manipulated.
• In contrast, a conceptual definition offers a theoretical explanation of a variable, describing its meaning and significance within a broader framework.

Question 3

Define constant and variable, as well as types of variables (discreet, continuous, dichotomous, polytomous).

Answer 3

• A constant is a value that remains the same throughout a study or experiment. It does not change or vary in response to experimental conditions or over time. Constants are crucial for providing a stable reference point and ensuring that other variable effects are observable.
• A variable is any characteristic, number, or quantity that can vary or change across different conditions, individuals, or over time. Variables are the primary focus of research, as they are manipulated or measured to observe changes, correlations, or effects.
• Discrete variables (finite range) are countable and take on distinct, separate values. They usually represent counts or specific categories where no intermediate values exist between adjacent values.
• Continuous variables can take on an infinite number of values within a given range. They can be measured in finer and finer increments.
• Dichotomous variables are a type of discrete variable that have only two possible values or categories.
• Polytomous variables are categorical variables that can take on more than two categories or values.

Question 4

Contrast independent and dependent variables, and describe the relationship between them. Define and contrast mediator and moderator variables.

Answer 4

• An independent variable is the variable that is manipulated or controlled by the researcher to observe its effect on other variables. It represents the cause or input that influences the outcome. In experimental research, the independent variable is deliberately changed to observe its direct impact on other variables.
• Example: In a study examining the effects of studying on test scores, the independent variable could be the number of hours spent studying.
• A dependent variable is the variable that is measured or observed to assess the effect of the independent variable. It represents the outcome or effect influenced by the independent variable.
• Example: In the same study on studying and test scores, the dependent variable would be the test scores, as they depend on the number of hours spent studying.
• A mediator variable explains the process through which the independent variable influences the dependent variable.
• Example: If studying (independent variable) affects test scores (dependent variable) through its impact on student motivation (mediator), then motivation is the mediating variable that explains how studying leads to higher test scores.
• A moderator variable changes the strength or direction of the relationship between the independent and dependent variables. It helps identify the conditions under which the effect of the independent variable on the dependent variable varies.
• Example: In the study on studying and test scores, if age (moderator) affects the relationship between studying and test scores (e.g., older students benefit more from studying than younger students), age is the moderator variable.

Question 5

Describe the four scales of measurement, including the properties that each scale adheres to. Provide an example of each.

Answer 5

• The four scales of measurement describe which property or set of properties a phenomenon adheres to, and thus determines what statistics we can use to describe and summarize the construct.
• Nominal: describes arbitrary labels used to identify individual categorizations (identity). Examples would include social security numbers, zip codes, or arbitrary numerical codes in research.
• Ordinal: used to order a hierarchical series, such as rank order (first, second, third…), or percentiles. Each value is greater or less than another (magnitude), though no standard interval exists between values. For example, the difference between first and second place may not be the same as the difference between second and third. Examples: Order of finish in a race or a contest, Letter grades: A, B, C, D, or F, Level of agreement (e.g., Likert scale): Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree
• Interval: contains an ordered series with equal intervals between values, but no absolute-zero point. Consider temperature in degrees Fahrenheit, a measure of heat. The difference between 10° and 15° is the same as between 90° and 95°, but a temperature of 0° does not indicate a complete absence of heat. IQ and SAT scores are interval scales. Examples: Scores on the College Board’s Scholastic Aptitude Test, which measures a student’s scores on reading, writing, and math on a scale of 200 to 800, Intelligence Quotient scores, Dates on a calendar
• Ratio: an ordered series with equal intervals between values and an absolute-zero or “starting” point. Distance from a starting point or temperature in degrees Kelvin are both ratio scales, as a score of zero indicates an absolute zero, or the complete absence of the construct being measured. Examples: Height, Income, Distance traveled, Time elapsed or time remaining

Question 6

Define and contrast descriptive and inferential/analytical statistics.

Answer 6

• Descriptive statistics are procedures used to summarize, organize, and simplify data, and may be used with either a sample or a population.
  Central tendency: mean, median, mode
  Variability: variance, standard deviation
• Inferential statistics are techniques that allow us to compare two or more samples to each other in order to make generalizations about the population from which the sample is drawn.
  T-test, ANOVA, Correlation

Question 7

Contrast population and sample.

Answer 7

• Population refers to the entire set of individuals of interest in the study.
• A sample is the set of individuals selected to represent the population.
  Sampling in the modern world:
  In-person, Phones and call or polling centers, Web research: email, websites, consumer surveys, Amazon Mechanical Turk and apps

Question 8

Define a simple random sample and its primary advantage in terms of research validity.

Answer 8

• In a simple random sample, each member of the population has an equal chance of being selected.
• The primary advantage of a simple random sample is that it helps ensure the sample is representative of the population, which enhances the study’s external validity. External validity refers to the extent to which the findings of a study can be generalized to the broader population. Since every individual in the population has an equal chance of being selected.

Question 9

Define and contrast probability and nonprobability samples. Give three examples of each.

Answer 9

Probability samples: Are one in which each member of the population has a chance of being selected.
- Simple random
- Cluster: groups of participants, usually location-based
- Stratified: proportional, random sample of homogeneous groups
- Systematic (every nth)
- Oversampling

Nonprobability samples: Better for direct exploration of topics in greater depth, qualitative studies
- Convenience
- Purposeful: individuals judged to be good sources of information
- Snowball: participants recruit other participants
- Self-selection

Question 10

Describe why sample size is important for a study in terms of sampling error and probability.

Answer 10

Sample size is crucial in research because it directly affects the reliability and accuracy of the study’s results.

• Sampling Error: Sampling error refers to the difference between the sample results and the actual population values due to studying only a subset of the entire population. Larger sample sizes tend to reduce sampling error because they better represent the population, leading to more accurate and reliable results
• Probability: In studies involving statistical inference, the sample size affects the statistical power of the tests. A larger sample size increases the probability of detecting a true effect (if one exists), reducing the risk of Type II errors (false negatives). It ensures that the study has sufficient power to identify significant relationships or differences.

Question 11

Give one “rule of thumb” on minimum sample size. How does this differ for qualitative designs?

Answer 11

• Minimum 100: For studies using statistical analysis, a sample size of at least 100 is often considered the minimum necessary to ensure sufficient power for meaningful statistical inferences.
• 300+: Sample sizes of 300 or more are generally considered more robust, especially for studies looking to generalize results to a broader population.
• 10% Rule of Thumb: For large populations, a sample size that is 10% of the population (up to a maximum of 1,000 respondents) is often suggested as a guideline.
• In qualitative research, the concept of sample size differs significantly from quantitative studies. Here, the focus is on obtaining rich, in-depth insights rather than numerical generalizability.
• Smaller Sample Sizes: Qualitative studies typically involve smaller sample sizes, average approx. 20-50 participants.
• Judgment: Studies are more reliant on researcher’s judgment than statistical analysis
• Focus on Saturation: Instead of a numerical target, qualitative research relies on reaching data saturation. Once additional interviews or observations don’t add new information or themes, the sample is considered sufficient.

Question 12

Define saturation in terms of qualitative sampling.

Answer 12

• Saturation is a key concept in qualitative research, referring to the point at which no new information or themes emerge from the data.
• This suggests that further data collection would likely yield diminishing returns in terms of new insights.
• Saturation helps researchers decide when they have enough data to address their research questions, allowing them to stop data collection and begin analysis.