Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

MGMT-803 > Validity > Flashcards

Validity Flashcards

1
Q

Statistical conclusion validity

A

(were stats appropriate?)

Validity of inferences about the correlation (covariation) between treatment and outcome.

  • Ensure adequate sample size for sufficient statistical power.
  • Apply appropriate statistical techniques and correct for multiple testing.
  • Control for random error and use reliable measurement tools
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Internal validity

A

(did A cause B?)

Validity of inferences about whether observed covariation between A (presumed treatment) and B (the presumed outcome) reflects a causal relationship from A to B as those variables were manipulated or measured.

Randomization!!!
- Use randomization to assign participants to groups.
- Implement control groups and use blinding.
- Employ a pretest-posttest design.
- Ensure consistent treatment administration.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

External validity

A

(can lessons be extended beyond the lab?)

Validity of inferences about whether the cause-effects relationship holds over variation in persons, settings, treatment variables, and measurement variables.

Generalizability!!!! and replicate!!!!
- Use random sampling to select participants.
- Replicate studies in different settings and with different populations.
- Test the intervention across various conditions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Construct validity

A

(what is A?)

Validity of inferences about higher-order constructs that represent sample particulars.

  • Clearly define and operationalize constructs.
  • Use multiple methods and measures to assess constructs.
  • Validate the measurement tools with pilot studies.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Ecological validity

A

(does the lab context mirror the real world?):

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Ambiguous temporal precedence

A

Internal validity
A causes B? or vice versa (poor sleep vs stress).
Longitudinal or time-series design, and include pre and post-test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Selection

A

Internal validity
differences in respondent characteristics ( young vs. old)
Random assignment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

History

A

Internal Validity
concurrent events (political events or laws)
Use a control group to compare the effect of external events on both the treatment and control groups.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Maturation

A

Internal validity
natural changes (older and aging)
Shorten the time span of the study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Regression

A

Internal validity
extreme scores reverting to mean (high stress ppl, lower level report)
Use repeated measures and include a wide range of participants to reduce regression to the mean effects.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Attrition

A

Internal validity
systematically drop out (a demanding treatment group)
Implement strategies to maintain participant engagement, monitor reasons for dropouts, and use intention-to-treat analysis
T-test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Testing

A

Internal validity
test exposure (2nd test with familiarity)
Employ alternative forms of the test or counterbalance the test forms to reduce the impact of repeated testing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Instrumentation

A

Internal validity
measure/condition changes over time (change in surveys)
Ensure consistency in measurement tools and procedures throughout the study, and calibrate instruments if necessary.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Additive and interactive effects

A

Internal validity
combined or dependent multiple threats (history + maturation)
Design the study to isolate variables and use statistical controls to account for potential interactions and additive effects.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Interaction of causal relationships with unit

A

External validity
variability with different unit types (effective in adults NOT in children)
Use random sampling to select participants and replicate the study across different populations.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Interaction of causal relationships over treatment variations

A

External validity
variability with different treatments (effects of different dosages)
Vary the levels of the treatment and replicate the study with different treatment protocols.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Interaction of causal relationships with outcomes

A

External validity
variability with different outcome observations (improved mood but not cognitive function from a treatment)
Measure multiple outcomes and use different methods to assess the outcomes to ensure generalizability.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Interaction of causal relationships with settings

A

External validity
variability in different settings (therapy effective in clinical settings but not at home)
Conduct the study in various settings or replicate the study in different environments to test for consistency.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Context-dependent mediation

A

External validity
mediator variability in different contexts (social support mediating stress reduction only in community setting)
Explore the mediating effects in different contexts and include diverse contexts in the study design to assess the robustness of the mediating effect.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Inadequate explication of constructs

A

Construct Validity
poor construct definition lead to incorrect inferences (vaguely defined well-being in psychological.)
Clearly define constructs and ensure they align with established theories; use a panel of experts for validation.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Construct confounding

A

Construct validity
failure to describe all relevant constructs (not consider both anxiety and depression in mental health study)
Distinguish the constructs from one another and measure them independently; use confirmatory factor analysis.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Mono-operation bias

A

Construct Validity
limited operationalization complicating inference (only use self-reports to measure happiness)
Employ multiple measures or observations for the construct to capture its complexity.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Mono-method bias

A

Construct validity
only use one method, becomes part of the studied construct (self-report)
Use different methods of measurement (e.g., observations, interviews, and surveys) to assess the construct.

24
Q

Confounding construct with levels of constructs

A

Construct validity
failure to describe the limited levels of studied construct (no specific type of aggression in a study on aggressive behavior)
Define constructs at consistent levels of specificity and avoid mixing abstract and concrete levels.

25
Q

Treatment-sensitive factorial structure

A

Construct validity
measured structure changing due to treatment, hidden by consistent scoring.
Perform factorial analysis both before and after the treatment to ensure the treatment does not alter the construct.

26
Q

Reactive self-report changes

A

Construct Validity
self-report changes by participant motivation changes (participants provide fakeanswers)
Utilize non-reactive measures and triangulate self-report data with objective data when possible.

27
Q

Reactive to the experimental situation

A

Construct validity
hypothesis guessing
ppl response reflecting perceptions of experimental situation (ppl know they are in the experimental and behave differently)
Design experiments to be as naturalistic as possible and use field experiments to reduce this reactivity.

28
Q

Experimenter expectancies

A

Construct validity
experimenter influence on ppl responses
Use double-blind procedures where neither the participant nor the experimenter knows the treatment condition.

29
Q

Novelty and disruption effect

A

Construct validity
unusual response to novel and disruptive treatment (positive response to a new tech just because it is new)
Familiarize participants with the experimental process to reduce the novelty factor.

30
Q

Compensatory equalization

A

construct validity
compensatory goods to non-treatment group (control group), to make sure control group received equalized compensatory
Prevent control groups from knowing what the experimental group receives to avoid compensatory behavior.

31
Q

Compensatory rivalry

A

Construct validity
control group motivated to perform as well as those treated (extra efforts from control group to match treatment group)
Keep participants blind to other groups’ conditions to reduce competitive efforts to compensate.

32
Q

Resentful demoralization

A

Construct Validity
negative response from ppl due to resentment (control group shows less engagement due to not receiving treatment)
Ensure equal treatment of groups in terms of interactions and communications to prevent demoralization.

33
Q

Treatment diffusion

A

Construct validity
treatment received by non-assigned participants (treatment information known by ppl in control group)
Isolate groups and maintain confidentiality about the specific nature of treatments to prevent sharing of treatment details.

34
Q

Low statistical power

A

Statistical conclusion validity

Insufficient experiment power leading to incorrect conclusions about significance

Increase the sample size or use more sensitive statistical methods.

35
Q

Violated assumptions of statistical test

A

Statistical conclusion validity

Violations leading to overestimation or underestimation of effect size and significance

Use nonparametric tests or transform data to meet assumptions.

36
Q

Fishing and error rate problem

A

Statistical conclusion validity

Repeated testing inflates statistical significance if not adjusted.

Apply corrections like the Bonferroni adjustment for multiple comparisons.

37
Q

Unreliability of measures

A

Statistical conclusion validity

Measurement error affecting the relationships between variables.

Use validated and reliable instruments; conduct pilot studies if necessary.

38
Q

Restriction of range

A

Statistical conclusion validity

Reduced variable range weakening relationships.

Ensure sampling covers the full range of the variable.

39
Q

Unreliability of treatment implementation

A

Statistical conclusion validity

Partial or inconsistent treatment implementation affecting effect estimation.

Standardize treatment protocols and training.

40
Q

Extraneous variance in the experimental setting

A

Statistical conclusion validity

Features of the setting inflating error and obscuring effects.

Control extraneous variables and use randomization.

41
Q

Heterogeneity of units

A

Statistical conclusion validity

Increased variability within conditions raising error variance.

Use matching or statistical controls to account for participant variability.

42
Q

Inaccurate effect size estimation

A

Statistical conclusion validity

Statistics overestimating or underestimating effect sizes.

Use appropriate statistical methods and provide confidence intervals.

43
Q

Hallmark of good study

A

1 variable 1 time.
Isolate changes! With 2 more variation, it is difficult to know what is the casual factor

44
Q

Double-blind procedure

A

What? This is a construct validity. Try to make sure the participant don’t know or cannot guess the hypothesis. Generally used in clinic trials to avoid bias. No one (participants or researchers) should know who receive the actual treatment and who receive placebo.

How to deal with? In the planning and setup to make sure it is indistinguishable to both participants and researchers. Implication such as same color, taste, equal treatment will help.

45
Q

Proxy variable

A

What? Using a different variable in place of another variable as a proxy variable – indirect measurement.

Whether it is appropriate? (1) High correlation (2) whether it is used before by building theoretical argument to build a plausible argument.

46
Q

Moderation vs mediation

A

(1) Mediation: how variable A and B related through C, A->C->B
(2) Moderation How variable A and B are related with the presence of C, A->B (C)

47
Q

Two groups and with group variance, Intervention

A

F-value=(Between-group variance)/(Within-group variance)
Need more between group variance than within group variance. to maximize the F-values by (1) enlarging between-group variance and/or (2) reducing the within-group variance. By using either experimental or statistical controls to archive it.

48
Q

Ceiling effects or flooring effects

A

What? Most of the score a located on the higher ranges, e.g., from 0 to 10 ranges, it sits between 7 and 10. Plot the data.

Two threats to the validity
(1) Statistical threat: Range restriction.
(2) Confounding threat: the construct validity:

49
Q

Solomon 4-group

A

O1 pre test
O2 post test
X treatment
Random test vs quasi-random test

50
Q

possible vs plausible threat to validity

A

possible threats” refers to any and all threats that could theoretically occur, given the circumstances and design of the study. – random dropout

“Plausible threats,” on the other hand, are those that not only could occur but also have a reasonable likelihood of occurring based on the specific context of the study. – systematic dropout

51
Q

Determining the presence of compensatory rivalry or resentful demoralization as a threat to construct validity based on visual inspection of data.

A

construct validity
compensatory rivalry: control group behave better on purpose
ressentful demoralization: control group behave negatively.

52
Q

History vs. Maturation

A

internal validity
History threat is out of researchers’ control, e.g. new law.
Maturation threat is the change over time for participants. To design a shorter test, if possible.

53
Q

Content vs. Construct Validity

A

Content validity is about whether the content can capture the concept fully.
Construct validity is about the method or operation that can capture the measure of the test.

54
Q

Proxy Measure

A

A proxy measure is an indirect measure of an outcome when direct measurement is not possible. It’s used when the actual variable of interest cannot be easily or ethically measured, so a related indicator is measured instead, with the assumption that it closely approximates the true variable

55
Q

Confounding Constructs with Levels of Constructs

A

construct validity
Confounding construct: the struct has confounding effects and not clearly defined, which causes the errors in the measurement
Level of construct, there are different levels of construct that are not design properly causing measuring errors.

56
Q

Exposure in pretest

A

To establish whether this is a plausible threat, researchers can use a Solomon Four-Group design. This design includes four groups: two of them receive the pretest, and two do not. Of the two that receive the pretest, one group receives the treatment and the other does not. Of the two that do not receive the pretest, one group receives the treatment and the other does not. By comparing the groups, researchers can determine if the pretest influenced the outcomes.

MGMT-803 (1 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions