Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

HSC 4713 > Lesson 11 (slides 1-21) > Flashcards

Lesson 11 (slides 1-21) Flashcards

1
Q

What is the plan called that establishes data collection procedures and hypothosis testing?

A

Evaluation design AKA research design

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What provides structure for the major components of the study?

A

Evaluation design

  • When, from whom, how, and what program procedures and measurements will be applied
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Overall, what is evaluation design attempting to address?

A

CAUSALITY

  • Did the intervention/program cause the effects?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the three design characteristics for improving causality?

A
  1. Comparison/control group similar to the experimental/exposed group
  2. Measurement of outcome variables before and after the intervention
  3. Minimal threats to internal and external validity
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

EVALUATION DESIGN NOTATION

R

A

Random assignment of an evaluation study participant (or unit) to a group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

EVALUATION DESIGN NOTATION

E

A

Experimental (intervention or treatment) group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does E1, E2, E3, …..En indicate?

A

Indicates planned exposure of the group to different intervention procedures.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

EVALUATION DESIGN NOTATION

C

A

Control (equivalent) group established only by random assignment

  • Indicates NO exposure to an intervention or exposure to a minimum or standard intervention procedures
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

EVALUATION DESIGN NOTATION

C

A

Comparison group established through any other method other than randomization

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

EVALUATION DESIGN NOTATION

X

A

Intervention procedures applied to an E (experimental) group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What does X1, X2, X3, …..Xn indicate?

A

Indicates an intervention consisting of multiple, different approaches

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

EVALUATION DESIGN NOTATION

O

A

An observation or measurement to collect data.

Including:

  • Tests
  • Interviews
  • Visual or audio ratings, or
  • Record interviews
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What does O1, O2, O3,….. indicate?

A

Indicates multiple measurements at different times

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

EVALUATION DESIGN NOTATION

T

A

Time when an observation, assignment to a group, or application of intervention procedures has occured.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What does T1, T2,T3,…. Tn indicate?

A

Indicates repeated observations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

EVALUATION DESIGN NOTATION

n

A

Number of participants (sample)

- Could be in the E, C, or C group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

CATEGORIES OF EVALUATION DESIGN

18
Q

Is random assignment used?

  • YES
A

Experimental design

19
Q

Is random assignment used?

  • NO

Is there a comparison group or multiple observations?

YES?

NO?

A

YES

  • Quasi-experiemental design

NO

  • Non-experimental design
20
Q

CATEGORIES OF EVALUATION DESIGN

Includes one experimental group (E) but no

control group (C) or comparison group (C)

A

Non-experiemental design

21
Q

What are the 3 research designs?

Which one is the weakest and why?

A

The three research designs:

  1. Non-experimental design
    • Weakest in regards to internal and external validity
  2. Quasi-experiemental design
  3. Experiemental design
22
Q

CATEGORIES OF EVALUATION DESIGN

  • Includes an experiemental group (E) and a comparison group (C) created by methods other than random assignment
  • Includes observations of both groups, both prior to and after application of intervention (X) procedures
    • Multiple waves of data collection
    • Does not account for all biases of internal validity
A

Quasi-Experimental Design

23
Q

CATEGORIES OF EVALUATION DESIGN

  • Includes random assignment of experimental (E) group and a control (C) group
    • Observations of both groups prior to and after application of the intervention (X) procedures
    • Yields the most interpretable and defensible results of evidence of behavioral impact
A

Experimental Design

24
Q

What is the ‘Gold Standard’ in evaluation design and why?

A

Experimental design

  • It yields the most interpretable and defensible evidence of behavioral impact
  • Asserts the greatest degree of control over biasing factors that compromise internal validity of results
25
Q

What are the most commonly used evaluation (and research) designs?

A
  1. One-shot case study
  2. Retrospective Pretest
  3. One-group pretest-posttest design
  4. Time series design
  5. Pretest-posttest control-group design
  6. Case study design
26
Q

Which design is this?

X——–O

(hint: How are paticipants faring after the services?)

A

One-shot case study

  • (non-experimental)*
  • Survey of program participants after participation in the program
27
Q

What is the absolute most simplest form of outcome evaluation?

WebRep

currentVote

noRating

noWeight

A

One-shot Design

Ex: How many people who attended a smoking cessation program are not smoking a month later?

- How participants are faring after the services.

WebRep

currentVote

noRating

noWeight

28
Q

What are the primary weaknesses of one-shot design?

A

Does not demonstrate change during or as a result of the program

  • No comparison
  • No multiple waves of data
29
Q

When is it best to use a one-shot design?

A

Best if used to answer descriptive research questions

30
Q

Why would it be difficult to establish observations are due to a disaster with a one-shot design?

A

Because there would be no comparison and no multiple waves of data collection

31
Q

Which design is this?

X——–O

***respondents are asked to recall information before and after the intervention

A

Retrospective Pretest

  • Survey of program participants after participation in the program
32
Q

What are the primary weaknesses of the retrospective pretest?

A

Recall Bias

  • Relies on memory
33
Q

Which design is this?

O1——-X——–O2

A

One Group Pretest-Posttest Design

  • Evaluator collects data before and after the program is conducted
34
Q

What are the primary weaknesses of the one group prestest-posttest design?

A
  • Potential selection bias
    • Attrition (drop-out)
  • Potential measurment bias
    • Poor validity & reliability
35
Q

What needs to be considered with internal validity?

A

Is the intervention the only (or most) plausible explanation for the changes?

36
Q

What is internal validity?

A

Degree to which measured change can be attributed to the program

37
Q

What is external validity?

A

Extent to which the program can produce similar effects in other populations

- Is the program generalizable?

38
Q

What are the two types of threats to internal validity?

A

Two primary sources of bias:

  1. Non-program related sources of bias
  2. Program related sources of bias
39
Q

What are the two sources of non-program related bias?

A
  1. Maturation
  2. Historical bias
40
Q

What are the three sources of program related bias?

A
  1. Attrition
  2. Selection bias
  3. Measurement bias
41
Q

Which sources of bias are uncontrollable and which are somewhat controllable?

A

Uncontrollable: non-program related

Controllable: program related

HSC 4713 (6 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions