Interpreting Outputs and Threats Lecture - Dr Wofford

Question 1

Degrees of Freedom

Answer 1

•Number of values within a distribution that are free to vary

• Generally n-1
• ANOVA: total degrees of freedom (df_t)= error degrees of freedom (df_e) + between groups degrees of freedom (df_b)

For One-Way ANOVA If: n= total data points collected, and m= number of groups being compared, & [] example if there was 3 groups and 30 points of data:

• df_t is total degrees of freedom (n-1 or m-1 + n-m) [29]
• df_e is error degrees of freedom (the same as within-group degrees of freedom) or (n-m) [27]
• df_b is between-group degrees of freedom (m-1) [2]

Must be able to do this math

____________________________________more notes

ANOVA is different.

A one-way ANOVA:

Three groups. Each are receiving a different treatment (need at least three to do ANOVA). IV = intervention or group (three of them), one DV which is test results.

error degrees of freedom = degrees within the groups

Question 2

Four Types of Threats to Validity

Answer 2

Types of threats:

1. •Statistical conclusion validity
2. •Internal validity
3. •Construct validity of causes and effects
4. •External Validity

Question 2

Interpret this Independent Samples T-test output

Answer 2

Unpaired t-test

Where it says sig = p-value

Derived from levines. Want that to be above the established alpha.

Don’t worry about F.

Two separate p-values

Mean difference is difference between groups.

Two columns: one is when the levine’s test is below alpha, the other is when levine’s test is above alpha (the better column)

Sig. (2-tailed) = p-value for this test

Standard error difference is the standard variablility between two groups.

Question 2

Interpret this One-Way ANOVA Output

Answer 2

Exact same thing that we had in the other tale we went over.

F is the F-stat. you look it up on the F-stat table to find significance (p)

ANOVA is t-test on steroids because it has more than two groups.

Question 3

Threats to Construct validity of causes and effects

Answer 3

Hawthorne effect: The effect of the DV results from subjects’ awareness that they are participating in the study.

• Placebo-type effect

This is the main one that she wants us to know

Question 3

Interpret this Repeated Measures ANOVA output

Answer 3

The results you don’t want

Question 5

External Validity (what it is and threats)

Answer 5

External Validity is How generalizable our results are to the population

•Threats include the following:

1. •Interaction of treatment with the specific type of subjects tested
   
   1. •Problem with factorial designs when >1 IV is being manipulated
2. •Specific setting in which the experiment is carried out
3. •Time in history when the study is performed

Question 7

Describe how to find

df_b

df_e

df_t

Answer 7

df_b = m-1 [number of groups - 1)

df_e = n-m [sample size - number of groups]

df_t = df_b + df_e OR n-1 [sample size - 1]

n can also be called total data points collected (sample size)

Question 9

Internal Validity Threats

Answer 9

1. •History: Unplanned external events occur concurrently with the IV and can affect outcomes
   
   • •Threat particularly with a crossover design
2. •Maturation: Changes in the DV result from a passage of time
   
   • •One group pre-test to post-test is most susceptible
3. •Attrition: Loss of participants over the course of the study
   
   • •Creates a bias
4. •Testing: Effects of a pretest on subject’s performance on post test
5. •Instrumentation: Changes in measuring instruments or methods between two points of data collection
6. •Regression towards the mean: When measures are not reliable, tendency for extreme scores on the pretest to regress towards the mean on posttest
   
   • •Increases the group mean on posttest due to chance variation

A big deal when reviewing articles

Question 10

Describe how to find

SS_b

MS_b

F

Answer 10

SS_b is between group mean square

1. For each group: Square the average between the mean of each group and the grand mean (mean of all the data)
2. Do this for each group (if you didn’t get that from 1)
3. Add these squres together

You now have SS_b

you can find MS_b by SS_b/df_b

you can find F by MS_b/MS_e

Question 11

Statistical Conclusion Validity (what it is an threats)

Answer 11

Degree to which inferences about relationships from a statistical analysis of the data are correct

• •Affected by the following:
  
  • •Low statistical power
  • •Violated assumptions of statistical tests
  • •Low reliability

Question 12

Describe how to find

SS_e

MS_e

F

Answer 12

SS_e is error mean square or within-group mean square

1. Square the average between the mean of group and each data point within the group.
2. Add these squares together

You now have SS_e*

You can find MS_e by SS_e/df_e

you can find F by MS_b/MS_e

* I’m not sure how to account for each group, maybe MS_e is the result of 2 added all together? or averaged? no idea really.

Question 13

Interpret this output for a Repeated Measures ANOVA

Answer 13

This is a between group and within group difference

This is what you want

Question 14

Interpret this Dependent Samples T-test output

Answer 14

Paired t-test

We do not need to check Levine’s because the exact same people are being tested.

Bottom part is the important part

This one was not significant

When reporting data, you are likely going to report to t-stat and the p-value

Question 15

Repeated Measures ANOVA (interpret this test output)

Answer 15

Different than the one-way anova

Within subject is measured more than one time.

Look for between-group significance (are the groups different?)

and

Look for significance over time (are groups different over time?)

Still get F-stats, just like before

But now we have two tables

Don’t worry about intercept part

Just look where it says group.

Witin groups

Look at time

Sphericity – test is called Mockley test - (assumption of the repeated measures test – just like Levine’s but for repeated measures test)

The named weird things are three different things e can use to correct for violating sphericity

We will just assume it was not violated, so only look at the first row.

Groups may be different, but their differences did not change over time.

Pre-test post test design you use

No such thing as a repeated measures t-test

This is unfortunately what you usually get.

Question 16

Draw A One-Way Analysis of Variance Table and figure out the degrees of freedom

if given the following:

20 subjects per group

3 groups

Probably Ignore calculating SS, MS, and F, but know the relationships. Draw relationships and equations if the following were true:

Between group averages: Grand Mean: 13, G_A mean: 10, G_B mean: 11, G_C mean: 9

Within group averages: G_A mean: 10 For sake of time, 10 data-points_A: 8, & 10 data-points_A: 13. Ignore groups B and C for this.

Most important part is to be able to write and calculate df, number of groups, and sample size if given any of the related numbers.

Answer 16

degrees of freedom: df_t = 59, df_e = 57, df_b = 2

SS_b = 3²+2²+4² = 29

SS_e = (2² x 10) + (3²x10) = 130

SS_t = 29 + 130 = 159

MS_b = SS_b/df_b = 29/2 = 14.5

MS_e = SS_e/df_e = 130/57 = 2.28

F = MS_b/MS_e = 14.5/2.28 = 6.36

if given the following:

20 subjects per group

3 groups

Between group averages: Grand Mean: 13, GA mean: 10, GB mean: 11, GC mean: 9

Within group averages: GA mean: 10 For sake of time, 10 data-points A: 8, & 10 data-points A: 13. Ignore groups B and C for this.

Most important part is to be able to write and calculate df, number of groups, and sample size if given any of the related numbers.