Confirmatiry Data Analysis Flashcards

1
Q

Confirmatory data analysis

A

Using research methods & quantitative method to verify/probe existing theories

Specify all research questions & hypotheses a priori

In hardline CDA would never use post hoc tests with corrections

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Tukey, CDA & Juries

A

Can think of research & data analysis like justice system

EDA is generating indictments of suspects

CDA is putting them on trial & getting conviction

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Hypothesis testing & falsification

A

Popper defined science as generating & testing falsifiable ideas

Generate specific hypotheses & plan your analyses in fill prior to any data collection or analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

False positives & replication crisis

A

Shady practices violate assumptions of CDA & the way that stats & p values were designed to be used

Overwhelming number of significant results which don’t replicate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Solution: pre-registration

A

Register method, hypotheses & data analysis before start data collection

Publish data regardless of findings

Vital to maintain clear distinctions between EDA & CDA or run risk or assuming all science is quantitative CDA

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Philosophies of science

A

Single studies by themselves are not science

Kuhn argues that science isn’t necessarily a real thing in the universe with an immutable definition but rather an agreed framework of approaches & theorising

CDA is not more scientific that EDA & vice versa

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Making models

A

Can make models through pure theorising, inductive use of existing data or EDA

EDA & CDA can be complementary & used together

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Testing models

A

P values & simple yes/no statistics don’t rlly work for testing the validity of more complex models

So instead have to use model fitting

Useful for situations where can’t frame entire theory in terms of simple hypothesis based on single result/finding

Let us draw out an entire theory as model & then test how well this fits our data compared to other theories/models

How we can run CDA for more complex formulations of theories

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Model fitting & structural equation modelling

A

Structural equation modelling is an umbrella term for range of different stats analyses including general linear models, paths analysis, confirmatory factor analysis & latent factor modelling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Observed variables

A

Refer to both observed & unobserved variables

Observed (manifest) variables are directly measured

Represented by rectangle diagrams

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Unobserved variables

A

Unobserved (latent) variables are theoretical/statistical assumptions by the researcher

Theoretically similar to how a factor analysis might generate explanatory factors that don’t appear in the variables/dataset

Represented by circle/eclipses in diagrams

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Observed vs unobserved variables

A

Latent variables often things we care most about as we can almost never measure anything directly (in psychology)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

When to use SEM/model fitting

A

When simple linear models aren’t sufficient to describe your data

Or when you have specific model that you think explains finding & want to test of data fits it

Or when you have multiple models you want to test against each other

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

How SEM differs to other stats e.g. SPSS

A

SEM uses slightly different terms for stats that we might be more familiar with

Significant might be used to refer to specific parameters but when considering the model overall we talk about (relative) fit of the models

Still occasionally use p values but more common to use confidence intervals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Model testing

A

Testing models against each other to see what fits data best

Unlike significance testing, this looks at relative fit

Balance between how well model describes data & how complex model is

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Comparing relative fit

A

Tendency not to use these teens absolutely-as in good/bad fit

Normally test various models against each other to see which has better fit

Always used as relative term

Can use significance tests to see if one model has significantly better fit

17
Q

Assessing model fit

A

Any piece of software used for SEM will have various different ways of estimating model fit

Some will be more/less use for larger/smaller sample sizes or in different contexts

Parameter estimates within models will typically be given with 95% confidence intervals rather than with p values

18
Q

Criteria can use for fit

A

Chi-square

root mean square error of approximation

Comparative fit index

Tucker-Lewis index