Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Psychometrics > Final > Flashcards

Final Flashcards

1
Q

In ethics, what do you need to worry about?

A
  • Test/ Questionnaire selection
  • Participants recruitment
  • Treatment of participants
  • Scoring and interpretation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is part of the treatment of participants?

A

Even decisions made prior to seeing any participants are part of ethical measures

  • Understand your measures
  • Follow appropriate procedures
  • Ensure participant wellbeing (don’t waste their time)
  • Give back feedback and make sure you’re the right person to do so
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is informed consent?

A

To make sure participants know what they’re agreeing to

Includes: estimating time commitment, their right to withdraw and goals of study but not hypotheses/ deception

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How do we ensure the confidentiality of participants?

A

By handling the data responsibly

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Should we deliver feedback appropriately after the experiment?

A

Yes, and should benefit participants

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is part of responsible data handling?

A
  • No unauthorized access to your data (even government, can give access to people that can prevent/ improve mistakes)
  • Safe retention of data (anonimity+ keep 5 years before destroying)
  • In testing, ensure data interpreted appropriately
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Is data removal allowed?

A

Yes, but need to be done responsibly

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What consists of responsible data removal?

A
  • Balancing need to respect participant’s contribution with the need to research accurate conclusions
  • Participants have the right to try to derail our study but not succeed at it
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What should we always remember in data removal?

A
  • Our statistics have assumptions we should meet

* Many statistics require complete data sets

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is the purpose of the APA guidelines?

A

To ensure usefulness and proper application of the techniques
In questionnaires: privacy, right people, correct setting, amount of time (pressure, back and forth, limited, etc.)= use tools correctly for tight purposes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Give an example of when the responsible use of questionnaires should be used and why? (hint: Beck Depression Inventory)

A
  • Each category assess one aspect of depression
  • Need to make sure the questionnaire doesn’t cause harm to participant (in this case, might cause a negative mood)
  • For suicidal thoughts and wishes, might create problems in terms of ethics, the role of the researcher to ensure participants don’t harm themselves (well-being of participants)
  • Need to make sure researcher is ok to deal with potential fall back
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What do ethics applications must establish?

A

Scientific merit, safe handling of data and protection of participant’s well-being

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Can we give out the personal data of participants?

A

No!
It is confidential info of participants and it is often regulated by the law
Protects the client/patient/participant
For research purposes, can give collective data but not individual

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are the assumptions of Factor Analysis?

A
  • Based on General Factor Model, factor analysis shares some important assumptions with classical measurement models or congeneric model
  • Errors are random and not correlated with latent variable
  • Items are correlated with each other because they share a common latent variable
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is a latent variable?

A

Unobserved influences on our measurements or constructs that we are trying to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is a factor?

A

Another way to refer to a latent variable and component but debate with maths

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Why would we use the term factor more than latent variable?

A

Main reason to use these terms instead of latent variable is to better acknowledge that the unobserved influence was derived empirically (looking into data where patterns are).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is factor analysis?

A

The process of trying to identify the latent variables that influence our measurements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

In factor analysis, what forms identifiable clusters?

A

Items that have a stronger association (correlations) with each other but weaker associations with other items will form identifiable clusters

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are we capitalizing on in correlation across items?

A

Similarities and differences

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What happens if items correlate strongly?

A

There will be one factor/ latent variable identified

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What are the rules for good factor analysis?

A

-Need quite large data to work effectively (will probably allow us to recreate pattern later as well)
It needs to be larger than for internal consistency reliability or validity analyses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What does a simple construct should produce?

A

One factor

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What are questionnaires meant to assess

A

One simple construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What is an iterative process and what should be viewed as such?

A

Iterative process: going through a cycle by starting over with change made
Factor analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What is some general info about items in factor analysis?

A
  • What you put in analysis dictates what you get out of it (garbage items=garbage analysis)
  • Adding or dropping even one item will change the outcome (small or big shift)
  • Every item has potential to create an item or influence creation of other factors
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

True or false: The more items you have, the more likely you are to find one specific factor

A

False!
The more items you add to a questionnaire, the less likely it becomes that you will find only 1 factor (so should remove extra questions to measure the 2 constructs)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What gets called factors?

A

The important clusters

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

How do we define the importance of a factor?

A

-Typically, when a factor has an eigenvalue bigger than 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

What does an eigenvalue meausre?

A

Measures the amount of information captured by a cluster/ pattern identified in the data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

What does an eigenvalue of 1 mean?

A

Indicates the factor captures as much information as one typical good item

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

What does parallel analysis create?

A
  • Creates random data with the same number of variables and observations as your data
  • Creates correlation matrix where eigenvalues are calculated
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

Parallel Analysis: What happens when eigenvalues from random data are larger than the eigenvalues from your real data?

A

Know real data not correlated better than random data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Does factor loading always need to be positive?

A

No!

Negatives do not damage the apparent accuracy of the exploratory factor analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

What do we use when exploratory factor analysis is left with a subjective decision to make?

A

Use our judgement! Some tools at our disposal to help

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

What is a scree plot and what does it do?

A

It is a visual representation of the eigenvalues obtained in analysis
-Shows which factor are above the 1 value, important to look for the steep vertical line

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

What is the minimum factor loading?

A

-1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

What does a rotation allow us to see?

A

The spread of variability among our factors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

What are the 2 forms of rotation

A

Orthogonal and Oblique

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

What is an orthogonal rotation?

A

It maximizes the squared variance in the factor loadings

  • Clusters are as different as possible and unrelated
  • Rotation is clock-wise after returned to vertical orientation for analysis. Can potentially reframe how we see things
  • It makes it easier to identify the differences between items in relation to their clustering
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

What is the underlying theory of orthogonal rotation?

A

2 unconnected variables (like the original model)

42
Q

What rotation is associated with varimax rotation?

A

Orthogonal rotation

43
Q

What rotation is associated with oblimim rotation

A

Oblique rotation

44
Q

What is an oblique rotation?

A

Maintains some relationship between the factors-clusters are different but related things

  • Used to minimize the squared loading covariance of the factors, while allowing them to be correlated
  • Don’t have to look much for eigenvalues anymore
45
Q

What do we expect to be the force on our measurement?

A

Only latent variable, other influences probably just random error

46
Q

What does PCA stand for?

A

Principal components analysis

47
Q

In PCA, what does it mean when we say that we expect all variance to come from a common source among the items?

A
  • Means there is no choice of extraction method (one way to identify cluster only) as these options under EFA are based on different ideas of the sources of variance
  • Quite possible for some of the common variance to still be error; accordingly we still want to discard any factors that seem too trivial on the basis of their eigenvalues
  • Everything is essentially meaningful until we look at eigenvalues
48
Q

Is PCA considered factor analysis? Why?

A

No

Because in general, don’t produce meaningfully different results as they are still doing essentially the same thing

49
Q

When should PCA be used?

A

If you want to maximize the amount of explained variance

50
Q

If someone asked you to do a factor analysis what should you do?

A

An exploratory factor analysis

51
Q

What does a confirmatory factor analysis allow for?

A

Allows complete control over which items theoretically go with which latent variable; with the right software, you could even specify expectations for how strong the relations should be. Tell the software exactly what we want

52
Q

What are the steps in the CFA process?

A

1-Need for a prior theory and check that theory
2-Input the model in Jamovi
3-Examine the model fit
4-If good fit, report the findings. If not, need to revise our model or our theory

53
Q

What is considered a good CFI and TLI as model statistics?

A

above .95 and everything below .9 is really bad

54
Q

What is considered a good RMSEA?

A

Want it to be below .05

55
Q

To be significant, what should be the p value of the factor loadings?

A

Below .05

56
Q

What is a disadvantage of an EFAr?

A

Allows very little control over the factor structure; theory can be introduced only through rotation or forcing a set number of factors

57
Q

What does a CFA tell us?

A

If we met our theoretical expectations

58
Q

If we are missing some data, what do we need to do in our CFA?

A

Need to calculate the mean for our calculations

59
Q

What is the Hierarchical Factor Analysis?

A

Model where you have one or more latent variable causing changes in one or more latent variable
o You have a primary latent variable that influences the other latent variables
o If common cause, will be highly correlated

60
Q

Why is it relevant to measure more than one factor?

A

Because in questionnaires, there can be more than one factor that we want to measure
Ex:
o A test for a course may not just capture one construct (reliability, validity, factor analysis)
o An IQ test captures G (overall intelligence) but it also has multiple components within it so people will vary, making distinct units
o A personality test assesses many aspects of personality

61
Q

What are the 5 factors of the 5-factor model?

A
  • Openness
  • Conscientiousness
  • Extraversion
  • Agreeableness
  • Neuroticism
62
Q

How do we know if each factor of personality is different from one another?

A

If the correlations are significant in big enough samples

63
Q

How much variance does the G factor explain in personality?

A

60%

64
Q

Do we have a hierarchal model in personality?

A

No, close but we still have 2 factors

65
Q

What is the other theory than the Big 5 to measure personality?

A

Stability (alpha, N, C, A) and plasticity (beta, E&O) also influencing personality so General factor for personality and these 2 are big factors as well

66
Q

How do we know which one makes the most sense whether we have a personality or not?

A
  1. One overall factor must account for a “great” amount of variance
  2. Any other factors should be more marginal
  3. The OCEAN subscales must relate strongly to the Big One
67
Q

What is the Big One in personality?

A

Behavioural characteristic of being emotionally stable, agreeable, conscientious, extraverted, and intellectually open individuals in comparison to neurotic, disagreeable, careless, introverted and close-minded persons

68
Q

What is the Big two in personality?

A

Combination of high versus low stability and plasticity where stability is somewhat more important than plasticity

69
Q

What is the meaning of the word standardized?

A
  1. Statistical manipulation where we convert our measurements to Z-score units prior to using them in an analysis (as done for r). Unstandardized means original units
  2. Means a measure has an expected set of procedures or that the items on a questionnaire have been previously established and we didn’t change them, that a measure has established normative information available, or some combination of all these. Unstandardized means one or more of these is missing.
70
Q

Could emotional intelligence also measure the Big One?

A

Yes

71
Q

What do we mean when we say that standardized measures are proprietary?

A

That we have to pay for them

72
Q

What’s the difference between a standardized and unstandardized measure?

A

Standardized: Controlled administration conditions, normative information available, interpretation guidelines available, instruction manual available, carefully vetted (?) and probably costly

Unstandardized: May not require controlled conditions, typically lack normative information, less interpretable for participants, mainly useful for research purposes, can have equivalent or better reliability and validity

73
Q

Are standardized measures better? Why or why not? Use an example from class

A

No, it’s not (ex: (free personality questionnaires are more reliable and efficient than the paid variety): Hamby, Taylor, Snowden & Peterson)
• Considered the possibility that setting a cost for something doesn’t necessarily mean it’s better than something free
• Argued that, for measuring personality, free tests are better: more reliable and efficient (validity not assessed= problem)

74
Q

What are the problems we can encounter if we don’t evaluate the psychometric properties of our tasks?

A

o Making less-than-optimal design decisions (e.g., power, task selected)
o Failure to find predicted results
o Incorrectly interpreting results

75
Q

Which Greek philosopher would have said that the unexamined measure should not be measured?

A

Socrates

76
Q

What is the AX-CTP?

A

A cognitive task used to assess individual differences in cognitive control ability
The ability to actively maintain and use goal-directed information to regulate behaviour

77
Q

What are the 2 forms in the Dual Mechanisms of Control?

A
  • Proactive: Control is implemented in advance, through maintenance of contextual information
  • Reactive: Control is implemented after an important event occurs
  • Cooper, Gonthier, Barch & Braver (2017) examined the psychometrics of this task (believed that you can find stable measurable differences in proactive or reactive in responding)
78
Q

What are 3 main qualities to address cognitive tasks such as the AX-CTP?

A
  1. Discrimination: Scores aren’t entirely random (e.g., high/low performance by some individuals)
  2. Reliability: Scores are a precise estimate of one’s ability
  3. Validity: Scores reflect the right cognitive faculty
79
Q

Can Inter-group biases affect task performance in undesirable ways?

A

Yes
Ex:
• Neurotypical young adults are strongly biased toward proactive control, setting performance at ceiling for all but AY trials (problem with understanding the differences between those groups)
• Reliability suffers in this case, but as the task was designed for clinical diagnostic purposes

80
Q

What does proper measurement need?

A

o Need good theories
o Need to capture all our dimensions well
o Need reliability
o Need convergent validity (ideally also discriminant validity)

81
Q

In the CES-D, why is content validity one of the main problems? And what effect can it have?

A

o CES-D has 20 items and 1/3 of them do not appear in any other commonly used measures of depression (shows disagreement)
o What effect can this have? Measuring something else in different studies so inconsistencies and affects replicability.
o Researchers could also engage in p hacking, only considering one correlation rather than all of the others that did not match expectations

82
Q

In the CES-D, why is reliability a problem?

A

• Inter-reliability for major depression diagnoses was .28 (presumably this is an r)
• Cronbach’s alpha has many limitations but in most cases is the only statistic ever used to demonstrate reliability
o At least 20% of studies don’t even report alpha
• Can have poor construct validity and high reliability

83
Q

In the CES-D, why is validity a problem?

A

• Much worse when looking at validity or rather not looking at it
• Adding or removing items is a common practice without providing a clear reason for doing so
o Changing items isn’t necessarily a bad thing; the lack of justification is
• Many studies don’t try to demonstrate validity and many that do simply provide a citation to some other study

84
Q

Why is critical thinking important?

A
  • To solve problems, need to increase knowledge about the importance of good measurement practices and be more critical of studies that ignore them
  • Alpha can’t tell you anything about validity when it’s high (is internal consistency best way to assess validity)
85
Q

What’s the difference between a test and a questionnaire?

A

Test: Items have right or wrong answers
Questionnaires: Do not (except with respect to the truth of the responses for that individual).

86
Q

Why might internal consistency reliability not make much sense for tests?

A

Depends on how many factors we expect to capture using the test, and how strongly correlated the factors are

87
Q

What is discrimination?

A

Ability to accurately separate individuals into high/low performers

88
Q

What is difficulty?

A

Likelihood of getting a question correct

89
Q

What is the formula for discrimination?

A

Discrimination= (Ph/Nh)-(Pl/Nl)
Ph=People responding correctly (High performing group)
Pl=People responding correctly (Low performing group)
Nh=Number of people in group (H)
Nl=Number of people in group

90
Q

What are the minimum and maximum discrimination scores?

A

Min: -1 (really bad)
Max: 1 (really good)
0=bad too

91
Q

What is the goal of discrimination?

A

Goal of discrimination is good categorization of individual cases into high and low performers

o Means we ideally have approximately half of people getting items wrong and we want to be able to predict in which half an individual falls with relatively good accuracy- for each item
o So, there is typically a connection between discrimination ability and item difficulty

92
Q

What is the formula for difficulty?

A

Difficulty= 1-(Correct/N)

93
Q

What are the minimum and maximum discrimination scores? What’s the ideal?

A

Min=0, Max=1, Ideal=0.5

94
Q

In designing a test, should I want to maximize the variability in the scores (full range of possible responses)?

A

Yes!

Means you should aim for most items to have a difficulty score near the optimal .50

95
Q

With optimal difficulty, do you optimize the probability of getting a high discrimination score?

A

Yes but it is not certain

Means your item can separate good performers from bad performers

96
Q

What psychological considerations should be taken into account for difficulty?

A

People inclined to want to do well so might or might not be a concern
Sometimes, need to sacrifice some psychometric quality to accommodate psychological considerations

97
Q

What can help us resolve the contrast between good psychometrics and psychological considerations? If you can, explain how it works.

A

Standardization!
• In this case, standardization means taking the raw scores which are likely very low and converting them into a new score that looks more acceptable
o Percentiles would be one way of doing it
o IQ scores typically use standardization via Z scores, where a Z of 0 becomes 100 (a psychologically pleasing number) and each Z difference of 1 adds or subtracts 15 points.

98
Q

What is mindfulness and what scale measures it?

A

Mindfulness is an active open attention to the present moment, also involving non-judgmental thinking about our experiences and a clear awareness of what is happening at any moment

The Measure: The Mindful Attention Awareness Scale (MAAS)

99
Q

What is the core construct of mindfulness?

A

State of consciousness

Attention can vary from person-to-person

100
Q

What steps did Brown and Ryan take to make a good scale/questionnaire to test mindfulness?

A

1-Started with large pool of questions
2-Experts reviewed and dropped some questions not related to constructs (calmness, wellbeing and patience)
3-Removed items with specific vocabulary and refined scale so that they could apply to general population
4-Shorter list reviewed by experts again
5-Several studies conducted to test scale
6-Assessed the validity of their questionnaire which showed:
• Positive correlations with openness to experience, emotional intelligence, and wellbeing
• Negative correlations with rumination and social anxiety
• A lack of association with self-monitoring

101
Q

In the end, what’s the problem with the MAAS scale?

A

Measured only one facet out of the 5 of mindfulness which is lack of attention.
If want to measure that aspect, it is a good scale otherwise not really

102
Q

Which scale was created to measure mindfulness taking inspiration from the MAAS scale?

A

The FFMQ measures the 5 factors of mindfulness: nonreactivity, observing, awareness, describing, nonjudgement

Psychometrics (1 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions