Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

Sampling and Measurement > Measurement process and measuring behaviour > Flashcards

Measurement process and measuring behaviour Flashcards

1
Q

What is Measurement?

A

Measurement is the assignment of values to outcomes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the 3 measurement principles?

A
  1. An Outcome variable belongs to one of four levels
  2. The qualities of one level are also characteristic of the next level
  3. The higher the level, the more precise the measurement
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Why are levels of measurement important?

A

Your IV and DV need to be defined as either four levels. Then you can determine the method by which you will measure them. Every variable studied must be operationally defined.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the 4 levels of measurement? Explain each

A
  1. Nominal
  2. Ordinal
  3. Interval
  4. Ratio
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is a discrete variable?

A

Values that have definite boundaries and can have nothing in between two values (number of students enrolled in a unit). All qualitative variables are discrete and are referred to as categorical variables (male and female).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is a continuous variable?

A

Continuous variables can assume any value on some scale and it is always theoretically possible for two values to have something in between (eg time, weight, height)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

A measure has high internal consistency reliability when

A

each of the items correlates with other items on the measure.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

More information can increase the _____ and ______ utility of your results

A

Power and utility

Always consider defining your variables in ways that maximises utility of information

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

In terms of information, higher level measurements have what properties?

A

They have more information about the true outcome of interest along the info/complexity scale

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

While behavioural and social science deals with mostly nominal and ordinal level data, most test score yield ____ level data?

A

Interval (caution)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How you choose to measure an outcome defines the ______

A

outcomes level of measurement (eg preference for a product measured in multiple ways)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is measurement error?

A

The discrepancy between the data found and the true value of measurement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What could account for measurement error?

A

Method error (the method, tools used)
Trait error (person themselves, the participants)
Temporary individual factors (fatigue, motivation, health)
Test administration (conditions, interaction between participant and examiner)
Luck

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

How can we decrease measurement error?

A

Increase reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How can we increase measurement reliability?

A
  1. Increase number of items/observations
  2. Eliminate ambiguity
  3. standardise conditions
  4. moderate difficulty
  5. minimise effects of external events
  6. standardise instructions
  7. standardise scoring
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is a correlation coefficient? Also known as Pearson correlation coefficient, Pearson’s r, the Pearson product-moment correlation coefficient (PPMCC)

A

The correlation coefficient is a statistical measure that calculates the strength and direction of the relationship between two variables. It provides a form of reliability.
It is represented by a number between -1 and +1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What are 4 types of reliability?

A
  1. Test-retest (measure of stability over time)
  2. Parallel forms (different forms of same test given to same participants)
  3. Interrater-Reliability (multilple raters agree in their observations of same thing)
  4. Internal consistency (responses at one time, focusses on consistency of items)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are 4 types of validity?

A
  1. Face validity - extent to which items on a test appear to measure the construct
  2. Content validity - extent to which the content of the measure compares with the universe of content that defines the construct
  3. Criterion-related validity - (predictive OR concurrent) extent to which a score indicates a level of performance on an criterion against which it is compared
  4. construct validity - extent to which an assessment corresponds to other variables as predicted by a theory
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are two types of Criterion-related validity?

A

Predictive and concurrent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are two types of construct validity?

A

Convergent validity and discriminant validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does internal validity refer to?

A

Internal validity refers to whether an experimental treatment / condition makes a difference or not, and whether there’s sufficient evidence to support the claim. It refers to the amount of control and accuracy in concluding that the outcome of an experiment is due to the independent variable.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What does external validity refer to?

A

Variables have been operationalised and defined and are representative of the population. It refers to the amount of generalisability.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are some threats to internal validity?

A
  1. history
  2. maturation
  3. testing
  4. instrumentation
  5. statistical regression
  6. selection of subjects
  7. mortality
  8. experimenter bias
  9. demand characteristics
    * Remember John Henry effect
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What are some threats to external validity?

A
  1. multiple treatments interference - treatments occur simultaneously
  2. reactive arrangements (participants knowledge of the experiment)
  3. experimenter effects
  4. pretest sensitisation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
How can we improve internal validity?
Randomly selecting individuals randomly assigning to groups use a control group
26
How can we improve external validity?
Careful adherence to good experimental process and practices Improve the research design random assignment attempt to normalise testing procedures and environment as a much as possible Validation studies
27
What are two types of sampling strategies?
Probability and | Non-probability sampling
28
What are the four types of probability sampling strategies?
1. Simple random 2. Systematic 3. Stratified 4. Cluster
29
What differentiates probability from non-probability sampling?
Probability sampling - likelihood of any one member of the population being selected is known Non-probability sampling - likelihood of selecting one member from the population is NOT known
30
What are the two types of non-probability sampling strategies?
Convenience | Quota
31
Why is sample size important?
You need to have a sample representative of the population - less representativeness means more margin for error, and the less precise your test of the null hypothesis is. Size has implications for the power and integrity of your test; its' sensitivity in detecting a significant result
32
What is the John Henry effect?
The tendency for members of a controlled group to adopt competitive attitude towards the experimental group thereby negating their status as controls
33
What is the Simpson's paradox?
A trend or result that is present when data is put into group that reverses or disappears when the data is combined.
34
What is the Hawthorne effect?
Type of reactivity in which individuals modify aspects of their behaviour in response to being observed. Can undermine integrity of research particularly the relationship between variables
35
The items in a personality test correlate strongly with one another. What kind of reliability or validity does this imply?
Internal consistency
36
What is incremental validity?
A type of validity that is used to determine whether a new psychometric assessment will increase the predictive ability beyond that provided by an existing method of assessment.
37
Why is your research question so important?
The way you frame it determines the way in which you go about measuring your variables. ie the intent of the research.
38
What is a test?
A tool that assess behaviour, albeit generally. It measures the extent of individual differences. a good test will differentiate people from one another reliably, based on their true score
39
Why is the interpretation of a score more important than the score itself?
Ie. A score of 10 on an exam wherein all the items are simple vs a score of 10 where everyone else in the group received scores below 5. It has to be compared to something or analysed so that it can have meaning
40
What are some ways in which tests can be administered?
Group vs individual Paper&Pencil vs performance Speed vs Power
41
What are achievement tests?
Achievement tests measure knowledge of a specific area; most commonly where learning is the outcome
42
What are the two types of achievement tests?
Standardised (WAIS) | Researcher-generated (custom made for a research problem)
43
Both types of achievement tests can be either _____ referenced or _____ referenced.
Norm-referenced (compared to others, Raven's progressive matrices) Criterion- referenced (driving test - need to obtain __% to pass, not compared to others)
44
Multiple choice achievement items are comprised of what elements..?
Stem Distractors Alternatives
45
List some advantages and disadvantages of multiple choice tests
``` Advantages: Ideal for assessing level of knowledge about a specific content domain assess any content easy to score tests for knowledge good distractors help in diagnosis ``` Disadvantages: Does not test for writing skills Presence of test anxiety
46
Why is using varied approaches to measuring behaviour beneficial?
Varied approaches assist in trying to falsify a theory or strengthen it. Can obtain convergent evidence (Or divergent)
47
What is Item Discrimination?
The ability of an item to differentiate among students on the basis of how well they know the material being tested. ie how well does it discriminate?
48
What is item analysis? | Why is this important?
Item analysis generates two indices which assess the effectiveness of a multiple choice test. It assist test authors asses the value of each item and decide whether it should be retained or replaced
49
What are the two indices we can analyse our test items on? How are they measured?
1. Difficulty index (range of difficulty). - the proportion of test takers who got the item correct is calculated 2. Discrimination index (how well does it discriminate between people?) - the proportion of test takers in upper group who got it correct, compared to those in lower who did
50
What happens when difficulty (within tests) increases?
As difficulty increases, discrimination is constrained
51
What are some types of tests?
``` Achievement (assessing content knowledge) Attitude Personality Intelligence Aptitude ```
52
What are 3 types of attitude tests?
``` Thurstone (favourable to unfavourable ranking of statements) Likert-type scale (statements assessed from strongly agree to strongly disagree, neutral in middle) Guttman scale (responses ordered from weaker to stronger) ```
53
What are two types of personality tests?
Projective (Thematic Apperception Test, Kinetic figure drawing, Rorschach technique) Structured (Big 5 personality)
54
What are two general ways we can we measure behaviour?
via tests via observing behaviour via questionnaires
55
What underlying notion does observing behaviour rely upon?
That given the same context, most people will behave in same/similar ways
56
What are two early examples of pioneer research into observing human behaviour?
Social Psychology - The Bystander Effect | Developmental Psychology - Mary Ainsworth - the strange situation
57
List some types of observational methods?
Systematic (full structure with coding system) | Naturalistic (field) observation
58
What are three types of naturalistic observation?
Full participant (researcher doesn't disclose) Participant as observer (researcher isn't a secret but kept quiet) Observer as participant (reliant on group members accepting observer present and over time they exhibit 'normal' behaviour)
59
What are some advantages/disadvantages to systematic observation?
Advantages - systematic, replication, high degree of reliability, more controlled Disadvantages - lack of ecological validity and behavioural spontaneity/realism
60
What are some advantages/disadvantages of naturalistic (field) observation?
Advantages - ecological validity, less subject to demand characteristics Disadvantages - poor control and replication difficult, greater potential for observer bias, ethics
61
Prior to observing behaviour, what are some key things you need to determine?
Decide on behavioural categories Define behaviours (ie on basis of form or consequence) What aspect are you measuring? Latency, frequency, duration Who will you observe and when? (continuous - for individual, or time-sampling - for groups)
62
What is measurement reliability?
The extent to which measurements differ from one occasion to occasion as a function of measurement error
63
What is a Reliability Coefficient?
Reliability coefficient is a measure of the accuracy of a test obtained by measuring the same individuals twice and computing the correlation of the two sets of measures.
64
What are the examples of two different DEGREES of reliability?
Reliability coefficient | Index of Concordance
65
Correlation, reliability and observers - comment...
We can compare people to see if they observed the same behavioural category with the same frequency. If they correlate highly (0.7-0.8) then we can say it is reliable.
66
Why are questionnaire valuable?
They allow collection of data from large numbers of people They can measure something that is not directly observable theoretical construct (ie cognitive or personality constructs) Captures beliefs, opinions etc
67
What is the jingle-jangle fallacy?
Jingle-jangle fallacies refer to the erroneous assumptions that two different things are the same because they bear the same name or that two identical or almost identical things are different because they are labeled differently
68
What should you be aware of if you want to devise your own questionnaire?
be aware of jingles and jangles | Be aware of response acquiescence and social desirability
69
What are the two types of questions in Questionnaires?
Open and closed

Sampling and Measurement (1 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions