Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Sampling and Measurement > Measurement process and measuring behaviour > Flashcards

Measurement process and measuring behaviour Flashcards

1
Q

What is Measurement?

A

Measurement is the assignment of values to outcomes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the 3 measurement principles?

A
  1. An Outcome variable belongs to one of four levels
  2. The qualities of one level are also characteristic of the next level
  3. The higher the level, the more precise the measurement
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Why are levels of measurement important?

A

Your IV and DV need to be defined as either four levels. Then you can determine the method by which you will measure them. Every variable studied must be operationally defined.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the 4 levels of measurement? Explain each

A
  1. Nominal
  2. Ordinal
  3. Interval
  4. Ratio
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is a discrete variable?

A

Values that have definite boundaries and can have nothing in between two values (number of students enrolled in a unit). All qualitative variables are discrete and are referred to as categorical variables (male and female).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is a continuous variable?

A

Continuous variables can assume any value on some scale and it is always theoretically possible for two values to have something in between (eg time, weight, height)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

A measure has high internal consistency reliability when

A

each of the items correlates with other items on the measure.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

More information can increase the _____ and ______ utility of your results

A

Power and utility

Always consider defining your variables in ways that maximises utility of information

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

In terms of information, higher level measurements have what properties?

A

They have more information about the true outcome of interest along the info/complexity scale

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

While behavioural and social science deals with mostly nominal and ordinal level data, most test score yield ____ level data?

A

Interval (caution)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

How you choose to measure an outcome defines the ______

A

outcomes level of measurement (eg preference for a product measured in multiple ways)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is measurement error?

A

The discrepancy between the data found and the true value of measurement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What could account for measurement error?

A

Method error (the method, tools used)
Trait error (person themselves, the participants)
Temporary individual factors (fatigue, motivation, health)
Test administration (conditions, interaction between participant and examiner)
Luck

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

How can we decrease measurement error?

A

Increase reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How can we increase measurement reliability?

A
  1. Increase number of items/observations
  2. Eliminate ambiguity
  3. standardise conditions
  4. moderate difficulty
  5. minimise effects of external events
  6. standardise instructions
  7. standardise scoring
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is a correlation coefficient? Also known as Pearson correlation coefficient, Pearson’s r, the Pearson product-moment correlation coefficient (PPMCC)

A

The correlation coefficient is a statistical measure that calculates the strength and direction of the relationship between two variables. It provides a form of reliability.
It is represented by a number between -1 and +1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What are 4 types of reliability?

A
  1. Test-retest (measure of stability over time)
  2. Parallel forms (different forms of same test given to same participants)
  3. Interrater-Reliability (multilple raters agree in their observations of same thing)
  4. Internal consistency (responses at one time, focusses on consistency of items)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are 4 types of validity?

A
  1. Face validity - extent to which items on a test appear to measure the construct
  2. Content validity - extent to which the content of the measure compares with the universe of content that defines the construct
  3. Criterion-related validity - (predictive OR concurrent) extent to which a score indicates a level of performance on an criterion against which it is compared
  4. construct validity - extent to which an assessment corresponds to other variables as predicted by a theory
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are two types of Criterion-related validity?

A

Predictive and concurrent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are two types of construct validity?

A

Convergent validity and discriminant validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does internal validity refer to?

A

Internal validity refers to whether an experimental treatment / condition makes a difference or not, and whether there’s sufficient evidence to support the claim. It refers to the amount of control and accuracy in concluding that the outcome of an experiment is due to the independent variable.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What does external validity refer to?

A

Variables have been operationalised and defined and are representative of the population. It refers to the amount of generalisability.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are some threats to internal validity?

A
  1. history
  2. maturation
  3. testing
  4. instrumentation
  5. statistical regression
  6. selection of subjects
  7. mortality
  8. experimenter bias
  9. demand characteristics
    * Remember John Henry effect
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What are some threats to external validity?

A
  1. multiple treatments interference - treatments occur simultaneously
  2. reactive arrangements (participants knowledge of the experiment)
  3. experimenter effects
  4. pretest sensitisation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

How can we improve internal validity?

A

Randomly selecting individuals
randomly assigning to groups
use a control group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

How can we improve external validity?

A

Careful adherence to good experimental process and practices
Improve the research design
random assignment
attempt to normalise testing procedures and environment as a much as possible
Validation studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

What are two types of sampling strategies?

A

Probability and

Non-probability sampling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What are the four types of probability sampling strategies?

A
  1. Simple random
  2. Systematic
  3. Stratified
  4. Cluster
29
Q

What differentiates probability from non-probability sampling?

A

Probability sampling - likelihood of any one member of the population being selected is known
Non-probability sampling - likelihood of selecting one member from the population is NOT known

30
Q

What are the two types of non-probability sampling strategies?

A

Convenience

Quota

31
Q

Why is sample size important?

A

You need to have a sample representative of the population - less representativeness means more margin for error, and the less precise your test of the null hypothesis is.
Size has implications for the power and integrity of your test; its’ sensitivity in detecting a significant result

32
Q

What is the John Henry effect?

A

The tendency for members of a controlled group to adopt competitive attitude towards the experimental group thereby negating their status as controls

33
Q

What is the Simpson’s paradox?

A

A trend or result that is present when data is put into group that reverses or disappears when the data is combined.

34
Q

What is the Hawthorne effect?

A

Type of reactivity in which individuals modify aspects of their behaviour in response to being observed. Can undermine integrity of research particularly the relationship between variables

35
Q

The items in a personality test correlate strongly with one another. What kind of reliability or validity does this imply?

A

Internal consistency

36
Q

What is incremental validity?

A

A type of validity that is used to determine whether a new psychometric assessment will increase the predictive ability beyond that provided by an existing method of assessment.

37
Q

Why is your research question so important?

A

The way you frame it determines the way in which you go about measuring your variables. ie the intent of the research.

38
Q

What is a test?

A

A tool that assess behaviour, albeit generally.
It measures the extent of individual differences.
a good test will differentiate people from one another reliably, based on their true score

39
Q

Why is the interpretation of a score more important than the score itself?

A

Ie. A score of 10 on an exam wherein all the items are simple vs a score of 10 where everyone else in the group received scores below 5. It has to be compared to something or analysed so that it can have meaning

40
Q

What are some ways in which tests can be administered?

A

Group vs individual
Paper&Pencil vs performance
Speed vs Power

41
Q

What are achievement tests?

A

Achievement tests measure knowledge of a specific area; most commonly where learning is the outcome

42
Q

What are the two types of achievement tests?

A

Standardised (WAIS)

Researcher-generated (custom made for a research problem)

43
Q

Both types of achievement tests can be either _____ referenced or _____ referenced.

A

Norm-referenced (compared to others, Raven’s progressive matrices)
Criterion- referenced (driving test - need to obtain __% to pass, not compared to others)

44
Q

Multiple choice achievement items are comprised of what elements..?

A

Stem
Distractors
Alternatives

45
Q

List some advantages and disadvantages of multiple choice tests

A 
Advantages: 
Ideal for assessing level of knowledge about a specific content domain 
assess any content
easy to score 
tests for knowledge 
good distractors help in diagnosis

Disadvantages:
Does not test for writing skills
Presence of test anxiety

46
Q

Why is using varied approaches to measuring behaviour beneficial?

A

Varied approaches assist in trying to falsify a theory or strengthen it. Can obtain convergent evidence (Or divergent)

47
Q

What is Item Discrimination?

A

The ability of an item to differentiate among students on the basis of how well they know the material being tested.
ie how well does it discriminate?

48
Q

What is item analysis?

Why is this important?

A

Item analysis generates two indices which assess the effectiveness of a multiple choice test.

It assist test authors asses the value of each item and decide whether it should be retained or replaced

49
Q

What are the two indices we can analyse our test items on? How are they measured?

A
  1. Difficulty index (range of difficulty).
    - the proportion of test takers who got the item correct is calculated
  2. Discrimination index (how well does it discriminate between people?)
    - the proportion of test takers in upper group who got it correct, compared to those in lower who did
50
Q

What happens when difficulty (within tests) increases?

A

As difficulty increases, discrimination is constrained

51
Q

What are some types of tests?

A 
Achievement (assessing content knowledge) 
Attitude 
Personality 
Intelligence 
Aptitude
52
Q

What are 3 types of attitude tests?

A 
Thurstone (favourable to unfavourable ranking of statements)
Likert-type scale (statements assessed from strongly agree to strongly disagree, neutral in middle)
Guttman scale (responses ordered from weaker to stronger)
53
Q

What are two types of personality tests?

A

Projective (Thematic Apperception Test, Kinetic figure drawing, Rorschach technique)
Structured (Big 5 personality)

54
Q

What are two general ways we can we measure behaviour?

A

via tests
via observing behaviour
via questionnaires

55
Q

What underlying notion does observing behaviour rely upon?

A

That given the same context, most people will behave in same/similar ways

56
Q

What are two early examples of pioneer research into observing human behaviour?

A

Social Psychology - The Bystander Effect

Developmental Psychology - Mary Ainsworth - the strange situation

57
Q

List some types of observational methods?

A

Systematic (full structure with coding system)

Naturalistic (field) observation

58
Q

What are three types of naturalistic observation?

A

Full participant (researcher doesn’t disclose)
Participant as observer (researcher isn’t a secret but kept quiet)
Observer as participant (reliant on group members accepting observer present and over time they exhibit ‘normal’ behaviour)

59
Q

What are some advantages/disadvantages to systematic observation?

A

Advantages - systematic, replication, high degree of reliability, more controlled
Disadvantages - lack of ecological validity and behavioural spontaneity/realism

60
Q

What are some advantages/disadvantages of naturalistic (field) observation?

A

Advantages - ecological validity, less subject to demand characteristics
Disadvantages - poor control and replication difficult, greater potential for observer bias, ethics

61
Q

Prior to observing behaviour, what are some key things you need to determine?

A

Decide on behavioural categories
Define behaviours (ie on basis of form or consequence)
What aspect are you measuring? Latency, frequency, duration
Who will you observe and when? (continuous - for individual, or time-sampling - for groups)

62
Q

What is measurement reliability?

A

The extent to which measurements differ from one occasion to occasion as a function of measurement error

63
Q

What is a Reliability Coefficient?

A

Reliability coefficient is a measure of the accuracy of a test obtained by measuring the same individuals twice and computing the correlation of the two sets of measures.

64
Q

What are the examples of two different DEGREES of reliability?

A

Reliability coefficient

Index of Concordance

65
Q

Correlation, reliability and observers - comment…

A

We can compare people to see if they observed the same behavioural category with the same frequency. If they correlate highly (0.7-0.8) then we can say it is reliable.

66
Q

Why are questionnaire valuable?

A

They allow collection of data from large numbers of people
They can measure something that is not directly observable theoretical construct (ie cognitive or personality constructs)
Captures beliefs, opinions etc

67
Q

What is the jingle-jangle fallacy?

A

Jingle-jangle fallacies refer to the erroneous assumptions that two different things are the same because they bear the same name or that two identical or almost identical things are different because they are labeled differently

68
Q

What should you be aware of if you want to devise your own questionnaire?

A

be aware of jingles and jangles

Be aware of response acquiescence and social desirability

69
Q

What are the two types of questions in Questionnaires?

A

Open and closed

Sampling and Measurement (1 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions