T3 Slide W5 Flashcards
1
Q
The Measurement Process
A
- What is the point of research if it can’t be measured
- Measurement is the assignment of values to outcomes
- How do we measure height?
2
Q
Principles of measurement in Research - 3 ideas
A
- an outcome variable belongs to one of four levels of measurement (Nominal, Ordinal, Interval, and Ratio)
- The qualities of one level, are also characteristic of the next level
- e.g., ratio measures such as height also capture ordinal information
- The higher the level, the more precise the measurement process, and closer you will be to the true outcome of interest.
3
Q
Levels of Measurement
A
- The relationship between what is being measured and the numbers that represent what is being measured
- Every variable must be operationally defined:
4
Q
Variables are Categorical or Continuous
A
- Categorical
- Names are distinct entities
- Simplest form is binary variable; can only go in one of two categories. eg male v female
- Continuous Variable
- Can take on any value on the measurement scale. eg: time on a stopwatch
5
Q
Levels of measurment in order of complexity
A
- Nominal
- Ordinal
- Interval
- Ratio
6
Q
Nominal Variable
A
- Nomin = name
- Differ in quality rather than quantity
- Characterises observations in a manner where they can only be placed in one category eg: eye colour
- May be given names or numbers but these have no intrinsic value. such as with NRL Jerseys
- Most IV’s are nominal
7
Q
Ordinal Variable
A
- Like nominal they permit classification tell us the order in which things have occurred
- Ordinal scales have no absolute zero point. ie: Horse racing
- Imply nothing about how much greater one ranking is than another
8
Q
Interval Variable
A
- Equal intervals on the scale represent equal differences in the value measured
- eg: temperature, although equal, be sure to consider interpretation of values along the scale.
9
Q
Ratio Variables
A
- Ratio meaning calculation
- Build on interval but also requires the ratios of values are meaningful
- Requires a true zero point not an arbitrary one
10
Q
Continuous variables are continuous or discrete
A
- Continuous = any level of precision such as time
- Discrete = certain defined values such as number of children in a family
11
Q
Categorical - Distinct Category
A
- Nominal Variable - more than two
- Ordinal variable - Same as nominal but a logical order ie: fail, pass, credit, distinction, high distinction.
12
Q
Continuous - Distinct Score
A
- Interval variable - equal entities represent equal difference
- Ratio variable - Same as interval but scores are meaningful ie: 50kg is twice as heavy as 25kg
13
Q
Levels of Measurement and complexity
A
- Nominal - Categorie
- Ordinal - orders
- Interval - meaningful distance
- Ratio - absolute zero
14
Q
Principles of measurement in research
A
- An outcome variable belongs to either nominal, ordinal, interval and ratio
- Characteristic of the next level eg: ratio measurements such as height also capture ordinal information
- The higher the level the more precise the result and closer you will be to the true outcome of interest
15
Q
Principles of Measurement in Research - Points to Ponder
A
- More information increases the power and utility of your results
- Sometimes you will be limited to what is available to you
- Always define your variables in ways that maximise the use of your information
- In behavioural and social sciences most data is usually nominal or ordinal however test scores yield interval level data
- How you choose to measure an outcome defines the level of measurement
- Variables may not completely fit this rigid framework in the real world
16
Q
Reliability and Validity
A
- You’re only as good as your tools
- You can have a great research question but will not succeed if your tools are unreliable
- The consistency and validity of a measurement tool are critical to good research
- Faulty tools lead to errors in accepting or rejecting the null hypothesis
17
Q
Reliability
A
- When measuring we assume that there will be a discrepancy found
- The True value of measurement
- Reliability decreases as error increases
- Reliability = True Scories
True Score + Error
18
Q
Ways to increase measurement reliability
A
- Increase number of items or observations
- Eliminate ambiguity
- Standardise conditions
- Moderate difficulty
- Minimise effects of external events
- Standardise instructions and Standardise scoring
19
Q
How to measure reliability
A
- We use correlation; a measure of relationships between things
- We can calculate a number that provides a gauge of relationship direction and strength
- Called Correlation Coefficient
20
Q
Correlation Coefficient
A
- This is a measure of the direction and extent of the relationship between two sets of scores.
- Range of a correlation coefficient is from -1 to +1
21
Q
Pearson’s r
A
- Pearson’s product moment correlation coefficient
- This coefficient will provide a gauge of how similar scores on a test are from time 1 to time 2
- This is one form of reliability
22
Q
Types of Reliability - Test-Retest
A
- A Measure of stability; how stable is a test over time,
- Measuring the same individuals at two points in time
23
Q
Operational Definition
A
The operational definition of a variable is the specific way in which it is measured in that study
24
Q
Name the different types of Reliability (4)
A
- Test-Retest
- Parallel Forms
- Interrater
- Internal Consistency
25
Q
Types of Reliability - Parallel Forms
A
- Different forms of the same test given to the same group of participants
- You might see this in a practice effects test
26
Q
Types of Reliability - Interrater
A
- Evidence of reliability when multiple raters agree in their observations of the same thing
- Rater to Rater, rather than time to time
eg: observational research
27
Q
Types of Reliability - Internal Consistency
A
- Uses responses at only one time
- Focusses on consistency of items
28
Q
Types of Reliability
A
- Test-Retest
- Parallel Forms
- Interrater
- Internal Consistency
29
Q
Measuring what we intend to . . .
A
- Our measues should be reliable and valid
- Validity refers to the results of the test
- It is never all or nothing
- Validity of the results interpreted in teh context where the test occurs
- Are the results understood within the context of the purpose of research?
30
Q
Name the Types of Validity
A
- Face Validity
- Content Validity
- Criterion-related Validity
- Construct Validity
31
Q
Types of Validity - Face Validity
A
- Extent to which items on a test appear to measure the construct
- Does it look like the items are asking relevant questions
- Will the test taker understand what is being measured
32
Q
Types of Validity - Content Validity
A
- The content of the measure compares with the universe of content that defines the construct
- are the items a representative sample of all possible items
33
Q
Types of Validity - Criterion-Related Validity
A
- A score indicates the level of performance on an external criterion against which it is compared.
- A measure of the extent to which a test is related to a criteria
- Two Types of Criterion Validity.
- Predictive - the future
- Concurrent - the present
eg: GPA predictive validity for performance in Honours?
- Beware the importance of identifying the criterion
34
Q
Types of Validity - Construct Validity
A
- An assessment corresponds to other variables , as predicted by some rationale or theory.
- Links the practical components of a test score to some underlying theory or model of behaviour
- Firstly, similar to criterion validity, you will look for correlations between your newly developed test and test already shown to tap the target construct
35
Q
Thoughts on construct validity
A
- If a measure has convergent validity, it should correlate with questionairres that measure:
- The same construct
- Related Constructs
- eg: Beck Depression Inventory II positivley associated with Hamilton Depression scale
- If a measure has discriminant validity it should not correlate questionairre that measure
- Different Constructs
- Unrelated Constructs
36
Q
The Relationship between Reliability and Validity
A
Reliability is a necessary but not sufficient condition of validity.
37
Q
Finding Reliability and Validity Information (7)
A
- Mental Measurements Yearbook
- Tests in Print
- Buros Institute of Mental Measurements - http://www.unl.edu/buros/
- Test Link – ETS Test Collection - http://www.ets.org/testcoll/index.html
- APA - http://www.apa.org/science/testing.html)
- PsycINFO
- Test manuals
38
Q
Sampling & Generalasibility
A
* Choose a research question and decide how to test it
- Decide who you will ask to participate in your research
- Vital Stage of researh development
- Who should I study
eg: I want to look at attitudes toward taking drugs in sport - Success of your research hinges on the way in which you select participants.
39
Q
Describe a Population
A
- Population = the collection of units to which we want to generalise our research findings.
- Populations can be large or narrow
- Generally, researchers aim to infer about general populations
- rare to access all members
- Data collected from a subset of the population is the Sample
40
Q
Describe a Sample
A
- A smaller collection of observations from a population that are used to infer characteristics about the population
- The bigger the sample the more likely it is an accurate reflection of the population
- Results vary slightly from sample to sample but tend to average out as similar
- Results have meaning when they can be generalised from sample to population
41
Q
Two Types of Sampling Strategies
A
- Probability Sampling
- Non Probability Sampling
42
Q
Probably Sampling
A
- The likelihood of any one member of population being selected is known
eg: If 4,000 students are at ACAP and 20 of them are in the Psychology Honours year, then the odds of selecting one Honours student as part of a sample is 20/4,000 → 0.005
43
Q
Non Probability Sampling
A
- The likelihood of selecting any one member from the population is not known
- If I don’t know how many Psychology Honours students are at ACAP, then likelihood of one being selected cannot be computed
44
Q
4 Types of Probability Sampling
A
- Simple Random Sampling
- Sustematic Sampling
- Stratified Sampling
- Cluster Sampling
45
Q
Simple Random Sampling
A
- Each member of the popluation has an equal chance of being selected to be part of the sample
- Equal - No bias that one individual will be chosen over another
- Independent - The choice of one individual does not create bias the reasearch nor is there bias in choosing another participant
46
Q
Four Steps of Simple Random Sampling
A
- Definition of the population
- List all members of that population
- Assign numbers to each members of the population
- Use of criterion to select sample that is wanted
47
Q
Systematic Sampling
A
- Every kth name on the list is chosen
- where k = any number 0 and the size of the sample to be selected 0 and the size of the sample to be selected.
- Easier than random sampling
- However
- less precise
- violates the assumption that each unit has equal chance of being selected
Procedure:
- We want a random sample of 8 names–Divide size of population by size of desired sample → 32 / 8 = 4
- 4 is the size of the step we want.
- Select one name from the list at random, and use this as your starting point (again, use table of random numbers)
- Then select every 4th name until a sample of 8 has been achieved
48
Q
Stratified Sampling
A
- Individuals in sample are NOT equal
and - these unequal characheristics are related to what is being studied in the research
- Can be done through SPSS
- Ensures profile of sample matches the population
- layers (or Strata) are fairly represented in the sample
49
Q
Stratified Sampling Procedure
A
- All males and females are listed separately
- Each member of each group receives a number
- From a table of random numbers, four males are selected at random from the list of 13 using the procedure outlined for simple random sampling
- From a table of random numbers, six females are selected at random from the list of 22 using the procedure outlined for simple random sampling
50
Q
Nonprobability Sampling Strategies
A
- Probability of selecting a single individual or unit is not known
- Impedes ability to generalise results obtained from sample to the larger population
- TWO primary types:
- Convenience
- Quota
51
Q
Convenience Sampling
A
- Uses a captive audience,
- i.e., much psychology research uses first year university students who must participate for credit
•THEREFORE:
- Sample is convenient, but not random
- Sears (1986) cautions a “A narrow database…”
52
Q
Quota Sampling
A
- Used where research requires stratified sample but proportional stratified sampling is not possible
- Selects individuals with characteristics desirable for the research but does not randomly select from the population a subset of all individuals
- Researcher continues to enlist participants in each strata until quota for research is achieved
- Easier than stratified sampling
- Less precise than stratified sampling
- Limits ability to generalise to population of interest
53
Q
Sample Size - How big is big?
A
- You want to have a sample representative of the population; less representative more error, and the less precise your test of the null hypothesis
- Sample size also has implications to the power of your test; its sensitivity in detecting a significant result….
