Measurement Flashcards
What are the ordinary concerns of measurement in business?
- Ensuring the units/individuals work on their Key Result Areas (KRA), where performance is measured using Key Performance Indicators (KPI)
- Securing that members possess and develop behaviours and competencies, or knowledge, skills, abilities and other characteristics (KSAOs)
What is measurement?
Measurement in the process of assigning numbers to objects in such a way that specific properties of objects are faithfully represented by the properties of numbers
What is operationalisation?
- The process of defining concepts into measurable factors
- The definition of a concept will affect the way we measure it
What are the different types of scales?
- Nominal Scale
- Ordinal Scale
- Interval
- Ratio
What are nominal scales?
- Categorical with no inherent order
- Assigns a value to an object for identification or classification purposes only
- Arbitrary scaling - each value can be assigned to any of the categories - size of the number tells nothing about the objects being measured
- Analysis involved counting and determining modal distribution
What are ordinal scales?
- Categorical with an inherent order/ranking
- Allows things to be arranged in order based on how much of some concept they possess
- Does not tell how close or far apart the objects are from each other
- Analysis can involve counting, measures of central tendency (mode, median and range)
What are interval scales?
- Indicates difference in values between points with no 0 point, therefore cannot say anything is double anything else etc
- Have both nominal and ordinal properties, but they also capture information about differcens in quantities of a concept
- Unit of measurement is arbitrary - scale does not exactly represent some phenomenon. Zero does not mean absence of the characteristic
- Temperature in F - 40 isn’t ‘double’ as hot as 20
What are ratio scales?
- Interval with an absolute zero
- Has all the properties of interval scale with an additional attribute of representing absolute quantities. Zero represents an absence of a concept
- Represent absolute meaning
- Analysed through counting, central tendency, variance and dispersion
What types of measures are there?
- Discrete measures
- Continuous measures
- Index measures
- Composite measures
What are discrete measures?
- Those that only take one of a finite number of values
- Most often used to represent a classification variable
- Nominal or ordinal measures
- Only statistical measure possible is the mode
- e.g. marital status, gender, ethnicity
What are continuous measures?
- those that reflect the intensity of a concept by assigning values that can take on any value along some scale range
- Mean and dispersion can be measured
- i.e ratio measures
- e.g. income, age
What are index measures?
- An index assigns a value based on how much of the concept being measured is associated with an observation
- Often are formed by putting several variables together
What are composite measures?
- Assign a value to an observation based on a mathematical derivation of multiple variables
- Summated scales
What are summated scales?
- Composite measures
- Reverse coding - deducting a variables instead of summing it where appropriate
What are the measures of the quality of measures?
- Reliability
- Validity
- Sensitivity
When is a measure reliable?
- A measure is reliable when it is accurate and different attempts at measuring something converge on the same result
- Scores vary because of variability in true scores or variability due to measurement errors
- If errors are responsible for much of the variability in the observed variance in test scores, they will be unreliable
What are sources of systematic error?
- Test characteristics and assessment conditions
- Perceptual errors
- Deliberate distortion & demand characteristics
- Characteristics of the test taker
What test characteristics and assessment conditions might cause systematic error?
- Poor test items
- Poor comprehension of test instructions
- Freedom from distractions
- Adherence to time limits
What is the goal in estimating reliability in measures?
Goal is to determine how much of the variability in test scores is due to errors in measurement
How can measure reliability be estimated?
- Alternate Forms
- Test-retest
- Internal consistency tools
How are alternative forms used to establish the reliability of a measure?
- More than one form of a test to measure the same concept is run
- Forms are administered to different groups to determine reliability coefficient, the extent to which the scores across the forms correlate
- Alternate test forms are difficult to develop and expensive to administer
- Difficult to establish that the measures are parallel
How is test-retest used to establish the reliability of a measure?
A questionnaire is used in a pilot test, then later, the same test is repeated and compared for consistency; administering the same scale or measure to the same respondents at two separate points in time to test for stability
What are the problems with using test-retest to establish reliability of a measure?
- Sensitised respondents
- Attitude change or maturation of subjects
- Time consuming
- Costly
How are internal consistency tools used to establish the reliability of a measure?
- Looks at the inter-correlations among items
- Internal consistency represents a measure’s homogeneity
What are the two steps to internal consistency tools in measuring reliability of a measure?
- Split half method
- Method of assessing internal consistency by checking the results of one half of a set of scaled items against the results from the other half: used to test equivalence reliability
- Coefficient alpha
- Compute the average of all possible split half reliability for a multiple item scale where the coefficient demonstrates whether or not the different items converge
- Represents the average of all possible split-half reliabilities for a construct
- 0.85-0.95 is very good reliability
How is validity related to measures?
- Of measurement
- The extent to which a measure represents the intended concept
- Of decisions
- The extent to which a measure can predict outcomes
What forms of validity are there in regard to measures?
- Face
- Content
- Criterion
- Construct
How does face validity relate to the quality of a measure?
- A scale’s content logically reflects what was intended to be measured
- Relies on census among experts that test items match definition
- On the surface the scale or test measures what it is supposed to measure
How does content validity relate to the quality of a measure?
- Ensures that the scale includes an adequate and represenative set of items that tap a concept
- The degree that a measure covers the breadth/domain of the concept
- Refers to how representative the measures are of the construct domain
How does criterion validity relate to the quality of a measure?
- The ability of the measure to correlate with established criteria
- How well does the measure work in practice? Is it practical?
- Is established when the measure differentiates individuals on criteroin it is expected to predict
- Represents measures of the outcomes that decisions are designed to produce, and thus the validity of the decisions
- Classified into either concurent or predictive validity,
distinguished based on time dimension
How does construct validity relate to the quality of a measure?
- The ability of a measure to reliability measure and truthfully represent a unique concept
- Refers to whether a test provides a valid measure of a specific attribute or concept
- Testifies to how well the result obtained from the use of the measure fit the theories around which the test is designed
- Consists of face, content, criertion, convergent and discriminant (independent) validity
What is sensitivity in regard to the quality of a measure?
- Refers to an insturments ability to accurately measure variability in a concept
- Increased by adding more response points or items
