Lecture 2 Flashcards
Validity of measurement
Research question of characteristic of media content
Each character requires valid measurement
Measurement validity: substantive accuracy of a measure - degree to which a measure covers a concepts content (reality, meaning) correctly and fully
Accurate measurement is harder for more latent characteristics
Manifest content
Directly observable (concrete) on its surface
Unambiguous meaning
Simple operationalisation
Single-item measure
Latent content
Not directly observable (abstract) below the surface must be revealed
Deeper ambiguous meaning
Multiple operationalisation - indicators
Measurement instrument with multiple items
The more latent a characteristic of media content:
The more meaningful and influential the context
The more meaning is judged constructed instead of observed by the coder
The more its meaning becomes dependent in pre existing mental schemes (frames) of the coder - projection
The more the focus of meaning making shifts from content to coder
More subjective (potential problem)
Manifest
AI or specific term (gender of actor) // middle is frames // latent: beauty
Intersubjectivity
Strong social norm
Operationalisation of latent media content (constructs)
/ Translation into a set of more manifest characteristics of content (indicators)
/ To break up -> to objectify quantitative
/ Indicators items combined must be covered the construct fully
Natural starting point:
Existing literature
Theoretical insights and empirical evidence
THeory must lead if it does not exist
Empirical ways to evaluate validity of measurement
(Instrument - validation)
Factor analysis
Construct validity (dichotomous in CA often - negative: they are not good for factor analysis)
Association between measurement instrument and:
Other version of same type of measurement (concurrent validity)
Plausible outcome variable (predictive validity)
Comparison outcomes with those of very different instruments (convergent validity)
(cross check among producers or consumers of content)
Social validity:
The social significance and meaning of the analysed media content beyond the academic community
/ among intended receivers of the content
/ combination of convergent and ecological validity: are researchers and users on the same substantive frequency?
Understand differently / audience understand in different ways than researchers
Should researchers determine the standard for meaning of content? Theoretical question
Is the chosen standard representative for that of the receivers of the content? Empirical question
Wider community beyond academic society?
Validity vs reliability of measurement
Reliability is a criterion for evaluating the quality of measurement
Agreement about categorising media content (among coders)
Reliability is a necessary but not sufficient condition for validity (reliability is a precondition for validity - no reliability > no validity)
Complication: In CA> both criteria of measurement quality are in tension with one another / raising one level might come on expense on of the other
Reliability» provide more instructions (1)
More coding rules
More detailed coding rules
Do more coder training
Creator mental focus on the rules
Consequence> coders will approach media content less naturally - like normal users
The social validity (ecological validity) of the measurement is impaired
(2) Simplify coding task:
Stronger focus on easiest observable aspects of latent
Remove difficult coding decisions
Reduce number of coding options / negative or positive
Consequence: The content reality of measured concept is too reduced
Content validity of measurement is impaired
Be realistic: a perfectly reliable and valid measurement instrument for latent media is impossible
Don’t fixate on just one criterion:
It’s self-defeating: a measurement must always satisfy both criteria: keep an eye on both
Content analysis still tends to be less attentive to validity
Don’t underestimate validity - matters more than reliability. Look for balance!
Coding process
Codebook: document containing all information relevant to the process of coding media content:
All variables to be coded (coding categories) for the study - full measurement instrument
Instruction to coders about when and how to code variables
Including coding rule and procedures that need to be followed systematically (quantitative)
Registration unit (observation unit):
Unit you judge when you code a variable - what is the unit level you make a decision about when you code a variable?
Reg unit can be: Textual media content: entire publication, web page, article, paragraph, sentence, word etc
Audiovisual media content (reg unit):
Entire production - movie, act, sequence, scene, shot etc
Reg unit
The choice of reg unit depends on the required level of detail for the study: more detail = more coding work!
Researchers can measure variables at different levels of registration within the same study (article level, or lower level)
Context unit:
Part of the content you take into consideration when you code a variable for each registration unit
Distinguished from reg unit only if the broader physical or temporal context of the media content you judge might influence the interpretation of users of that content > validity.
Mainly an issue with small reg units. Smaller the units, greater the importance of context
For large reg units: news article, often non-issue: reg and content coincide. (no broader context to take into consideration)
Does the photograph depict a human expression of sadness? Is it actually that? Broader context changed our interpretation on how to code the reg unit.
Reg unit should also include a caption.
