Evaluating Sources: Reliability & Validity Flashcards
Define Validity? What does validity of research refer to?
Property of particular statements, are they sound, legit, authoritative?
How well research carried out, are findings trustworthy, is research useful
Define Objectivity, what’s the issue? [3]
Unbiased observation
Data collection methods are ‘standardised’
BUT researcher influences production of knowledge
Define Reliability, what’s the issue? [3]
Predictability and causal relationships
Replicability in different contexts
BUT qualitative researchers often interested in effects of diff contexts
Define Generalisability, what’s the issue? [3]
Generalisability of findings from sample to population
BUT its seldom practical to gather qualitative data from large enough sample
Insights may prove useful in other contexts
What are the conventional and alternative for evaluation criteria?
Internal Validity > Credibility
External Validity > Transferability
Reliability > Dependability
Objectivity > Confirmability
What are the ways of enhancing Validity? [8]
ARMPIT TD Triangulation Member checks Peer debriefing Disconfirming case analysis Audit trail Immersion in the setting Thick description Reflexivity
Describe Enhance Validity Methods: Triangulation
Combination of methods in study of same phenomena to address weakness of any method with strength of another
Confirmability
Describe Enhance Validity Methods: Member checks [3]
Respondent validation
Participants comment on analysis
Challenge and expand
(not always feasible/practical)
Describe Enhance Validity Methods: Peer debriefing
Comparing researcher’s coding
More than one perspective for analysis
Inter-rater reliability and consistency of coding
Describe Enhance Validity Methods: Disconfirming Case Analysis [3]
In Qualitative analysis, disconfirming instances, deviant or negative cases
Identifying cases that do not fit themes
Report disconfirming cases
Describe Enhance Validity Methods: Audit Trail
Evidence linking raw data to final report
Re-traceability of steps
Recording decisions
Establishes transferability and confirmability
Describe Enhance Validity Methods: Immersion in the setting
Ethnography studies
Prolonged engagement
Research becomes less visible, behaviors observed more realistic
Describe Enhance Validity Methods: Thick Description
Hollistic description
Sufficient breadth and depth of analysis
Researcher competence in method of analysis
Sampling
Describe Enhance Validity Methods: Reflexivity
Personal Reflexivity: acknowledging who you are and how your personal interest and values influence research process
Acknowledging central role of researcher
Reflective journal (audit trail)
Can judge content of research in context of perspectives/assumptions it was made in
Important concepts in Evaluating sources? [3]
Sensitivity to context
Coherence and transparency
Impact & Performance
Briefly describe Important Concept: Sensitivity to Context
Holistic description
Literature review (theoretical & empirical)
Range of lit from diff disciplines
Context of participants (sociocultural, perspective)
Characteristics of researcher
Research Setting
Briefly describe Important Concept: Coherence and Transparency
Extent it makes sense as a whole
Fit between theoretical approach, research question, methods employed and interpretation of data
Interpretive qualitative study cannot test hypothesis
Transparency - how well reader can see what was done and why
Briefly describe Important Concept: Impact and Performance
Potential to make difference
Theoretical and Practical importance
