Chapter 11: Validity and Reliability of a Research Instrument Flashcards
Instrument
term used for a measurement device
Researcher-Completed Instruments
- rating scales
- interview schedules
- tally sheets
- flowcharts
- performance checklists
- observation forms
Subject-Completed Instruments
- questionnaires
- self-checklists
- attitude scales
- personality inventories
Usability
ease with which an instrument can be administered, interpreted, and scored
Usability Considerations
- how long will it take to administer?
- are the directions clear?
- how easy is it to score?
- do equivalent forms exist?
- have any problems been reported by others who used it?
Validity and Reliability in Research
- whether research instrument achieves truthfulness of the results
Validity
- extent to which an instrument measures what it is supposed to measure and performance
- almost impossible for an instrument to be 100% valid
Internal Validity
the extent to which you are able to say that no other variables except that ones you are studying caused the result
External Validity
the extent to which results of a study can be generalized to the world at large
Types of Validity
- face validity
- content validity
- construct validity
- criterion-related validity
Face Validity
- least scientific method of validity as it is not quantified using statistical methods
- direct measurement is obtained by asking people to rate validity of a test as it appears to them
- only good if there is a reasonable agreement among the raters
Content Validity
- subjective measure
- ask whether the content of a measure covers the full domain of a concept
Construct Validity
- collection of behaviors associated in oder to create an imagine/idea invented for a research purpose
- extent to which a test captures a specific theoretical construct or trait, and it overlaps with some of the other aspects of validity
Criterion-Related Validity: Concurrent Validity
- refers to the extent to which the result of a particular test or measurement correspond to those of a previously established measurement for the same construct
Criterion-Related Validity: Predictive Validity
uses the scores from the new measure to predict performance on a criterion measure administered at a later time
Reliability
degree to which an assessment tool produces stable results
Test-Retest Reliability
- obtained by administering the same test twice over a period of time to a group of individuals
- scores from time 1 and time 2 can then be correlated in order to evaluate the test for stability over time
Parallel Forms Reliability
- obtained by administering different versions of an assessment tool to the same group of individuals
- scores from the two versions can then be correlated in order to evaluate the consistency of the results across alternate versions
Inter-Rater Reliability
- measure of reliability used to assess degree to which different judges or raters agree in their assessment decisions
Internal Consistency Reliabiliy
measure of reliability used to evaluate the degree to which different test items that prove the same construct produce similar results
Factors Affecting the Reliability of a Research Instrument
- wording of the questions
- physical setting
- respondent’s mood
- interviewer’s mood
- nature of interaction
- regression effect of an instrument
Validity and Reliability in Qualitative Research: Traditional Criteria
- internal validity
- external validity
- reliability
- objectivity
Validity and Reliability in Qualitative Research: Alternative Criteria
- credibility
- transferability
- dependability
- confirmability
Qualitative Research: Credibility
- involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in research
- from this perspective the purpose of qualitative research is to describe or understand the phenomena
Qualitative Research: Transferability
- degree to which the results of qualitative research can be generalized or transferred to other contexts or settings
- from a qualitative perspective transferability is primarily the responsibility of the one doing the generalizing
Qualitative Research: Dependability
- need for the researcher to account for the ever-changing context within which research occurs
- researcher is responsible for describing the changes that occur in the setting and how these changes affected the way the research approached the study
Qualitative Research: Confirmability
degree to which the results could be confirmed or corroborated by others
