Chapter 11: Validity and Reliability of a Research Instrument

Question 1

Instrument

Answer 1

term used for a measurement device

Question 2

Researcher-Completed Instruments

Answer 2

• rating scales
• interview schedules
• tally sheets
• flowcharts
• performance checklists
• observation forms

Question 3

Subject-Completed Instruments

Answer 3

• questionnaires
• self-checklists
• attitude scales
• personality inventories

Question 4

Usability

Answer 4

ease with which an instrument can be administered, interpreted, and scored

Question 5

Usability Considerations

Answer 5

• how long will it take to administer?
• are the directions clear?
• how easy is it to score?
• do equivalent forms exist?
• have any problems been reported by others who used it?

Question 6

Validity and Reliability in Research

Answer 6

• whether research instrument achieves truthfulness of the results

Question 7

Validity

Answer 7

• extent to which an instrument measures what it is supposed to measure and performance
• almost impossible for an instrument to be 100% valid

Question 8

Internal Validity

Answer 8

the extent to which you are able to say that no other variables except that ones you are studying caused the result

Question 9

External Validity

Answer 9

the extent to which results of a study can be generalized to the world at large

Question 10

Types of Validity

Answer 10

1. face validity
2. content validity
3. construct validity
4. criterion-related validity

Question 11

Face Validity

Answer 11

• least scientific method of validity as it is not quantified using statistical methods
• direct measurement is obtained by asking people to rate validity of a test as it appears to them
• only good if there is a reasonable agreement among the raters

Question 12

Content Validity

Answer 12

• subjective measure

- ask whether the content of a measure covers the full domain of a concept

Question 13

Construct Validity

Answer 13

• collection of behaviors associated in oder to create an imagine/idea invented for a research purpose
• extent to which a test captures a specific theoretical construct or trait, and it overlaps with some of the other aspects of validity

Question 14

Criterion-Related Validity: Concurrent Validity

Answer 14

• refers to the extent to which the result of a particular test or measurement correspond to those of a previously established measurement for the same construct

Question 15

Criterion-Related Validity: Predictive Validity

Answer 15

uses the scores from the new measure to predict performance on a criterion measure administered at a later time

Question 16

Reliability

Answer 16

degree to which an assessment tool produces stable results

Question 17

Test-Retest Reliability

Answer 17

• obtained by administering the same test twice over a period of time to a group of individuals
• scores from time 1 and time 2 can then be correlated in order to evaluate the test for stability over time

Question 18

Parallel Forms Reliability

Answer 18

• obtained by administering different versions of an assessment tool to the same group of individuals
• scores from the two versions can then be correlated in order to evaluate the consistency of the results across alternate versions

Question 19

Inter-Rater Reliability

Answer 19

• measure of reliability used to assess degree to which different judges or raters agree in their assessment decisions

Question 20

Internal Consistency Reliabiliy

Answer 20

measure of reliability used to evaluate the degree to which different test items that prove the same construct produce similar results

Question 21

Factors Affecting the Reliability of a Research Instrument

Answer 21

• wording of the questions
• physical setting
• respondent’s mood
• interviewer’s mood
• nature of interaction
• regression effect of an instrument

Question 22

Validity and Reliability in Qualitative Research: Traditional Criteria

Answer 22

• internal validity
• external validity
• reliability
• objectivity

Question 23

Validity and Reliability in Qualitative Research: Alternative Criteria

Answer 23

• credibility
• transferability
• dependability
• confirmability

Question 24

Qualitative Research: Credibility

Answer 24

• involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in research
• from this perspective the purpose of qualitative research is to describe or understand the phenomena

Question 25

Qualitative Research: Transferability

Answer 25

• degree to which the results of qualitative research can be generalized or transferred to other contexts or settings
• from a qualitative perspective transferability is primarily the responsibility of the one doing the generalizing

Question 26

Qualitative Research: Dependability

Answer 26

• need for the researcher to account for the ever-changing context within which research occurs
• researcher is responsible for describing the changes that occur in the setting and how these changes affected the way the research approached the study

Question 27

Qualitative Research: Confirmability

Answer 27

degree to which the results could be confirmed or corroborated by others