Chapter 9 Flashcards - Qualitative Evaluation Approaches

Question 1

What is construct validity?

Answer 1

The extent to which the test accurately assesses what it’s supposed to

Question 2

Why can’t qualitative data use construct validity?

Answer 2

There are no primary data collection instruments used in qualitative which means we don’t have anything to give merit to in order to assess the test

Question 3

What is internal validity and why does qualitative have threats to internal validity

Answer 3

Internal validity: Extent to which a study measures the cause and effect relationship between variables with its own specific conditions

• Not implementing interventions or treatments to study participants leads to threats to internal validity

Question 4

What is trustworthiness about when it comes to qualitative evaluation approach?

Answer 4

Convincing an audience that study findings are worth paying attention to

4 Aspects of trustworthiness: Truth value, applicability, consistency, neutrality

Question 5

What is truth value?

Answer 5

Falls under trustworthiness and it is refering to the credibility of the study
* Results are believable from perspective of the participant
* Topic researched thoroughly enough to be an accurate representation of what participants have said, experiences, or viewpoints

Question 6

What is applicability?

Answer 6

Falls under trustworthiness -** transferability of the study**
* The extent to which findings are applicable/transferable to other contexts or participants
* Forming **understandings **that may be relevant from one context to the next depending on similarity between contexts

Question 7

How should applicability be viewed?

Answer 7

Not in isolation; always in the context of the other criteria of trustworthiness

Question 8

What is consistency?

Answer 8

Falls under trustworthiness and it is the dependability of the study
* Need to account for ever-changing context within which research occurs
* Constructivist paragidm that focuses on embracing diversity of participants perspectives while seeking patterns & shared meanings

Question 9

Goal of consistency?

Answer 9

Make the research process more rigorous and findings more dependable
* Understand variability of study findings

Question 10

What is neutrality?

Answer 10

Falls under trustworthiness and is confirmability of the study
* Essentially the degree to which findings of a study are based on participant experience and not bias of researchers

Question 11

What is important to understand about neutrality and researchers?

Answer 11

Researchers must contain their own biases and should consider how own experiences can play a role in the meaning-making process when conducting qualitative studies

Question 12

What’s true about trustworthiness and quantitative terminology

Answer 12

They are both similar with the terms they use but depends on the context of the study

E.g. Similarity of consistency (qualitative) and reliability (quantitative) - consistently finding same results may be good for quantitative but may not be purpose of qualitative

Question 13

Strategies of Trustworthiness, rigour and validation can target what?

Answer 13

• Design of the study
• Data generation & fieldwork
• Analysis and interpretation

Question 14

What are common strategies to enhance trustworthiness, rigour and validation of a qualitative study?

Answer 14

1. Audit Trail
2. Member check
3. Peer debrief
4. Present negative or discrepant info
5. Prolonged engagement
6. Purposeful sampling
7. Researcher Reflexivity
8. Rich, Thick Descriptions
9. Triangulation

Question 15

What is the audit trail?

Answer 15

Systematic record-keeping approach that maintains detailed description of entire research process

Question 16

What does the audit trail do?

Answer 16

Enhance transparency and credibility by providing detailed amount of steps taken, data collected, etc.

• Allows other researchers to follow researcher’s thought process

Question 17

Example of Audit Trail

Answer 17

Audit trail in the study of older adults experiences using a DVD-delivered exercise program

• Allows external reviewer to detect issues in data analysis and assumptions made by researcher

Question 18

What is member check? How is beneficial to enhancing trustworthiness?

Answer 18

Participants review data generated and study themes/findings.

• Participants have opportunity to delete, add or alter info

Question 19

Example of member check showed?

Answer 19

Interview of 17 adolescent female competitive dancers to see what influences their continued participation

• Participants given opportunity to give feedback on meanings that researcher had drawn from the themes

Question 20

What is peer debrief? How does it help with enhancing trustworthiness?

Answer 20

Seeking other professional willing to be the peer

• Peer pushes researchers to critically reflect on study findings
• Ensure the researchers findings are grounded in data rather being influenced by own perspectives

Question 21

Example of Peer Debrief

Answer 21

Peer debriefing in health and wellness of indigenous youth in Canada

• During data analysis a lot of coding of data was done and this is where a peer researcher would step in to monitor bias when studying notes made during coding process

Question 22

What is “Present Negative or Discrepant Info”?

Answer 22

Presenting info countering main study findings highlights opposing views and unique experiences
* Reality check forcing researchers to critically examine their assumptions and theories

Question 23

What is prolonged engagement?

Answer 23

Sustained time spent with participants in the field - help participants feel more natural in the presence of the researcher

Building rapport allows access to hidden insights that no other participant would let any other researcher know

Question 24

Prolonged engagement example?

Answer 24

Spending 6 months in community centre evaluating lifestyle intervention for people with type 2 diabetes
* Engaged at length with study participants
* Deeper understanding of participants and their environment

Question 25

What is purposeful sampling?

Answer 25

Recruiting rich-info participants who can answer the research topic/question the best because they would know or understand
* Not just participants that are appropriate for the study, but who will meaningfully inform

Question 26

Challenge of purposeful sampling and an example

Answer 26

Challenge: Difficult for reader to judge trustworthiness if full details are not provided

• Example: Interview with 15 AT educators and purposeful sampling of full-time educators which gave more richer insight into their teachings

Question 27

What is researcher reflexivity?

Answer 27

Researchers position themselves to reflect on their bias, values, experiences to consider how these variables may shape or inform their research

Similar to bracketing - researcher implementing their own thoughts into the research to set aside their own experiences so the focus is the participants

Question 28

What does researcher reflexivity consist of?

Answer 28

1. Reflecting on one’s experiences to the phenomenon being explored
2. Considering how one’s experiences shape the research process

Question 29

Example of researcher reflexivity?

Answer 29

Study of positive and negative body images at Brock University included reflexivity section:

• Overview of entire research team - research interests, training, and experience

Question 30

What is rich and thick descriptions?

Answer 30

Generating thorough descriptive data aligns with meaning-making process & allows researchers to gain greater understanding of phenomenon of interest
* Sharing findings in rich manner & highlighting unique and complex experiences of the participants

Question 31

What are some guidelines for rich, thick descriptions?

Answer 31

• Usage of multiple sources & methods of data generation
• Direct quotations to illustrate experiences in their own words
• Explain meanings of observed phenomena from both researcher and participant perspectives
• Situate findings within broader social, cultural, historical context
• Vivid and engaging language to convey richness

Question 32

Example used for rich, thick descriptions?

Answer 32

Tai chi class which some participants expressed how they felt joining and how they felt between and after sessions. One participant expressed how they felt uneasy but after a while got used to it and started enjoying it while also showing his grandchildren his newfound agility.

Thick descriptions like these capture the emotion behind the people and leads to insightful, rich data

Question 33

What is triangulation?

Answer 33

A way to cross check findings with multiple data sources, perspectives and methods
* I.e. using one-on-one interviews and observation to overcome potential misinterpretations

Question 34

Example of Triangulation in the text?

Answer 34

Sport psychology researcher working with Olympic athletes wants to understand role of social support at the olympics

• Spends prolonged time in the field
• ** Multiple methods of data generation (triangulation)**
• Personal journal maintained (researcher reflexivity)
• Didn’t invite members to check however
• Not peer debriefing
• Not having an audit trail