Week 12 : Ethnography & Evaluation Research

Question 1

What is Ethnograpy?

Answer 1

• researchers immerse themselves in the lives & social worlds of the ppl they want to understand
• understand the world as the research participants experience it
• contribute to social science theories

Question 2

History of ethnography

Answer 2

• principle tool of anthropology
• sociologists used to only study subcultures within their own culture but this has changed now

Question 3

Why use ethnography?

Answer 3

• richness of information… what ppl say vs what they do, ‘taken-for-granted’ ideas and relationships
• understanding causal mechanism… causality is does X cause Y? but causal mechanism is how does X cause Y?

Question 4

What do ethnogrpahers study?

Answer 4

• specific phenomenon bounded by time & space
• study both what people say and do
• life in an institution (e.g. asylumns, schools)
• life in a particular social group (e.g. deviant groups)
• life in a community/community studies - take the entirety of social life into account (e.g. neighbourhood studies in big cities
• more studying down than up (likehomeless communities not elites)

Question 5

Ethnographer roles - researcher can adopt one of four roles

Answer 5

1. covert researcher & participation… complete participant
2. no covert researcher & participation… participant observer
3. no covert researcher & no participation… observer
4. covert researcher & no participation… covert observer

Question 6

Ethnographer roles

1 - complete participant

Answer 6

• ‘go undercover’
• immerse themselves in a fieldwork site
• keep their identity as a researcher a secret

Question 7

1 - complete participant

Pros & cons

Answer 7

• pros… access to specific groups
• cons… ethics (deception), reactivity (researchers presence affects) & ‘going native’ (cognitive dissonance, lose identity & anxiety)

Question 8

Ethnographer roles

2 - participant observer

Answer 8

• the researcher tells at least some of the people being studied about his/her real identity as a researcher
• Still participates fully in the social life of the setting
• most common
• subjects’ informed consent can be assumed by their acceptance of the researcher being present, however, not every person the ethnographer encounters will have given consent

Question 9

2 - participant observer

pros & cons

Answer 9

• pros… fewer ethical concerns & authenticity
• cons… Hawthorne effect - refers to the fact that merely being observed often changes subjects’ behaviour (usually fades quick tho)

Question 10

Ethnographer roles

3 - Observer

Answer 10

• The researcher tells ppl they are being observed but does not take part in the subjects’ activities & lives
• E.g. a lot of newspaper reporters who spend time in the field adopt this role

Question 11

3 - observer

Pros & Cons

Answer 11

• pros… suitable for profesisonal activities such as sports, medical, coutroom, etc. & even fewer ethical concerns
• cons… Hawthorne effect

Question 12

Ethnographer roles

4 - covert observer

Answer 12

• Observe ppl who do not know they are being observed/studied
• most often in public situations/preliminary analyses where many ppl are present & the researcher can fade into the crowd

Question 13

4 - covert observer

pros & cons

Answer 13

• pros… least likely to alter the dynamics of the world they study
• cons… greatest risk of misunderstanding the situation & systematic observations (checklist & timeline)

Question 14

Approaches to Theory in Ethnography

Answer 14

• Grounded theory approach… Inductive - researcher does not begin with a theory but develops a theory on the basis of experiences and observations in the field
• Extended case study approach… deductive - researcher starts with an established theory & chooses a field site, or case, to improve upon or modify the existing theory

Question 15

Choosing a topic….

Answer 15

• sometimes problems arise when researchers immerse themselves in a field site with no real question
• It is best to begin with a practical or theoretical question about the subject you want to study
• may change over time but should be able to specify why your feld site is worth studying

Question 16

Sampling

Answer 16

• also called case selection/field site selection
• time & space & focus on which subjects
• purposive sampling - cases are delibetately chosen on the basis of features that distinguish them from other cases

Question 17

Insiders vs outsiders

Answer 17

• Insider pros… access, acceptance, understanding & perception that ppl will act more ‘naturally’
• outsider pros… distance to see and identify insider assumptions & the ‘ignorance’ to question the behaviours that seem natural to the group

Question 18

Negotiating access…

Answer 18

• gaining access through gatekeepers
• build rapport with people in the institution/community (limit the ‘real you’ though)
• make relationships with key informants

Question 19

When to leave the field?

Answer 19

• usually when you have reached saturation
• sometimes there is a natural end (e.g. end of a school year)

Question 20

creating data with fieldnotes

Answer 20

• field notes - the data produced by a fieldwworker, including observations, dialogue & thoughts about what is experienced in the field
• writing up good field notes takes twice as long as being in the field

Question 21

What should go into fieldnotes?

Answer 21

1. direct observations… a chronological account of your observations & experiences, episodes & physical descriptions
2. researcher’s inferences and ideas… your own feelings & reactions during the day & your sociological analysis of the situation

Question 22

Writing the ethnography

Answer 22

1. realist tales… most common, objective, third person
2. confessional tales… reflect on on role, personal, first person
3. advocacy tales…documenting a wrong & advocating for political change

Question 23

New applications of ethnography

1 - team ethnography

Answer 23

• conducted by two or more scholars working together
• cover multiple research sites/subgroups of the ppl being studied simultaneously
• compare their results through triangulation

Question 24

New applications of ethnography

2 - business ethnography

Answer 24

• market research… to understand the cultural meanings people attach to products
• usability research… to help engineers and software developers understand how consumers engage with websites and technology

Question 25

New applications of ethnography

3 - visual ethnography

Answer 25

• taking photographs of & filming people in their everyday lives
• These video ethnographies open up private lives to social science scrutiny in ways ethnographers could not accomplish in person (e.g. baby cams)

Question 26

New applications of ethnography

4 - cyber ethnography

Answer 26

• the study of how we behave in online communities/online life
• also called netnography
• communities form online through chatroms & discussion boards
• followers of blogs & players of online video games interact with one another

Question 27

Validity & reliability

Answer 27

• Reliability is not strong because participant observation relies heavily on researchers’ perceptions & interpretations
• validity higher…. high internal, low external (triangulation)

Question 28

What is evaluation research?

Answer 28

• goal is to determine whether a social intervention produces its intended effects
• working for translation… implementing the components of an evaluation research project on a larger scale (e.g. a school policy)
• seeks to answer specific questions that are important to diverse stakeholders, who are the various parties with interests in the outcomes of the evaluation

Question 29

social intervention

Answer 29

• A specific policy implementation or change that is intended to modify the outcomes or behaviours of individuals/groups
• key is it needs to be SPECIFIC

Question 30

challenges of evaluation research…

Answer 30

1. politically charged contexts
2. logistical challenges (differential attrition rates between treatmnet & control groups)
3. ethical concerns (what if it’s harmful/not effective? & what about denying people an opportunity?)

Question 31

3 steps in evaluative research

Answer 31

1. formulate the evaluation research question
2. measure the desired outcome
3. implement the intervention and asses its effects

Question 32

3 steps in evaluative research

1 - Formulating the Evaluation Question

Answer 32

• should be done with the stakeholders
• reasonable scope… feasibility in time & space
• answerability… specific, measureable outcomes
• e.g. “Do abstinence-only sex education classes reduce teen pregnancy?”—is both reasonable and answerable

Question 33

3 steps in evaluative research

2 - Measuring the Desired Outcome

Answer 33

• Researchers must be able to effectively measure the outcome of interest & ensure their outcome measures are reliable and valid
• Reliability is crucial if we are measuring whether an intervention produces a change in some outcome
• self-reported measures are less reliable than performance measures or behavioural measures
• self-reported measures are sensitive to participants’ moods & social desirability bias

Question 34

3 steps in evaluative research

3 - implementation

Answer 34

• researchers must decide which research method is best suited to implement & evaluate the intervention
• randomized fiedl experiment is the gold standard of evaluation research… intervention, treatment & control group
• researchers frequently rely on quasi-experimental methods tho

Question 35

3 - implementation

Two key factors affect the choice of research method in evaluation research

Answer 35

1. The control & treatment groups should be as equivalent as possible (randomized field experiment)
2. The purpose or goal of the evaluation (effect & effect size - randomized & quiasi field experiments) (qualitative - why and how it works)

Question 36

Quasi experimental methods

Answer 36

• ‘natural experiment’
• change happened beyond the control of researchers
• Still attempt to compare the effect of the treatment to a control, but they lack random assignment of individuals to the two groups (could be biased)
• When treatment/control groups are formed by a procedure other than randomization, the procedures are said to use non-equivalent comparison design….

Question 37

how to establish ‘treatment’ & control groups in quasi-experimenting?

1 - matching

Answer 37

• After selecting participants for the treatment group, the researchers assemble a control group by selecting individuals who are identical, or ‘matched’, on specific characteristics
• Hard part is deciding which characteristics should be used to match and which should be ignored

Question 38

how to establish ‘treatment’ & control groups in quasi-experimenting?

2 - statistical controls

Answer 38

• When researchers control for the effects of additional variables to ensure the treatment & control groups are equivalent, they are using statistical controls
• When their research is complete they can use statistical techniques to assess the effects of a large variety of characteristics that they collect in the outcome of interest

Question 39

how to establish ‘treatment’ & control groups in quasi-experimenting?

3 - reflexive controls

Answer 39

• When researchers compare measures of an outcome variable on participants before & after an intervention, they are using reflexive controls (participant is their own control)
• pre-post design… researchers measure the outcome of interest once before the intervention, introduce the intervention & then measure the outcome again
• time-series design… takes multiple measures of the outcome of interest over time (can be long before/long after the intervention)
• Multiple time-series designs…compare the effect of an intervention over multiple locations