Nov. 26/28th - Chapter 6: Observational Methods

Question 1

READINGS

Question 2

Achieving the goal of understanding real-world phenomena - 2 methods

Answer 2

1. Quantitative
2. Qualitative

Question 3

Quantitative Approach

Answer 3

• …numeric data, then analyzed using statistics
• Involves collecting data in the form of numbers
• EX: self-reports, reaction-time tasks, physiological measures = numeric data

Question 4

Qualitative approach:

Answer 4

• non-numerical form; using an interpretive approach
• Offers deep descriptions of peoples behaviour in natural settings, through people explaining their experiences in their own words
• Involves collecting in-depth info on few individuals or within a limited setting, and conclusions are based on interpretations of the investigator
• Greater variety in the paradigms/procedures associated with qualitative approaches

MAIN IDEA: THE COLLECTION & INTERPRETATION OF ANY FORM OF DATA OTHER THAN NUMBERS CONSTITUTES A QUALITATIVE APPROACH. RATHER THAN A STATISTICAL ANALYSIS OF NUMBERS, A QUALITATIVE REPORT MAY DESCRIBE DIFFERENT THEMES THAT EMERGED FROM ANSWERS

• In some cases, qualitive data might be collected and then converted into numbers for a quantitative statistical analysis
• Would still be considered a qualitative approach, as non-numerical data are being collected

Question 5

Naturalistic Observation:

Answer 5

• research method in which researchers immerse themselves in a particular natural setting
• Observations are typically made over an extended period of time using a variety of information-collection techniques

Question 6

Naturalistic Observation: Field Notes

Answer 6

• Data can be gathered in the form of detailed field notes
• Field notes can include information about all aspects of a situation, including:
  o Setting
  o Patterns of personal relationships
  o People’s reactions 2 events

Question 7

Naturalistic Observation: Need not rely solely on passive observation, but can also involve…

Answer 7

• Interviewing key informants to gain inside information
• Talking to people about their lives
• Taking pictures of the environment

Question 8

2 goals for researchers:

Naturalistic Observation

Answer 8

• The first goal of researchers is to describe the setting, events, and persons observed
• The second is to interpret what was observed; depending on the specific qualitative approach used, this may involve identifying common themes (as in phenomenology) or developing a theory that can generate hypotheses for future work
• Final report may reflect the chronological order of events or it can be organized around the theory developed by the researcher
• Specific examples of events that occurred during observation are often used to support and illustrate the researcher’s interpretations
• Good reports support validity of the interpretation by using multiple sources of confirming evidence
• Published qualitative report includes both specific observations and the researcher’s observations

Question 9

2 Key Issues in naturalistic observation: Overview

Answer 9

1. whether to actively participate in the situation you’re studying, or not to participate but still observe
2. whether to conceal your purpose or presence from the other people in the setting

Question 10

What combinations of approaches can be used?

Naturalistic Observation

Answer 10

• Qualitative approach
• Mixed approach - gathering quantitative in addition to qualitative
• Fully quantitative approach - collecting only numerical data

Question 11

2 Key Issues in naturalistic observation:

Active participation - yes or no?

Non-participant observer, Participant observer

Answer 11

• whether to actively participate in the situation you’re studying, or not to participate but still observe
• Non-participant observer: outsider who does not become an active part of the setting being observed
• Participant observer: assumes an active, insider role; participant observation allows the researcher to observe the setting from inside, allowing researchers to experience event in the same way as the other participants

Question 12

Issues with Partipant Observers

Answer 12

• Potential problem: observer may lose the objectivity necessary to conduct scientific observation
• Remaining objective my be especially difficult when the researcher already belongs to the group being studied and identifies with that community
• There is great risk that the observations with be biased

Question 13

2 Key Issues in naturalistic observation:

Whether to conceal your purpose or presence from the other people in the setting

Concealed observation, Non-concealed observation

Answer 13

• Do you tell the rest of the group that you’re a researcher interested in studying them? Do you allow people to be aware that you’re present and observing
• Concealed observation - concealed presence
• Preferable, as the presence of the observer may influence the behaviour of those being observed; results in less participant reactivity. If people are unaware of them being observed, they can’t react to this act of observation
• Can be preferred from an ethical viewpoint
• Non-concealed observation
• Can become non-problematic as reactivity can subside quickly, as in people becoming used to the observer and behaving naturally in the observer’s presence
• Especially when observer isn’t physically present, but the study is rather being recorded (which provide a rich source of qualitative data

Question 14

Naturalistic Observation

HOW TO DECIDE ONE’S PRESENCE OR PURPOSE

Answer 14

• Depends on ethical concerns and on nature of the particular group being studied
• There are degrees of participation and concealment: a non-participant observer may not become a member of the group, but may over time become accepted as a friend or simply part of the ongoing activities of the group

Question 15

EXCEPTIONS for informed consent in naturalistic observation

Answer 15

• **Observation in public places when anonymity is not threatened and behaviour is not manipulated **is typically considered exempt from ethical review
• In such situations, informed consent is usually unnecessary
• Overall, researchers must be sensitive to ethical issues when conducting naturalistic observation
• Consider whether observations are made in a public place, with no clear expectations that behaviour will be private
• Issues of informed consent resurface when collecting data with the audio-recording devices, as p’s become so used to it
• To ensure ongoing informed consent, participants are given all audio files before data analysis

Question 16

Naturalistic observation cannot be used to study all issues

Useful…

Answer 16

• When investigating social settings
• For gathering data in real-life settings and generating hypotheses for later lab experiments
• High ecological validity: Conducted in situations similar to everyday life

Question 17

Naturalistic observation cannot be used to study all issues

Difficult…

Answer 17

• In controlling the setting in naturalistic observation
• In challenging to test well-defined hypotheses under specified conditions
• To conduct overall: Cannot be scheduled at a convenient time, place (Similar to field experiments in general)
• Process of interpreting data: Researchers sort through broad measures to identify common themes, develop hypotheses, code data….
• Can’t generalize (LOW EXTERNAL VALIDITY)
• Reactivity vs. ethics

Question 18

Systemic Observation:

Answer 18

The careful observation of one or more specific behaviours in a particular setting
* Unlike naturalistic obs., which takes place in the real world, systematic obs. is often created by the researcher
* A lot less common
* Often used within a quantitative rather than qualitative approach
* Researcher is interested in only a few very specific behaviours
* Observation of these behaviours are easily quantifiable
* Researcher has strong hypotheses about the behaviours

Question 19

Systemic Observation

What 3 things must researchers establish?

Answer 19

• Which behaviours are of interest
• Choose a specific setting in which the behaviours can be observed
• Develop a coding scheme to record and categorize observations

Question 20

Systemic Observation

Coding Schemes

Answer 20

Purpose is to quantify qualitative observations, written responses, live actions, or images produced by participants

• Should be as simple as possible, allowing observers to easily categorize behaviours
• Especially important in coding live behaviours rather than viewing videotapes
• Videotaping p’s is often preferable, so that observers can carefully review their coding

Question 21

How can researchers develop coding

Answer 21

If only coding for a few things, you want to be confident that you’re coding the right ones and not missing anything important

• Researchers might choose to conduct a naturalistic observation study to help them come up with meaningful categories for their coding scheme, which they could then use to code behaviours observed in a more controlled systematic observation study
• Sometimes researchers use coding schemes that have been developed by others; major advantage is that research already exists that validates the coding scheme’s use for particular purposes, and training materials are usually available

Question 22

Issues in systematic observation:

Answer 22

1. Inter-rater reliability
2. Participant reactivity
3. Sampling behaviours

Question 23

Issues in systematic observation

Inter-Rater Reliability

What level in published research? Achievable with live coding?

Answer 23

• When conducting systematic observation, two or more raters are tasked with coding the behaviours; reliability is indicated by high agreement
• High levels of agreement in all published research - because coders are often trained extensively on the coding scheme before they begin coding the data in question (practicing on prior or example data)
• Can be difficult to obtain high inter-rater reliability with live coding, unless the coding scheme is very simple
• Video-recording has the advantage of providing a permanent record that can be coded later by many observers
  Also means that coding can be double-checked

Question 24

Issues in systematic observation

Participant reactivity

Answer 24

Presence of observer can affect p’s behaviours by producing p reactivity (like in naturalistic observation)
* ** Can be reduced by** *concealed observation *(EX: one way mirrors, hidden cameras)
* Esp. important to debrief p’s and offer the opportunity to forbid researchers to use their data when concealed tools are used
* Also by allowing enough time for people to become used to the presence of the observer and recording equipment

Question 25

Issues in systematic observation

Sampling Behaviours

Answer 25

• Researchers must make decisions about how to sample behaviours
• For many research q’s, samples of behaviour taken over a long period provide more accurate and more useful data than single, short observations
  Can eliminate potential short-term effects

Question 26

Case Studies

Answer 26

Provides a detailed description of an individual person or a setting, such as a business, school, or neighborhood
* Can be based on naturalistic observation, but this isn’t necessarily the case
* Can be a description of a patient, or a historical account of an event; involve library research and interviews
* Useful when an individual possesses a particularly rare, unusual, or noteworthy condition: Can provide unique data about what is humanly possible for a range of psychological phenomena
* Extreme caution must be taken when interpreting the results of a case study, as it is inappropriate to generalize results from one case to the population;an issue of external validity

Question 27

Archival research - The researcher does not collect any original data; rather, analyzes existing data such as (7)

Answer 27

1. Public records
2. Newspaper articles
3. Blog posts
4. Census data
5. Public social media posts
6. Data published online for public access
7. Even other reports by other researchers

Question 28

Archival Research

Type of data extracted from archival sources differs depending on…

Answer 28

…whether the researcher is using a qualitative approach or a quantitative approach

• Three sources of archival research data are:
  1. Census data in the form of statistical resources
  2. Survey archives
  3. Written records

Question 29

Archival Research

Census Data or Statistical Records

Answer 29

• Statistical records are collected by many public and private organizations
• EX: Stats Canada maintains the most extensive set of census records available to researchers, available for analysis
• Forms of public records can be used as sources of archival data, such as sports league stats

Question 30

Archival Research

Survey Archives

Answer 30

• Repository of data from surveys that is available to any researchers who wish to analyze them
• EX: major polling organizations make the data from their surveys available online
• These sorts of archives are often extremely important, because most researchers do not have the financial resources to conduct surveys of randomly selected national samples - which are crucial for making descriptive claims about specific population

Question 31

Archival Research

Written Records/Mass Media

Answer 31

Written records: stored documents, including:
* Diaries and letters that have been preserved by historical societies
* Public documents such as speeches by politicians
* Ethnographies: detailed descriptions of other cultures written by anthropologists

Mass media: basically any cultural product that is archived and accessible
* Books
* Magazine articles
* Movies
* TV programs
* Newspapers

Question 32

Relying on existing archives of written documents and mass media can be used to study (3):

Answer 32

1. Complex behaviours
2. Mental illness
3. Culture - since mass media are products of cultures

Question 33

Working with archival data:

Content analysis and Interpretation

Answer 33

• Content analysis: systematic analysis of existing archives, such as written documents and mass media records like movies and TV shows
• In order to analyze, a *coding scheme *is often used to quantify the information presented. Sometimes the scheme is quite simple and straightforward. In other instances, researcher must work to carefully define the coding categories
• Whenever coding is conducted, raters must be trained on how to use the coding scheme appropriately, especially when categories might be nuanced or complex
• Software solutions can also aid with content analysis, however can be rather simple
• Nuanced coding and interpretation of texts and mass media will continue to require human thought
• Software can also be used to analyze mass media archives

Question 34

Content analysis and Interpretation - allows researchers to:

Answer 34

…study interesting questions - some which cannot be studied in any other way
* Hypotheses can be tested and new hypotheses may also be generated
* Archival data are a valuable supplement to more traditional data-collecting methods, as they represent spontaneously occurring real-world behaviour
* As a result, offering enhanced ability to generalize to the real world; GOOD EXTERNAL VALIDITY

Question 35

3 challenges to Content analysis and Interpretation

Answer 35

1. Desired records may be difficult to obtain: Data can be placed in long-forgotten storage, been destroyed, or may be owned by companies or institutions that aren’t willing to share them
2. Archival research can’t control what data were collected nor how they were recorded: Can never be 100% sure of the accuracy of information
3. As work is non-experimental, alternative explanations for observed relationships exist, so we cannot make causal claims about these associations

Question 36

LECTURE

Question 37

Specific Research Methods:

Quantitative: 5

Answer 37

DISCOURSE ANALYSES (coding-themes):

1. Experiments
2. Correlational
3. Longitudinal
4. Quasi-Experiments
5. Systematic Observation

Question 38

Specific Research Methods:

Qualitative: 4

Answer 38

CONTENT ANALYSIS (counting frequencies)

1. Case studies
2. Naturalistic Observation
3. Interviews
4. Focus Groups

Question 39

Discourse & Content Analysis:

Discourse Analysis: What is discourse?

Answer 39

A social practice that constitutes our social relations
* The way we talk and write positions us in relation to others – our discursive practices both reflect and shape the social order and social relations.

Discourse analysts seek to understand how our uses of language reflect and shape our beliefs, experiences, interactions and identities.
□ Who has power?
□ What is authentic (normal)?
□ Who is the actor? Who is to blame?