non - experimental research observational methods Flashcards

1
Q

Overview

A
  • Goals of observational methods
  • Subjectivity & objectivity in observations
  • Naturalistic & structured observations
    Observational measures in qualitative, correlational, and experimental designs
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Goals of observational methods

A
  • Describe behaviour in a way that is verifiable
  • Describe the variables that are present and the relations among them
    Test hypotheses
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Subjectivity and objectivity

A

Dallenbach and other researchers used the introspective method because they were interested in the conscious experience of a situation or task.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Subjectivity & Objectivity

A

when we can verify someyhwimhg with two or more observers

  • Researchers using the introspective method had rigorous training to eliminate bias in self-observations
  • But introspection is fundamentally subjective: I cannot verify your introspections and you cannot verify mine
  • Introspection therefore violates the principle of objectivity – the idea that science produces public knowledge
    The shift away from the introspective method led to the development of a specific criterion for measuring behaviour – that it can be verified by two or more observers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

types of observation

A
  • In naturalistic observations researchers study the behaviours of humans and other species in their normal environments
  • Structured observations allow researchers to evaluate responses to specific situations - allows researchers to evaluate specific set of behaviours eg aisnworths ss
    Shared features of naturalistic & structured observations
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Behaviour does not create a record

A
  • Observers
    Interpretation - interpretative component of observational data
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Methodological Challenges for
Observational Methods

A
  • Absence of control - limits our causal inferences
  • Reliability
    • Observer bias - address with coding categories being checked across multiple raters - inter-rater reliability
    • Participant reactivity
  • Ethics
    • informed Consent - changes observation as ptps know
      Recording behaviour - ptps need to know they are being recorded in advance
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Observation in Qualitative Research - festinger

A
  • Leon Festinger & colleagues joined a group predicting an apocalypse - naturalistic observation - what happened when people heard evidence inconsistent with their beliefs
  • Observers did not reveal their purpose in the group
  • Observers relied on memory
  • The influx of new people joining the group changed the group - no awareness of researchers but know inc in membership a change not consistent with naturalistic observations
    named it Theory of cognitive dissonance - changed bevies to accomodate the dissonance

present in group day ‘acpolcalyse happened’ - didn’t
members made excuses eg date changed due to increase in group members

issues -
- can’t evaluate reliability - no record
- small sample - not representative
- joined group to observe them - ethical issue

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Observation in Qualitative Research

A
  • Qualitative research is subjective: it aims to capture an individual’s point of view & acknowledges that our understanding of reality can only be approximate, not exact.
  • Qualitative research is concerned with the richness of description.
  • Qualitative research is narrative rather than quantitative.
    Qualitative research is iterative.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

types of analysis

A

grounded theory - uses categories to unite and explain data

discourse analysis - evaluates talk as social action

thematic analysis - less specialised, less dependent on theory

conversation analysis - structure of conversation. Jefferson transcription

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

observation in correlational research

A

Crowley, Callanan, Tenenbaum & Allen (2001) observed parent-child conversations about science in a science museum.
Children in consenting families (N=298) were given a sticker which also indicated age. - got informed consent

Conversations were coded for explanations, directions, and evidence.
Explanation included:
* causal connections within the exhibit interface (e.g., “When you turn that fast, it makes more electricity” at an exhibit including a hand-cranked generator),
* relations between observed phenomena and more general principles (e.g., “You see all those colors because the bubble reflects different kinds of light” at an exhibit where visitors can pull a sheet of bubbles up in front of a black background), and
* analogies to related phenomena (e.g., “This is just like that one time when our plants died because we forgot to water them” during a time-lapse video of withering bean sprouts).
Reliability was assessed by having 20% of the interactions coded by more than one rater.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Lee & Aronson (1974)

A

predicted that moving the room forward would produce optic flow patterns similar to what would normally accompany backward body sway, leading infants to adjust to counteract this apparent backward sway, which would result in infants swaying forward

“The behavioral response to a swing of the room was categorized according to the scheme shown in Table 1. Unless there was a clear change in posture during the 2.5 sec that the room was moving, a “zero” response was recorded” (Lee & Aronson, 1974, p. 531).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

summary

A

Observational methods broaden the questions we can address – whether within qualitative, correlational, or experimental designs.
Observational methods also bring risks.
We can use careful design to address those risks and create opportunities for clear evidence that will help us address valuable research questions with benefits for all areas of psychology.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

varieties of observational research

A

naturalistic or participant observation

some observational studies are more global observing a variety of behaviours while others are narrower focusing on specific behaviour.

researchers also place a varying degree of structure on the setting being observed. this can range from zero when the researcher simply enters some environment and observes behaviour without trying to influence it in any way to quite a bit, when the researcher creates a structured setting and observes what occurs in it.

studies with a higher degree of structure often take place in a laboratory environment and are sometimes called laboratory observation studies.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

naturalistic observation

A

the goal is to study the behaviours of people or animals as they act in their everyday environments. in some cases semi-artificial environments are sufficiently ‘natural’ for the research to be considered a naturalistic observation.

In order for the researcher to feel confident the behavior being observed is typical in the observed environment, it is important that it not be affected by the experimenter’s presence. There are two strategies for accomplishing this. First, in some naturalistic studies, the observer is hidden from those being observed. In other studies, the observer may not be present at all; some naturalistic observation studies (including Research Example 29, described later in this chapter) use video recorders. The videos are viewed later and scored for the behaviors being investigated. In some naturalistic observations, especially those involving animals, it can be impossible for the observer to remain hidden; the subjects quickly sense the presence of an outsider. Under these circumstances, the observer typically makes no attempt to hide. Rather, it is hoped that after time, the animals will become so habituated to the observer that they will behave normally. With some species, the process can take quite a bit of time.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

participant observation

A

Occasionally, researchers will join a group being observed, or at least make their presence known to the group, thus making the study a participant observation. The chief virtue of this strategy is its power to get the investigator as close to the action as possible. Being a participating member of the group can give the researcher firsthand insights that remain hidden to a more remote observer. In some cases, perhaps because a group is closed to outsiders and therefore not availa­ ble for naturalistic observation (e.g., a college fraternity), participant observation might be the only option. Participant observation is a common technique of qualitative research because the descriptions usually involve a narrative analysis of the group being studied.

17
Q

challenges facing observational methods - absence of control

A

some degree of control occurs in observational studies but in general the observational researcher must take what circumstances provide. because of this lack of direct control the conclusions from observational studies must be drawn very carefully.

despite the lack of control obsevrtional research can be a rich source of ideas for further stufy and it can sometimes serve the purpose of falsification an important strategy for theory testing.

An observation consistent with theoretical expectations provides useful inductive support, but an observation that contradicts a theory is perhaps even more informative

Despite control difficulties, observational research can provide important and useful informa­ tion. It can call ideas into question, and it can also suggest hypotheses for further study.

18
Q

challenges facing observational methods - observer bias

A

In observational research, observer bias means having preconceived ideas about what will be observed and having those ideas color one’s observa­ tions.
Bias can also occur because observational studies may collect huge amounts of information. Deciding which observations to report involves reducing this information to a manageable size, and the choices about what to select as relevant and what to omit can be affected by preconceived beliefs. Biasing effects can be reduced by using good operational definitions and by training observers to identify the precisely defined target behaviors. When actually making the observations, behavior checklists are normally used. These are lists of predefined behaviors that observers are trained to spot.

in addition to defining behaviors with precision, another way to control for observer bias is to have several observers present and see if their records match. This is inter‐rater reliability, a concept you encountered in Research Example 12 in Chapter 7. Within the context of observa­ tional studies, this type of reliability is also called inter‐observer reliability and is usually meas­ ured in terms of the percentage of times that observers agree. Of course, both observers could be biased in exactly the same way, but a combination of checklists, observer training, and agreement among several observers generally controls bias. Bias also can be reduced to the extent that procedures are mechanized. For example, a video recording increases objectivity by making it possible for the same event to be observed multiple times. Finally, bias can be reduced by introducing sampling procedures for systematically selecting a subset of the available information for observation and analysis. For example, a procedure called time sampling is sometimes used in observational studies. Rather than trying to maintain a continuous record of everything occurring, behavior is sampled at predefined times and only at those times. These times can be selected according to some rule, or they may be randomly selected. Similarly, event sampling selects a specific set of events for observation; others are ignored.

19
Q

challenges facing observational methods - participant reactivity

A

reactivity—that is, your behavior would be influenced by the knowledge that you were being observed and recorded. Obviously, this problem can occur in observational research and is the reason for the popularity of devices like two‐way mirrors. The problem also exists when animals are the subjects of observation and the observers cannot hide. As mentioned earlier, researchers assume that after time the animals will become accustomed to the presence of outsiders, but it is difficult to evaluate the extent to which this occurs. Reactivity can be a special problem for participant observation, in which the observers are involved in the group activities.

Reactivity can be reduced by using unobtrusive measures. These are measures taken of behavior, either directly or indirectly, when the subject is unaware of the measurement being made. Direct unobtrusive measures include hidden video or audio recordings or behavior samples. Indirect unobtrusive measures record events and outcomes one assumes resulted from certain behaviors even though the behaviors themselves were not observed.

20
Q

challenges facing observational methods - ethics

A

reducing reactivity raises the ethical problem of invading privacy and lack of informed consent, particularity if children or members of some other special population are being observed.

some researchers are hesitant to conduct research in the field because of concerns over privacy rights, informed consent, and even the possibility that researchers could be charged with a misdemeanor crime (e.g., disorderly conduct) or be sued. However, the APA eth­ ics code (see Standard 8.05 of the APA code of ethics) condones the use of naturalistic observa­ tion and does not require informed consent or debriefing, provided certain safeguards are in place.

Participant observation is a fairly common form of qualitative research today, and the issue of consent is a matter of some debate. Informed consent of the group being observed from within is now common and its absence requires a strong justifica­ tion (Taylor & Bogdan, 1998). Analogous to the habituation rationale for naturalistic observa­ tion, it is assumed that even if group members know they are being observed, they will eventually get used to the participant observer and behave naturally. On the other hand, in some cases, the group being studied would be unlikely to consent to a participant observer.

naturlalitsic observation example - Crowley, Callahan, Tenebum and Allen 2001 - looked at children in science museum mentioned in another flashcard

covert participant observation - farrington and Robinson 1999 gained some insight into the phenomenon of homeless by using a ptp observation methodology. purpose of study was to discover ‘identity maintenance stratgeies’ used by people who came in regularly to the homeless shelter.

21
Q

analysing qualitative data from non-experimental designs

A

thematic analysis - which is a method of identifying and analyzing patterns of responses (or themes) within qualitative data (Braun & Clarke, 2006). One way to think of thematic analysis is that the researcher is discovering recurring themes emerging from the data; this approach would be an inductive thematic analysis because you are allowing the data to guide the discoveries of themes. In contrast, researchers doing a more theoretical thematic analysis may predict certain themes prior to data analysis, based on their deductive reasoning rooted in theories about the phenomenon they are studying. A researcher using thematic analysis may easily use both approaches in their analysis of qualitative data in order to more fully capture the phenom­ enon being studied.

22
Q

archival research

A

Archival research involves use of information already collected for some other purpose, rather than by collecting new data. It often includes independent variables, but because these variables are non‐manipulated (i.e., no random assignment), archival research can be considered non‐experimental research. Data from archival studies can be subjected to a wide range of statistical analyses including techniques we have already considered (e.g., correlation and regression), as well as two techniques we will describe in a few pages, factor analysis and meta‐analysis.

23
Q

archival data

A

Archival research uses archival data, which refers to information already gathered for some reason aside from the research project at hand. These data range from public information such as census data, court records (e.g., felonies and misdemeanors), genealogical data, corporate annual reports, and patent office records to more private information such as credit histories, health history data, educational records, personal correspondence, and diaries. The term archives refers both to the records themselves and to the places where the records are stored. Archives are located in places such as university libraries, government offices, and computerized databases.

Internet‐based technology is transforming the way scientists conduct research, as psychologists can use the Internet to collect and analyze large datasets. The term big data is used to describe the vast amount of data available in electronic databases that can be extracted and analyzed with the use of advanced data analytic tools. Sources of information vary and include databases like those listed above, cell phone records, social media sites like Facebook and Twitter, and other data stored “in the cloud” from Smartphone apps or electronic monitoring devices like a Fitbit or a Google watch. Once extracted, data can be used to answer psychological scientists’ empirical questions about numerous topics.

the data extracted from archival sources sometimes stand on their own ready for analysis.

Sometimes, the archival information must undergo a content analysis before statistical pro­ cedures can be applied. Content analysis can be defined as any systematic examination of quali­ tative information in terms of predefined categories. Although content analysis normally occurs with verbal materials, that is not always the case. Because content analysis involves a degree of subjectivity, these procedures also typically include multiple coders and inter‐rater reliability estimates.

The most obvious strength of archival research is that the amount of information available is virtually unlimited, and the possibilities for archival research are restricted only by the creativity of the investigator. Yet archival data can create problems for researchers. Despite the vast amount of data available, some information vital to a researcher may be missing, or the available data may not be representative of some population.

Another problem with archival research is experimenter bias (Chapter 6). In archival research, this bias can take the form of attending more closely to records that support one’s hypothesis or interpreting the content of records in a way that is biased by one’s expectations. The problem can be difficult to avoid completely because the researcher doing archival research typically is faced with much more information than can be used, and the information can often be open to several interpretations. But the problem can also be managed most of the time by using the control pro­ cedures described in Chapter 6 (e.g., not disclosing the hypothesis to those responsible for coding or classifying the archival data—in effect, a double blind procedure). One problem often faced by researchers that does not occur with archival research is partici­ pant reactivity. For those participating directly in a research study, the knowledge that their behavior is being observed can influence that behavior in ways that yield a distorted result.

By its nature, archival research does not allow for random assignment in between‐subject designs, which makes it non‐experimental. Like other non‐experimental research, however, these studies often involve sophisticated attempts to control for potential threats to internal validity.

24
Q

analysing archival data

A

Because archival research is non‐ experimental, researchers often use various types of multiple regression analyses to test the strengths of certain predictors on various outcome (criterion) variables. Another approach to handling a large amount of data, including data from survey research and archival research, is to use an approach called factor analysis. Factor analysis is a multivariate technique in which a large number of measured variables are correlated with each other. It is then determined whether groups of these variables cluster together to form factors.

Factor analysis is a multivariate statistical tool that identifies factors from sets of inter‐ correlations among variables. It would most likely yield the same two factors we just arrived at by scanning the matrix. The analysis also determines what are called factor loadings. These, in essence, are correlations between each of the measures and each of the identified factors.

Factor analysis itself only identifies fac­ tors; what they should be called is left to the researcher’s judgment. Once such factors may be identified by a researcher, then decisions can be made on whether to use those factors as predic­ tors in a regression model.

25
Q

meta analysis - a special case of archival research

A

a meta‐analysis, where a researcher statisti­ cally analyzes the effect sizes from various completed studies on a particular topic. Recall that effect size is an estimate of how large an effect is in a study, taking into account measures of variability. Because a meta‐analysis uses data from pre‐existing sources (that is, completed research studies), it can be considered a form of archival research. Depending upon the research designs, different effect sizes can be calculated. Although the specifics behind how to conduct a meta‐analysis are probably beyond the scope of this research methods course, it is an interesting approach to learn.3 The primary goal of a meta‐analysis is to systematically synthesize the scientific literature on some phenomenon and combine the results across studies to better understand the phenomenon. Two main questions can be answered with a meta‐analysis. First, is the effect consistent across many other studies that test the same effect? And second, if the effect is consistent, then what is the size of the overall effect across studies? There are many examples of meta‐analyses in the psychological literature and many are done to better estimate an effect from many studies that may vary in the conditions used and methodological procedures employed.

Meta‐analyses can demonstrate the overall strength and reproducibility of the effect across many studies.