Test 3 Flashcards
(T/F)Experimental designs are largely used to make inferences about the communication practices of a large population.
F
(T/F) Experimental designs have low external validity
T
Which of the following best describes an outcome of using random assignment in experimental designs?
Random assignment guards against selection biases because participants cannot self-select exposure to the treatment group.
Referring to the eye gaze study above: In asking the participants to take the interview seriously, as if it were really a job interview, the researchers were trying to establish:
External Validity
Which of the following is the minimal standard coefficient to argue that an instrument or research coders are performing reliably?
70%
What threat to internal validity occurs when the researcher’s knowledge of the research purpose unconsciously influences his/her treatment of the participants?
Experimenter Effect
Qualitative researchers ensure that their research conclusions “ring true” to the informants of the study. The researchers establish
Credibility
Ethnographers use field notes and reflexive journals to ensure the trustworthiness of their studies.
Tractability
I would like to study what people say when they apologize and how many times they say certain phrases. I want to use all sorts of letters that contain apologies. I should use _______.
Content analysis
A researcher examines the frequency of different kinds of disconfirming messages found in interviews with recently married couples. He conducted frequency counts of messages that were impervious, interrupting, tangential, and others What type of question is guiding this researcher’s quantitative textual analysis?
A question about distributional structure.
(T/F) Manifest content is studying by coding the visible or surface features of the content.
T
(T/F) Quantitative text analysis is more unobtrusive than survey designs
T
Which of the following terms is used to describe what happens when coders begin to interpret the text slightly differently as time passes from the training process?
Coder drift
Dr. Gonzalez had two independent coders unitize his text. The first coder identified 750 units. The second coder identified 765. The two coders agreed on 695 units of analysis. Is the unitizing reliability acceptable?
Yes
What is the name of the people who formally enable (or disable) the researcher’s access to the field?
Gatekeepers
What is the purpose of the literature review in participant observation research?
A literature review provides the theoretical sensitivity for the researcher, preparing her for what might be happening in the phenomenon of interest.
What type of sampling is typically used when selecting the field site in participant observation research?
Purposive
You are conducting a project for an introductory communication studies course. In this project you have decided to examine how people use territorial markers at the largest campus dining hall. Over a four-week period you go to the dining hall at various times of the day, sit in a remote area and watch the students’ use of territorial markers. What kind of role are you taking one in this project?
Complete Observer
Why do participant observers seek to articulate that they have had prolonged engagement in the field?
Prolonged engagement is important for demonstrating the credibility of the interpretations.
When is a participant observer able to confidently leave the field?
The researcher can confidently leave the field once saturation has been achieved. (repeated results)
What comm topics are appropriate to experiment?
1: when some communicated-related variable serves as the independent variable in affecting some attitude, perception, or behavior (dv) –persuasion processes. Measure some feature of a comm message, fear appeal or emotional appeal, to determine whether it produces change in experimental participants attitudes beliefs or behaviors
2: when some noncommunication variable serves as the independent variable in affecting some communication-related dependent variable. “how is communication affected by some phenomenon or factor” –task’s complexity on the effectiveness of decision-making communication.
3: communication-related variables as both the dependent and independent variable.
How is external validity related to experiments?
The generalizability of experimental findings to the real world. Even if the results are an accurate gauge of what happened in the experiment, do they tell us anything about real life in society?
Strengths/weakness of experimental method
+: isolation of the experimental variable and its effect over time. Participants have a certain characteristic and following the administration of an experimental stimulus, they’re found to have a different characteristic
+: little time and money and relatively few participants, we can replicate a given experiment several times
-Artificiality, might not occur in more natural settings
What are the requirements of a casual relationship?
- Time order (IV before DV)
- Correlation between the IV & DV
- No other rival explanations that could explain the relationship between the IV and DV
How do researchers achieve control in experimental studies?
- Equivalent groups: same age, gender, race
- Control extraneous variables
- Manipulation checks:experimental participants are often asked their perceptions of the conditions to which they were exposed
- Double blind
- Random assignment to control and experimental group
Comparison group
No treatment or alternate treatment
Control group
No treatment, should resemble the experimental group in all other respects
Confederate
Person the researcher plants into the experiment
Placebo
Administered to a control group to make sure they believe they, like the experimental group, are receiving an experimental drug
Random Assignment
Experimental and control groups are similar in all ways so that at the end of the experiment points to the effect of the experimental stimulus, randomly assigned to either group
Eliminates selection biases
Pre-experiments design
- One shot case study, single group of participants is measured on a dependent variable following the administration of some experimental stimulus
- Weakest for finding new cause-effect b/c doesn’t eliminate threats to internal validity
- most common in comm research because dealing with humans no baseline comarison
Quazi-Expriment
Use pretests and posttests, more trustworthy
- can do time-series which studies processes occurring over time
- nonequivalent control group, create a comparison group that is similar to the experimental or treatment group but i not exposed to the independent variable. Uses matching instead of random assignment
- multiple-time-series: combination of the last two
Full/true experiment
Random assignment of participants to experimental groups and control groups
Validity
As close to the truth
Gunfire example: On the target but not in all in one place (not consistent, repeatable)
Measurement Reliability
Consistency, repeatable, should get the same findings every time
a. Multiple administration techniques
- test-retest method: same test in 2 different points in time with same group
- alternate form method: 2 equivalent versions (A &B), wording could result in different answers
b. Single-administration techniques
- split-half method: first half/last half, 400 sample at least
- interobserver reliability: content analysis, 2 people coding same data
- reliability coefficient: correlation between 2 sets, higher you get the more reliable
Measurement Validity
Whether or not you measure what you say you are measuring
a. Content validity: does measurement reflect the content or concept being studied
i. face validity: common sense, does it look right
ii. panel validity: experts (4-5)
b. Criterion related validity: compare to find if your measure is valid
i. concurrent validity: vs. old, want the same results
ii. predictive validity: current measure can predict future behavior
c. Construct validity: validity inferred from theory if consistent to what theory says
External Validity
Generalization of a study: the findings of a study have applicability on other contexts or with people other than those who participated in the study
a. sampling procedures: boost or compromise study
b. ecological validity: environment of study, setting mirrors real life validity
c. replication
Reliability
Repeatable. Gunfire: all consistent but off target. Teacher sees student face zoning out and assumes they aren’t paying attention, when really they could be just tired
Observed score formula
o=T+e
T= one Truth,
e= error
2 types of error
- random error: unreliability
- systematic error: invalidity
- i.e. survey questionnaire
Internal validity
Possibility that the conclusions drawn from the experimental results may not accurately reflect what has gone on in the experiment itself
Content analysis
Researcher analyzes a content, form, function, or structure of a message. i.e. nat. geo.
Interaction Analysis
Who said what to whom, network analysis
Quantitative Text Analysis
Analyzing texts for social artifacts, getting at the microscopic details of communicative messages
(emails of people trying to get into 398)
Texts
Anything where symbols are used, huge variety
What questions can quantitative texts answer
Distributional: frequencies, how many times the wife yelled
Interactive: who talks to whom about what, network
Sequential: pattern that follows interaction
Content Analysis
- Set problem/RQ & H
- Determine genre of text
- Sampling (unit of analysis & unit of observation
- Coding (count & record observations)
- Run statistical tests
- Write up (report results)
Coding
Process of unitizing text into coding units, then have at least 2 coders categorize each coding unit against a coding scheme that has been deductively or inductively developed (from theory) by the researcher
-coding unit: the chunk of text to be coded
Manifest content
The visible, surface content. Little interpretation involved, superficial
Latent content
In underlying meaning, deep
Intercoder reliablity
- Unitizing reliability: agree on identifying the coding units
- Categorizing reliability: agree in categorizing the coding units
- Need 70% or better
Coder drift
When coders start to interpret texts differently as time passes from the training process
Coding schemes
Can develop inductively, observing patterns in the talk and then formalizing those into a set of categories that coders use in coding all of the interaction
Intercoder reliability Equation
2(U)
_____
U1+U2
Ethnography & advantages
Enter the field or site to learn the emic perspective
-unfiltered, native
+: find truth not Truth, different perspectives you can’t see with quantitative studies, find new phenomena, multiple interpretations
Ethnography sampling technique
Nonrandom because you are looking for a specific subset
Influences of statuses in ethnography
Reflect on your social status so you are aware of your position, basically what makes up who you are
Reflexivity
Being aware of your statuses and how that effects your participation in the study
Saturation
When patterns emerge & repeating, then you can leave the field
Theoretical sensitivity
Understanding what you are going into so you are prepared what people say about your topic, and what to observe
Reflexive journal
Chronological notes, what you did, how it made you feel, etc.
Field notes
Direct observation, no feelings, can write when you’re off site
Prolonged engagement
Staying in site for a long time to establish credibility
Ethical considerations for ethnographers
- Is it ethical to talk when they don’t know you are recording
- Is it ethical to get information for your own purposes from people you hate?
- Is it ethical to see a severe need for help and not respond to it directly
- It is ethical to be strategic in your relations?
- It is ethical to take sides?
Qualitative interviewing
Interaction between an interviewer and a participant in which the interviewer has a general plan of questioning but no specific question, interviewer establishes a general directin for the conversation and pursues specific topics raised by the respondent. Focused on the meanings and the rules of meaning-making
+multiple interpretations/perspectives, see truths not Truth, give marginalized people a voice, find new phenomena
Structured qualitative interviewing
Interviewer has standardized wording for all questions and asks respondent pre-established questions with a limited set of response categories.
- more of a survey, close ended questions
- reflect the researchers point of view rather than the informant
- limit the depth of insight that can be gained from open-ended questions
- do not display much adaptation to the particular informant
Semi-structured qualitative interviewing
List of questions interviewer wants answered by the informant. Usually open-ended to maximize in depth responses
+: can only want information on specific questions
+= make explicit comparisons between informants
