exam 3 Flashcards
A list of questions to extract specific data from a particular group of people.
A method of gathering data (can be qualitative or quantitative) – MOSTLY QUANTITATIVE
Survey
Collect information from the sample at the same point in time.
Cross-sectional Surveys
Information is collected at different points in time to study changes in time.
Longitudinal Surveys
Surveys given where members of the population can change and sampled over many times over different points of time to see if any trend appear.
Example
Restaurants survey customer satisfaction to show trends in sales and store operations.
Customer participation varies greatly.
Trends are able to detect needs for improvement although may not have the best validity.
Trend Study
Surveying where the type of sample does not change, but the members and individuals surveyed change and are of a cohort.
A cohort is a group of people with a shared characteristic.
A trend study samples a population whose members may change over time; a cohort study samples a particular population whose members do not change over the course of the survey.
The same people do not necessarily participate from year to year, but all participants must meet whatever categorical criteria fulfill the researcher’s primary interest.
Common cohorts that may be of interest to researchers include people of particular generations or those who were together around the same time period.
Example: A researcher wants to measure public opinions on marijuana. Each decade an individual was born is considered a cohort. People’s opinions when surveyed may be different but the fact that they all have a characteristic in common.
Cohort Study
The key difference between a cohort study and a panel study is that unlike in the case of a cohort study, in a panel study, the same individuals are used in throughout the study.
Constantly surveying the SAME sample throughout time.
Example: A researcher wants to investigate parolee satisfaction with an outpatient re-entry program and reintegrating to society.
Panel Study
Survey Questions are based on operational definitions.
But also include other variables and characteristics (i.e., age, sex)
Afilter questionis designed to identify some subset of survey respondents who are asked additional questions irrelevant to the entire sample (e.g., “If yes,).
Survey Questions must be concise, easy to understand, clear wording.
Prevent the use of double negatives, double-barreled questions, and answers, jargon, slang.
Double-Barreled Question: asking multiple questions as though they are a single question can confuse survey respondents.
Double Negative Questions: includes two negative words, potentially confusing or misleading the participant completely.
Survey Questions must have neutral wording.
Avoid leading language, pressing for social desirability.
Example: How bad would you rate police officers? Rate how guilty you think they are form 1 – 10.
Survey Questions
Questions that offer the respondent a closed set of responses from which to choose.
Structured (Fixed Response)
Questions that offer the respondent a closed set of responses from which to choose and an additional free response.
Partially Structured
A rating question (also called a Likert questions)asks survey respondents to compare different items using a common scale(e.g., “Please rate each of the following objects on a rating scale of 1-10, where 1 is ‘not at all important’ and 10 is ‘very important. ‘”).
Aranking questionon the other hand asks survey respondents to compare a list of different objects to one another (e.g., “Please rank each of the following items in order of importance with #1 being the most important object to #10 being the least important object.”).
Rating and ranking
Non-structured questions, or open-ended questions, are questions where there is no list of answer choices from which to choose.
Respondents are simply asked to write their response to a question.
Non-Structured/Open Ended
Email: If you choose an email as your distribution method for sending out your survey, traditionally, your survey is linked in the email
Pros
Easy to carry out using all email platform services (mass emailing)
Participants have easy access.
Email recipients can reach out to connect with the research team if they have any questions.
Cons
The email with the survey link can be lost in inboxes and reminder emails may need to be sent.
Social Media: When you want to bring your survey to a wide, non-specific audience, social media is a great route to take.
Pros
Free
Social posts can further be targeted by tagging the message with specific hashtags or topics, which help target people with similar interests. This can mean that non-followers or non-members can also find the survey.
Cons
You may not have a large social media presence.
You may need to spend money on promoting your social media posts.
Followers may be reluctant to participate.
SMS (Text Messages): SMS will be most effective where the recipient knows and trusts the sender.
Pros
A lot of other social and work activities happen through SMS and notifications,
This method is not limited by geography or time-zones.
Cons
Your message could be considered spam, especially if the number is not linked to an existing contact.
Unlike social media basic posts, SMS text messages cost the organization money to arrange and send out. Costs will be dependent on the phone service.
QR Code: When your survey’s QR code is scanned, respondents will be immediately directed to your survey.
Pros
QR codes do not track information identifying your respondents.
If you have a long link to your survey or if you want to capture details in one click, a QR code can save your participant’s time.
QR codes are easy and free to use.
Cons
Survey participants must opt in to take the survey and capture the QR code to begin. The supporting content around the QR code must be persuasive for participants to make this extra effort.
Since this is newer technology, older generations of participants may find using this method confusing or need instructions on how to scan the codes.
Avoid
Open-ended questions
Mutually exclusive and exhaustive response options
Too many questions
Beware
Fence-sittersare respondents who choose neutral response options, even if they have an opinion.
Floaters: are those that choose an answer to a question when really, they don’t understand the question or don’t have an opinion.
Interviews are a systematic way of talking and listening to respondents and are another way to collect data from individuals through conversations.
Interview
Advantages
It provides flexibility to the interviewers.
The interview has a better response rate than mailed questions, and the people who cannot read and write can also answer the questions.
The interviewer can judge the non-verbal behavior of the respondent.
The interviewer can decide the place for an interview in a private and silent place, unlike the ones conducted through emails which can have a completely different environment.
Disadvantages
Conducting interview studies can be very costly as well as very time-consuming.
An interview can cause biases. For example, the respondent’s answers can be affected by his reaction to the interviewer’s race, class, age or physical appearance.
Interview studies provide less anonymity, which is a big concern for many respondents.
There is a lack of accessibility to respondents (scheduling and geography)
Has predetermined questions with fixed wording, usually in a pre-set order.
The use of mainly open-ended questions is the only essential difference from a survey.
structured interviews
Has an underlying structure created by your core questions while also allowing the option to explore new avenues of thought with your participants as these opportunities arise?
Question wording can be changed, and explanations given; particular questions which seem inappropriate with a particular interviewee can be omitted, or additional ones included.
Semi structured interviews
The interviewer has a general area of interest and concern, but lets the conversation develop within this area. It can be completely informal.
aims to understand participants’ narratives or life stories, as they understand and make sense of them.
Unstructured Interview
Attending fully to what the speaker is saying by focusing wholly on what is being said.
Requires the listener to give full attention to the speaker until either the message has been received or the speaker has finished speaking.
active listening
Reflective listening builds on active listening and involves verbally reflecting back to the speaker the emotions inherent in their message.
When interviewing someone try using reflective phrasing such as –
“So, you feel…”
“It sounds like you…”
“You’re wondering if…”
“For you it’s like…”
Note that these examples include the word “you,” which emphasizes that the interviewer is actively listening and reflecting back what he or she heard the interviewee say. This helps to check for understanding.
reflective listening
Process of synthesizing and re-stating what the interviewee has said in order to capture key concerns and issues.
To confirm that you have understood the interviewee’s story or issue.
To change topics, close discussion, or clarify something.
To collect your thoughts
To show the interviewee that you have heard and respect their point of view.
summarizing
a question composed of more than two separate issues or topics, but which can only have one answer.
Example:” Are homeless shelters expensive and useful?”
double barreled question
a question composed of two negative words.
Example: “Are you not never left home alone?
double negative question
Leading questions push respondents to answer in a specific manner, based on the way they are framed.
Example: “Tell me how useless homeless shelters are”; “How guilty do you think they are?”
leading questions
the range of processes and procedures whereby we move from the qualitative data that have been collected into some form of explanation, understanding, or interpretation of the people and situations we are investigating.
Qualitative Data Analysis:
a process of breaking down data into smaller units, determining their importance, and putting pertinent units together in a more general form.
data analysis
data cleaning’
The process of preparing and organizing raw data into meaningful units of analysis:
Text or audio data transformed into transcripts.
Image data transformed into videos, photos, charts.
As you review your data, you find that some of it is not usable or relevant to your study.
Raw Data Management
‘chunking’
Data reduction forces the researcher to make choices about which aspects of the assembled data should be emphasized, minimized, or set aside completely.
Immersion: Get a sense of the data holistically, read several times
Classify and categorize repeatedly, allowing for deeper immersion.
Memoeing: Write notes in the margins.
Chunking: Preliminary classification schemes emerge, categorize raw data into groupings
Data reduction
a process of reducing the data into smaller groupings so they are more manageable.
The process also helps you to begin to see relationships between categories and patterns of interaction.
Sections of text transcripts may be marked by the researcher in various ways (underlining in a colored pen, given a numerical reference, or bracketed).
coding
generated when similar issues and ideas were expressed by participants.may be labeled by a word or expression taken directly from the data or by one
created by the researcher because it seems to best characterize the essence of what is being said.
theme or category
Ordinary themes – themes that a researcher might expect to find.
Unexpected themes – themes that are surprises.
Hard-to-classify themes – themes that contain ideas that do not easily fit into one theme or that overlap.
Major and minor themes – themes that represent the major ideas and the minor secondary ideas.
the action of providing a written account of spoken words.
Transcribing
also known as “intelligent”)transcriptionserves as a good representation of the interview’s content itself. Among all the other types of transcriptions, it usually looks more formal and less distracting, which is why it’s called “clean”. Here are the characteristics of clean verbatim method:
Filler words (uhms, mhhm) are left out.
Repetitive words (like, kind of) are excluded or adjusted.
Stutters and interjections are also excluded.
Readability is improved by correcting minor grammar mistakes and phrasing (for example, by removing slang words)
clean
transcription method not only captures the content of the interview (what?), but also the manner in which it was conducted (how?). Asmooth verbatim transcriptshould include:
All the filler words
Interjections, stutters
Original content with no paraphrasing
No grammar correction
This method is certainly more detailed, than clean transcription. Here the emphasis is on the original content of the interview, but also not to an extent, where every small detail is recorded.
smooth
(also known as “strict”)transcriptiongoes one step further, than smooth verbatim, by considering:
Non-verbal cues
Emotional reactions
Sounds from the external environment.
Pauses
Laughter, throat clearing, body language – those are all noted down by the interviewer. Full verbatim transcription is all about minor details, that make up the context of the interview and behavioral patterns, expressed by the interviewee.
full
In its simplest sense, evaluation research is the process of determining whether a social intervention has produced the intended result.
Evaluation research reflects social scientists’ increasing desire to make a difference in the world. At the same time, there is the influence of:
an increase in federal requirements that program evaluations must accompany the implementation of new programs; and
the availability of research funds to fulfill those requirements.
eval research
people or organizations who have an interest in your research project or affect or are affected by its outcomes.
stakeholder
Inputs: Resources, raw materials, clients, and staff that go into a program (i.e., creating a re-entry program)
Program process: The complete treatment or service delivered by the program (i.e., people begin using services)
Outputs: The services delivered, or new products produced by the program process (i.e., outcome from participants)
Feedback: Information about service delivery system outputs, outcomes, or operations that is available to any program inputs (i.e., meetings and reviews to make better)
Needs assessment: A type of evaluation research that attempts to determine the needs of some population that might be met with a social program.
Ex: Preparation for terrorist events/natural disasters; Identify States with large population cities at risk of terrorist event or natural disaster
Evaluability assessment: A type of evaluation research conducted to determine whether it is feasible to evaluate a program’s effects within the available time and resources.
Process evaluation: Measuring the degree of congruence between the plan for providing services and treatments and the ways they are actually provide.
Example: Implementing body cameras in police departments
Is it having the desired effects, e.g., reducing complaints from public? Reducing excessive use of force? Etc.
Impact evaluation: Analysis of the extent to which a treatment or other service has the intended effect.
Efficiency Evaluation: measuring how the program uses the resources given.
Efficiency analysis: A type of evaluation research that compares program costs to program effects; it can be either a cost-benefit analysis or a cost-effectiveness analysis.
Cost-benefit analysis: A type of evaluation research that compares program costs to the economic value of program benefits.
Cost-effectiveness analysis: A type of evaluation research that compares program costs to actual program outcomes.
Black Box Evaluation: This type of evaluation occurs when an evaluation of program outcomes ignores, and does not identify, the process by which the program produced the effect.
Program Theory: A descriptive or prescriptive model of how a program operates and produces its effects.
Theory-driven Evaluation: A program evaluation that is guided by a theory that specifies the process by which the program has an effect.
Stakeholder approaches: An orientation to evaluation research that expects researchers to be responsive primarily to the people involved with the program.
Utilization-focused evaluation: Evaluator forms a task force of program stakeholders who help to shape the evaluation project so that they are more likely to use its results.
Action research or participatory research: Program participants are engaged with the researchers as co-researchers and help to design, conduct, and report the research.
Appreciative inquiry: Eliminates the professional researcher altogether in favor of a structured dialogue about needed changes among program participants themselves.
Researcher approaches: An orientation to evaluation research that expects researchers to emphasize the importance of researcher expertise and maintenance of autonomy from program stakeholders.
Evaluators who adopt this approach derive a program theory from information they obtain on how the program operates and extant social science theory and knowledge, not from the views of stakeholders.
Integrated approaches: Stakeholders and evaluators: An orientation to evaluation research that expects researchers to respond to concerns of people involved with stakeholders as well as to the standards and goals of the social scientific community.
Quantitative methods compare outcomes.
Used in evaluation research that attempts to identify the effects of a treatment, interest in comparing outcomes between experimental and control groups, or tracking change over time in a systematic manner.
Qualitative methods: Program process
Used in investigating the program process and help in learning how different individuals react to the treatment.
They help in understanding how social programs actually operate.
