POST-MIDTERM Flashcards
Give an example of field research. Depict the genral assumption, the methods, the findings, the general interpretation. What will follow this ? What is notable? What did the researcher note ?
Social organization in a public housing project
- General assumption: public housing projects are often associated with high levels of crime and low levels of social control.-The researcher chose a British community with i) a high level of crime and ii) containing a significant number of public housing projects.
- She spent 18 months “getting involved in as many aspects of life there as possible from attending tenant meetings, the mothers and toddlers group, and activities for young people, to socializing with some of the residents in the local pub“.*Foster, “Informal social control …“, British Journal of Criminology(1995)3
Conducted “extended interviews” with 45 residents in the community, a few from a contiguous community, and 25 ‘officials‘ – e.g., police, housing officers.
- Found:
- Crime was not perceived as a major problem by the residents.Crime existed – but was tolerated by the residents.
- Housing problems were seen as a major problem.
- There were “hidden economy“ crimes – for example, hiding income from welfare inspectors.
- There was fairly effective social control – for example shaming practices that reduced or prevented crime.
- Note the absence of hypotheses – the work was organized around a general interest in the subject rather than specific hypotheses.
The larger interpretation: crime is less of a problem for people (in high crime areas) where i) there are support networks – people can watch out for each other - and ii) there was someone to whom people could turn when they had a problem.
- She might have gone on to test this through further research on support networks and the characteristics of the people to whom residents could turn for support.
- The researcher emphasized that, through her fieldwork, she became an insider and this allowed her to go beyond outsider perceptions of what a high crime community is like.
Depict web surveys.
Their use has increased considerably.
They can take one of two forms: i) a respondent is directed to a web site with a questionnaire; ii) the questionnaire is either embedded in, or attached to, an e-mail message.
Information on progress through the questionnaire can be provided (percent completed, a bar crossing a box).
They have a relatively bad reputation.
What is significant about participant recall ? What should be done ?
There is strong evidence (for example, from studies of crime victimization) that the accuracy of recalled information declines over time.
This does not mean that questions requiring recall should not be asked. It means that the questions should be formulated to aid recall
What is the guetemala experiment, what are its ethical issues ?
- This study was also funded by the U.S. Public Health Service.
- It was led by a researcher who went on to participate in the later stages of the Tuskegee experiments (which continued to the beginning of the 1970s) along with researchers from Johns Hopkins University and the Rockefeller Foundation.
- The study was conducted from 1946 to 1948, after the discovery of penicillin. Its purpose was to assess the effect of penicillin the prevention and treatment of venereal diseases.
- The research involved deliberately infecting prostitutes, prisoners, soldiers, and mental health patients with syphilis, gonorrhea, and chancroid.
- 1,308 people Guatemalans were exposed to one of these diseases; 678 received treatment. There was, then, an untreated control group.
- Guatemala was chosen for the study because such research could not be conducted in the US. Guatemalan government officials agreed to a study but the details of the research appear to have been withheld from them.
- While the process of infecting subjects stopped in 1948, patient follow-up and analysis of the data continued into the 1950s.
ethical issues
- The subjects appear to have not been informed of the nature of the study and most of them were not in a position to give consent. Informed consent, then, was absent.
- Subjects risked or experienced harms.
- The researchers were assessing the effectiveness of penicillin. Had it been ineffective the subjects would have been harmed.
- Even though penicillin proved effective it is likely those who received it would experience some physiological damage from the initial infection.
- The control group members were left untreated
What is informed consent ? what does it include (7)
- Participants in research must give informed consent (cf. the Tuskegee experiment). Informed consent statements usually include:
- a description of the purpose and procedures of the project, including expected duration;
- a discussion of any risks or discomfort that might be associated with the project;
- a guarantee of anonymity and confidentiality of records;
- the coordinates of the researcher, so that subjects can contact him or her with any questions;
- a statement that makes it clear that participation is voluntary and that the subject can withdraw at any time;
- a statement of alternative procedures that may be used (if applicable);
- a statement of benefits or compensation that might be provided to subjects;
- an offer to provide a summary of the finding
_____________ is the major problem with experimentation in the social science.
External validity
What should we use when dealing with randomization?
There WILL be sampling error associated with each groups so we want to use inferential statistics. Sampling errors requires proceeds that respond to them, which is inferential statistics.
What is an important ethical component of social research? why is it a problem ?
Some research is only possible on deception
Deception is a problem because subjects need to get inform consent. Which they cant give if there are being decepted
What are data collection methods in surveys ?
Self-administered questionnaires
Phone surveys
Web surveys
Face-to-face interviews
What are open-ended vs closed ended questions?
Open-ended: respondent is asked to provide his or her own answer to a question
Closed-ended: respondent is asked to select an answer from a list provided by the researche
What is sampling error?
The difference between a sample statistic used to estimate a population parameter and the actual (unknown) value of the parameter.
Depict focus groups. what can they be used for ? how many people in them ususally- how do you determine that ? who are they led by ? when would you use a smaller group? How do you compose your group? what is valuable to do during them ?
- Interviews using a small group rather than a single individual.
- Have the advantage that they yield information both on people’s views about an issue and on their interactions over the issue. Bear in mind that people’s opinions are usually developed through interaction with others.
- Those interactions yield information on negotiated agreements and disagreements and on shared meanings and understandings.
- Can be used to test questions to be added to surveys – the focus group discussions, for example, can be used to identify response options of closed-ended questions.
- They typically involve 6 to 10 people. This is because: i) as group size increases so do scheduling difficulties; ii) in larger groups some members are likely not to participate.
- Complicated or emotionally charged issues are thought to work better with smaller groups.
- It is usually better to group participants with similar characteristics or experiences – though this would not be the case if the researcher were interested in understandings negotiated across social categories – for example, between men and women
- Groups can be stratified according to age, gender, race, occupation, etc., in order to explore the extent to which responses vary across these categories.
- Groups are usually led by a trained moderator who facilitates and guides the discussion.
- How many groups? As in qualitative research more generally, stop running groups when response saturation occurs. This tends to happen at 10 to 15 groups.
-Taping discussions is valuable. The moderator needs to ensure that the taped voices can be matched to participant
What are pros of face to face interviews ?
Pros
–High response rate (around 80% to 85% for the best survey organizations, like Statistics Canada)
–Decreased number of “don’t knows” and “no answers”
–Questions can be clarified
–Probing for alternative answers
–Can reach a population that has no fixed residence
–Interviewer can observe respondents and the setting in which they are interviewed
What does present value mean ?
- Cost and benefits are spread over time.
- Assume: income earlier is better than income later.
- If I get income today I might choose to invest some of it and get extra income in the future.
- If I get income today, I may not have to borrow money and incur interest payments.
- There is a rate of interest, which is usually positive: it implies that to get money now I have to pay a larger sum (principal plus interest) spread over some period of time in the future. Evidently, this implies that money now is worth more than money in the futur
What are general problems of ethics during social research ? how can they be triggered ?
More general problem : psychological abuse, stress, loss of self esteem — falsely telling men that they have feminine personality, creating situation of high fear, gruesome photos, asking to harm others, falsely tell students they failed a test
What is the difference between secondary data analysis and use of existing statistics
Secondary data analysis
- Data collected by others
- Researcher’s own statistical analysis
- Often done in quantitative research
Use of existing statistics
- Data collected by others
- May be someone else’s statistical analysis – though descriptive statistics from a source can often be combined for statistical analysis.
- Used in both quantitative and qualitative research
Considers interval validity in the case of One-group pretest-posttest.
History – other events may occur between the two observations, especially as the time period between the observations lengthens.
Maturation – between the two observations the students may grow older, more fatigued, bored (each depending on the time lag between observation).
Testing – for educational tests, for example, the first experience with the test may improve students performance at the second test. Scores on IQ tests increase by 3 to5 points without any supplemental training
Instrumentation (‘instrument decay’) – there may be changes in the measuring instrument. Observers of classroom behaviour may get tired, or more skilled at observation, or more blasé.
Regression towards the mean – in remediation instruction where poor performers are selected because they did poorly, there are likely to have been random effects accounting for the initial poor performance that will not be present during O2.
Since there is only one group there are no issues of differential selectionor mortality.
What are the 2 examples of regression towards the mean ? What did we use in the second to counter act ?
Low end of the distribution example: those who did badly are given a coaching program, they are tested again, their scores go up, can we infer it is because of the program ? If you observe the scores of a test , two things influence : 1. Cognitive ability 2. Ability and Radom factors. So we looking at the bottom 20%, the random factors will most likely be negative. You cant infer causation here because the low scores of this selected group had an accumulation of negative factors and It is unlikely they will experience the same negative factors. For the lower ground, it is likely that their performance will improve regardless because they were subject to negative factors.
Higher end of distribution example: How do you chose an advisor ? Rating based on previous performance. But this not valid, because the random factors came into play- they will probably come back down again, they are likely to do worse because their first performance was based on luck. Those who did well tend to regress towards the mean. The high ranking are produced by 1. Ability knowledge 2. Luck. So mostly firms who did well will do worst later, those who did worst will do better later
So we use Index Funds: buy a distribution of share that represent the distribution of the market as a whole. (Instead of having someone picking shares for you) So your performance is based on the market not on individuals, plus saving the fees.
Depict unstructured interviews. What is it in practice ?
- ‘Unstructuredness’ is, in practice, a question of degree.
- The interviewer will have a list of topics. But the course of the interview and the topics may diverge considerably from the list.
- Thus, in research on ‘clubbers’ “interviews were very much ‘conversational’ in style, although all the interviews were taped. The first interview was designed …. To put the clubber at ease while also explaining fully and clearly in what ways I was hoping for help; to begin to sketch in details of the clubbers preferences, motivations and histories; and to decide how to approach the night(s) out that I would be spending with the clubber … The main content of the second [more relaxed] interview consisted of comments, discussion, and the question about the club visits … And the nature of the night out as an experience … [D]iscussion occasionally diversified …. To cover wider aspects of the clubbers’ lives.” (Malbon, Clubbing: Dancing, Ecstasy and Vitality, 1999].
- It should be clear, nonetheless, that there is more structure than in a casual conversation because the research will be motivated by some initial ideas about what to look for.
Depict strengths and weaknesses of qualitative interviewing/
Strengths:
Respondents play a larger role in determining the content of the interview. This means that their views are more likely to be faithfully represented.
There is the possibility of unexpected topics being raised. These unexpected topics may provide the basis for new theorizing.
Weaknesses
The very rich, descriptive, data yielded by the interviews is often hard to analyse. Software described by B&R considerably helps with this.
- Since each interview is a different conversation reliability is an issue. The ‘measurements’ (so to speak) of other researchers examining the same subject are unlikely to tap exactly the same thing. Nor, for that matter, need consecutive interviews by the same researcher.
- Qualitative interviewing necessarily involves small samples, so generalization is likely to be a problem
What are self-administered sruveys ? How do they work ? How are response rates? What can increase their responses, how (4)?
The most common form is the mail survey: a questionnaire, a set of instructions, and envelope for the respondent to send back the completed form.
For anonymity, include a postcard with the questionnaire – to be returned at the same time. This allows you to keep track of who has returned their questionnaire, without identifying which questionnaire belongs to which respondent.
Response rates vary. They are lower than telephone or face to face interview response rates.
Follow-up mailings increase the survey response and may be administered in various ways:
- Letters of encouragement to non-respondents;
- New copies of the questionnaire with a follow-up letter.
- A total of three mailings is usually about right.
- Two or three weeks between each mailing is reasonable.
How can social research create a possibility for legal harm ? what is also worth questioning ?
There is possibly for legal harm to subjects
For example : reporting people who violated law , leading to direct worsening situation by raising security and therefore uncover cases
Is not reporting observed behaviour also a Crime ?
Of challenges to external validity, two are not solved by any of the designs we’ve considered, which ones ?
the interaction between selection and the experimental effect, and the effect of experimental artificiality.
What are the 4 potential purpose of a survey ?
- Exploration
- Observation
- Explanation
- Generate data of a large range of items
What is the The Tri-Council Policy Statement? What does it involve (7) ? What does that mean for universities ?
The Tri-Council Policy Statement on the ethics of research involving humans demands:
- respect for human dignity;
- respect for free and informed consent;
- respect for vulnerable persons;
- respect for privacy and confidentiality;
- respect for justice and inclusiveness;
- balancing harms and benefits;
- minimizing harms.
- Universities that wish to receive funding from one of the councils must develop procedures that are consistent with these broad principles, including ethics review committees
What are the main advantages of non-probability samples? (3)
Advantages:
- Less expensive than probability
- Often the only feasible method
- Can yield info that may not be available in probability sample with a moderate size. For example, we can illuminate information on: deviant/unusual cases, cases of people whootherwise would’ve been impossible to collect through probability sampling, cases of people who represent minorities
What are quasi-filter questions ?
Quasi-filter: Here is a statement about another country: “The Russian leaders are basically trying to get along with America.” Do you agree, disagree, or have no opinion on that?
Describe the four types of probability sampling.
Simple random sampling/equal probability of selection: Participants are randomly selected
Systematic sampling: Potential participants are on a list. Every kth participant is included in the sample.
Stratified: Researcher divides sample into separate groups (strata). Then, a probability sample (often random) is drawn from each group. Ensures that your sample proportions are equivalent to population proportions.
Cluster: Researcher divides population into separate groups (cluster). Then a simple random sample of clusters is selected from the population.
what are 2 research on qualitive intervews example? how do they differ?
A research example (1)Milgram, Obedience to Authority (1969)
- Recruited men to participate in a study ostensibly on memory and learning.
- Research subject was assigned the role of ‘teacher’: asked to test learners’ memory of word lists and to apply electric shocks of varying severity when the learner gave the wrong answer.
- The learner was, in fact, an actor.The ‘learner’ was not visible to the ‘teacher’ but could be heard by him. The ‘learner’/actor signalled discomfort and pain in response to the shocks.
- Most of the ‘teachers’ accepted the instruction to apply shocks up to levels marked as dangerous on the gauge of the equipment they were using.
- Evidently, this is experimental research, not a focus group.
Gamson, Fireman, and Rytina, Encounters with Unjust Authority (1982
- The researchers presented themselves as market researchers and recruited participants for their study.
- The participants were read a summary of a legal case said to be before the courts.
- The case put an oil company in a highly unfavourable light.
- But the coordinator pressured the participants to “give false opinions (favourable to the sponsoring oil company) for the videotape being made for a potential legal proceeding“.
- Many of the participants refused to do so.
- What was the difference with Milgrom‘s experiment? Its focus group-like character meant that social support led many participants to resist the authority of the (apparent) researche
What are different methods of measuring dispersion?
Range: Maximum value – minimum value
Standard deviation: Average of the squared deviations from the mean
Depict and illustrate open coding .
Open coding illustrated
(1)Robson studied the long-term effects of being bullied as a child.
When she open coded her interview transcripts the following themes emerged:
distrust of intimate relationships in adulthood;
empathy toward other people;
awkwardness in social settings
Here are responses that were open coded awkward in social settings.“I try to avoid group situations out of habit … Not that I fear anymore that I am going to get teased or harassed or fucked with, but I just have these habits that are hard to break. Avoiding groups, avoiding being the centre of attention, because when I was the centre of attention when I was growing up, it was for bad things, so now it’s hard to be centre of attention even though it might be for something positive.” (male, mid 20s)
It’s only been since I’ve been married that I’ve started to form good solid relationships with women … I’m still not really confident in social situations around people that I don’t know very well, I’m not confident … I think that the friends I do have are a lot more deeper and more meaningful because I knew how to form those when I was a kid – deeper more meaningful relationships.” (female, mid 20)
If one wish to predict the scores for each party, what decision rule should we use ?
f we assign 51%, we will make fewer errors
Where possible, in research design, randomization is _________. Why?
Where possible, randomization is always better than matching because random assignment deals with unobservable as well as observables. When matching, there might be unobservable characteristics not part of the matching process which influence the result, you get a distribution within the two groups taking into account both kinds of characteristics.
Depict the he Community Employment Innovation Project program in brief
The project involved the following components:
- HRSDC provided $30,000 in planning funds to each community;
- communities mobilized local project sponsors to identify community projects that might usefully be implemented;
- unemployed community members could be hired to carry out the projects and were paid wages from $280 per week early in the project to $325 per week later in the project, and were covered by EI and Nova Scotia Workers’ Compensation;
- they worked for up to 35 hours per week;
- they could be hired for up to three years;they could switch between the CEIP work and other employment;
- compensation included statutory holidays and ‘personal days’ that could be used for vacations or sick leave;
- projects included local beautification, services for the poor, support for seniors, support for youth, services to the disabled, the arts
What is explanation ? give an example.
when the addition of the control/test variable causes the zero-order relationship to disappear and it does so because the original relationship was spurious
Relationship between regular breakfast consumption and the teenage pregnancy
The percentage point difference is 25; there is clearly a zero-order relationship between breakfast consumption and pregnancy.
Does the table settle the issue of causation?
It is possible that the relationship is spurious. For example, regular breakfast consumption may be associated with a stable home life.
So: control for time spent at home
For parents who spent both high and low amounts of time at home the percentage point differences between breakfast pattern and teenage pregnancy effectively disappear.
The percentage point difference before controlling for occupation was 25.
Evidently, the partial relationships differ from the zero-order relationship
The result of adding the test variable (parents’ time at home) has the same effect on the relationship between breakfast habits and pregnancy as did adding occupation on the relationship between gender and pay.
what does the correlation coefficient measure ?
the distance between the point and the line
Depict Systematic Sampling
Systematic Sample: you work with a list and takes every kth element from the list , Implies sampling interval equal to population sample size. ( the sampling interval = Sample ratio = sample size/population size = 100/1000= 1/10)
What is interpretation? Give an example
after adding the test variable the association between the two variables of interest disappears
i) adding a test variable eliminates the relationship between the independent and dependent variables in partial tables and ii) the test variable can be considered to be caused by the independent variable but prior to the dependent variable, within the elaboration model that is called interpretation
Income, gender, and medical occupation
The percentage point difference is 18; there is clearly a zero-order relationship between gender and income.
Does the table settle the issue of causation?
It is possible that the relationship is spurious. For example, male medical professionals may be in different, and differently paid, occupations from female medical professionals.
So: control for occupation
The percentage point differences for both doctors and nurses effectively disappear.
The percentage point difference before controlling for occupation was 25
We know that occupation does not cause gender; that is to say, any causal connection must go from gender to occupation, rather than vice versa.
It is also most likely that occupation causes income rather than vice versa. So, occupation must be an intervening variable, between gender and pay
What are the 5 ways of collecting data ?
Experiments
Surveys
Qualitative interviews
Observation
Unobtrusive Research
Depict and illustrate Axial coding.
A study of a marijuana legalization group in Western Canada generated the following open codes: distrust, partying, and marginalized people.
Then the researcher attempted to connect these coded observations.
The use of marginalization was linked to distrust of various entities – the government, the police, for example. The researchers conclusion: marginalization led to distrust of institutions.
But distrust also led respondents to marginalize themselves, as did their models of ‘partying’; both distrust and partying led to marginalization
What is the major purpose of program evaluation research ?
se programs involve expenditures. This raises the question: Are the expenditures worthwhile?
What can we say about the relevance and threats of knowlege questions?
Researchers may be interested in the extent of knowledge of something or other.
Such questions may be threatening to respondents who are reluctant to reveal their ignorance.
What is a problem in using official statistics as data in research ? What are examples ? What does that mean?
- Official statistics are the result of a social process (reporting and recording a crime, deciding which crimes to report, court decisions on the fact and character of a sexual assault).
- Statistics agencies have different practices (StatsCan versus US Census Bureau).
- What statistics means depends on institutional design and practices (high school graduation rates in Australia, drop-out rates in Quebec)
- Different data sources have different strengths and weaknesses (inequality – the Census versus survey data)
- Crime, Rape, Australia graduation, Cannabis income, income inequality in canada (who tend to undercount the very poor)
The unit of analysis is usually an aggregate; there is the problem of the ecological fallacy.
- Using already collected data means that the validity of the measures may be problematic (e.g., the unemployment rate and the state of the labour market).
- Reliability may be problematic – for example, changes in the definition of rape/sexual assault over time (and no doubt across countries).
- Conclusion: for the purposes of research, official statistics are often flawed. Therefore, use them cautiously! For example, use multiple indicators of the state of the labour mark
Describe the pretest-posttest control group design. How is it fundamentally different from a pre-experimental design? What does it resolve ?
R O1 XO2R O3O4
- This is a true experimental design. We can compare O2-O1 with O4-O3.
- Equivalence of the groups at the first point of observation is achieved through randomization, not through matching
- Solves issues of internal validity
Questions should provide categories that are…. (3)
Mutually exclusive. The respondent can plausibly fit into one, and only one, of the response options, which is not the case if the categories overlap.
Exhaustive. The categories that all respondents can fit themselves into one of the categories provided. “Are you working or unemployed?” is unsatisfactory because it excludes the category “not in the labour force”.
Balanced. Usually, it is better to allow responses that range between equal, polar, opposites – say, from very satisfied to very unsatisfied – as in a typical Likert scale
What are the 3 true experimental design ?
- Pre-test, Post-test, control group design
- Post-test only control group design
- Solomon four group design
Depict and explain randomization ? How did Pajer constitute his groups ?
R= randomization which deals with how we constitute the groups we study.
Randomization= number table and assign people based on the number generate
What do we have to think about when we consider external validity?
Interaction
You have two things present, the stimulus and the conditions of experiment and its the interaction of the two that creates a treat to generalizability.
What are advantages and disadvantages of focus groups?
advantages:
They provide the social context for judgments that exist in real life.
- Group dynamics may bring out unanticipated issues – that might have been hidden had individuals alone been interviewed.
- There is high face validity – what is being measured is identified through discussion among group participants.
- Relatively low cost.
disadvantages:
The social dynamics of the group may be hard to control. Some members may dominate the discussion.
- Taped discussions are challenging to analyze
- How differences across focus groups in conclusions or views should be interpreted is not always obvious. That may indicate a problem of reliability
Sampling methods fall into one of two categories:____________ and ____________. Explain the difference between these categories.
In probability sampling methods, each population element has a fixed and known (non-zero) chance of being chosen for the sample. In other words, subjects in a population have a random and therefore equal opportunity to be selected as a representative sample.
In non-probability sampling methods, we do not know the probability that each population element will be chosen, and/or we cannot be sure that each population has a non-zero chance of being chosen. The basis of selection is likely to be arbitrary and opposed to random, so the opportunity of being selected is not specified and is unknown.
What are problems in cost-benefit analysis ?
- How valid are the measures of benefits (or costs) that don‘t usually have dollar values?
- What interest rate should be used to discount to a present value? The Community Employment Innovation Project used a 5% discount rat
What are the 2 types of question? How do you choose between them?
Statement: “I regularly get enough sleep at night” - Likert scale
Question: “How many hours do you regularly sleep at night” - 1-5, 6-7, 8-9
*Choice between question and statement- best of all is that you ask both to get more information
-Ask them both to get information about how they sleep and how they feel about it
What is the elaboration model ? when do we use it ? what is its logic ? what does it specifies?
We want to explore the causal relationship between two variables.
We know that other factors might influence one or both of the variables.
The elaboration model specifies: i) how we might go about analyzing the relationships between three or more variables; ii) identifies different possible ways of interpreting tables after a third variable has been adde
Give example of survey question and statement. Would you use both, explain ?
Questionnaire construction: Questions versus statements
How many hours do you regularly sleep at night?
Less than 4
5-6
7-8
9 or more
How do you feel about the following statement: “I regularly get enough hours of sleep at night”
Strongly disagree, disagree, agree, strongly agree
If you include both, you can compare the responses using the two different measures
What are the problems of Interval validity ? (7)
History : it is possible that between o1 and o2 , x might not have been the only thing that happened. Other evens might of happened that explain the change between o2 and o1
Maturation : time passes between o1 and o2 , the subjects might change, the difference may not produced by x but by changes in subjects produced by time
Testing: If there is a pre-test, is possible that the post pre-test results are determined by the pre-test rather than the experimental effect. Initial observation could change attitudes or behaviours of the subjects for the second observation, may not be do by the intervening variable. the first observation may change the second of x ( o1 may produce o2)
Instrumentation: does the measurement instruments remain the same between the observations
Statistical regression towards the mean: If you select members on the basis of some extreme performance it is likely that their performance will change overtime with or without experimental effect. When you have an extreme score, it is produced by what we are trying to measure AND random factors, the score at a second point in time is likely to get closer to the mean
Low end of the distribution example: those who did badly are given a coaching program, they are tested again, their scores go up, can we infer it is because of the program ? If you observe the scores of a test , two things influence : 1. Cognitive ability 2. Ability and Radom factors. So we looking at the bottom 20%, the random factors will most likely be negative. You cant infer causation here because the low scores of this selected group had an accumulation of negative factors and It is unlikely they will experience the same negative factors. For the lower ground, it is likely that their performance will improve regardless because they were subject to negative factors.
Higher end of distribution example: How do you chose an advisor ? Rating based on previous performance. But this not valid, because the random factors came into play- they will probably come back down again, they are likely to do worse because their first performance was based on luck. Those who did well tend to regress towards the mean. The high ranking are produced by 1. Ability knowledge 2. Luck. So mostly firms who did well will do worst later, those who did worst will do better later Index Funds: buy a distribution of share that represent the distribution of the market as a whole. (Instead of having someone picking shares for you) So your performance is based on the market not on individuals, plus saving the fees.
Differential Selection : If you have two groups, can you be confident the two groups are constituted from the same population.
Experimental Mortality: Do people disappear more frequently in one group than the other.
What does Skopol mean by social revolution ? What are two sufficent causes according to him?
By social revolutions Skocpol means: i) a change in state structures – a political revolution; and ii) a change in social structures by which she means which class is dominant.
There are two sufficient causes of a social revolution: i) a crisis of the state (military defeat, sovereign default); ii) a pattern of class dominance which determines who will lead the revolution
What are 2 Quasi-Experimental Design? Depict them.
Times-series experiment
- common in workplace experiments
- O1, O2, O3, O4 x O5, O6, O7, O8
________
_________/
Equivalent time sample design
- x0 O0 x1 O1 x0 O0 x1 O1 …..
- x is introduced then reduced, then introduced and so on
Evalutate the interval valitidy of Pre-test, Post-test, control group design (R O1 x O2, R O3 O4).
Are Internal Validity issues adressed ?
- History: yes. Because you are comparing to a control group. Whatever other factor that could influence must have been present for both groups.
- Maturation: yes. The two groups mature at the same rate.
- Testing: yes. Both groups have a pre-test so there should not be differences between the two groups resulting from testing.
- Instrumentation: yes. The observer mature at the same time.
- Regression towards the mean: yes. there should be regression towards the mean in both but if one is larger we can see the size of that effect is.
- Differential selection: yes. Because we’ve randomly selected the members and we’ve dealt with the sample error using inferential statistics.
- Experimental Mortality: no. x might have an effect on the willingness of people to remain in the study (positive or negative)
What is the formula for a bivariate equation What about a mutliple one regression ?
The bivariate equation is yi= a+bx
Where: yi = the values of a dependent variable
a = the intercept
b = the regression coefficient
xi = the values of an independent variable
The equation for two independent variables isyi = a+b1x1i+b2x2i
What are open and close ended questions ?
Open:
What is your favourite TV program ? (factual question)
Whats the most important issue in Canada today ? (judgment question)
Close:
respondent is asked to provide an answer based on a list designed by researcher (answer structured by the list)
Close ended = 1 to 5 and list (+other)
What are the findings of CIEP?
- Of those who attended the information session, 1006 out of 1620 EI recipients signed the enrolment form while 516 of the 557 IA recipients did so. Note the self-selection into samples this implies.
- The samples differed:
EI 58% males, IA 62% females;
EI mean age 40, IA mean age 35;
EI 69% high school diploma; IA 60% high school diploma;
EI more likely to live in two person household;
EI household incomes on average higher than IA household incomes;
IA sample reported somewhat lower health.
Note that these differences in personal characteristics may influence the extent to which the program affects those enrolled in it.
)As compared to the control groups:
employment rates for both EI and IA participants during the period of the program were higher, then dropped to the same when the program finished;
14 months post program, EI and IA participants earned about the same as controls; their earnings had fallen back to pre program levels.
immediately at program end, more EI and IA participants collected EI (of course, during the program fewer of them collected EI), then the prevalence of EI collection fell back to the same level as the controls.
As compared to the control groups:
both EI and IA participants tended to have somewhat more highly skilled post-program jobs (this raises the questions: why was their pay not higher?);
both EI and IA participants had about the same attitudes towards work (“I like going to work“) and welfare (“It‘s wrong to stay on welfare“);
EI participants developed somewhat more social ties that might be used to find jobs, but only if they had a high school diploma or equivalent;
IA participants did not develop more useful social ties.10
Findings (4)As compared to the control groups:
both EI and IA participants were more likely to have engaged in unpaid volunteer work while in the program (providing information or help educating the public, teaching or coaching for an organization, office or administrative work, collecting serving or delivering food, volunteer driver);
volunteering was not associated with enhanced social capital among the EI participants;
volunteering was associated with enhanced social capital among the IA participants.11
The researchers also looked for community effects through “a quasi-experimental, comparison sites design“ – matching the six communities studied with similar communities. They found:
there were improvements in the performance of the organizations for which program participants worked;there were some increases reported in the average social capital in the communities studied;
some attitudinal evidence of greater ‘social cohesion‘ – trust in police officers, trust in close friends, trust that a wallet would be returned;
more or less no statistically significant differences in employment rates, wages, income, or economic activity
What are Interval Validity and External Validity
Interval Validity : to what degree of confidence we can infer a causal relationship between the variables
External Validity: to what degree are the relationships we find are representative
what is a correlation coefficient ? explain its logic. What can you think of it as ?
There are differences between the observations on the scatterplot and the regression line.
The regression line is calculated to minimize the squared differences between the observations and the line. The regression procedure is commonly called ordinary least squares(OLS).
The larger the distances between the observations and the line, the poorer the line is as a description of the relationship.
Pearson’s product-moment correlation is based on these distances. The larger the distances, the smaller the correlation coefficient; the smaller the distances, the larger the correlation coefficien
In this case r= -0.84.
The square of r (r2) = 0.71 and is interpreted as the proportion of the variance in the dependent variable (number of children) explained by the independent variable (years of education).
You can think of it as a measure of the reduction in error in predicting the dependent variable score as a result of knowing the independent variable scale
what is the research design in the Community Employment Innovation Project
- Within the six CEIP communities, recruit potential participants from people on EI (Employment Insurance) and IA (Income Assistance) by:
- randomly selecting from an HRSDC list of EI recipients;
- recruiting IA recipients from a list provided by the Nova Scotia Department of Community Services of those interested in participating;
- inviting those selected to an information meeting;those selected from each group were randomly assigned to experimental and control groups.
- A “multi-methods” approach to program evaluation:
- individual impacts using i) a baseline survey, ii) three follow-up surveys; iii) administrative data files; iv) CEIPs Project Management Information System;
- community effects using i) a three wave longitudinal survey administered to community residents from the six CEIP communities; ii) local administrative data (for example, on crime); iii) in-depth interviews and focus groups with “key community stakeholders“; iv) local observations (e.g., had the attractiveness of the community improved?); v) scans of local medi
Under what circumstances would there be large divergences in the mean and the median?
The mean will diverge greatly from the median in the circumstance of outlying cases (for example, a median salary for a Country X is $70,000, but the mean income is over $100,000 because a small sect of the population makes over $500, 000).
What are some issues with evaluation research?
The research design.
The specification of outputs – that is, program consequences
The issue of costs and benefits.
Measuring costs and benefits over tim
What do sampling errors require ?
Sampling errors requires processes that respond to them, which is inferential statistics.
What is the problem with the one shot case study ?
It’s problem should be obvious: we have no idea whether or not the stimulus produced any change at all.
There is, of course, an implicit comparison with other events, unsystematically observed. For example, the owner of the restaurant may have a sense of likely tip amounts based on previous experie
What is a posttest-only control group design? What types of advantages or disadvantages does it have in terms of validity compared to the pretest-posttest control group design?
R X O1
R O2
This is a true experimental design
The pretest in the pretest-posttest control-group design makes it possible to establish the initial scores of the experimental and control groups.
But randomization should accomplish the same thing as a pretest (combined with the use of inferential statistics). In fact this design meets the internal validity issues about as well as the pretest-posttest control-group design
And, evidently, the absence of a pretest removes the possibility of the challenge to external validity from reactive effects of testing.
The other two possible sources of external invalidity – selection biases and experimental artificiality - remain.
Still, being able to quantify any change in the score of the experimental group is an advantage of the pretest-posttest control group design
Survey have an _________. Why (3)?
Omnivous character
- Multiple item may be needed to construct indexes and scales
- Hypothesis may be formulated without sharply definite operational framing *** , a survey that contains multiple items allows the exploration of different operationalization.
- once you find a collaborative respondent, you want to get the more of it
Evaluate the internal and external validity of the equivalent time sample design ?
Internal Validity
Hstory: As the number of occasions on which the stimulus is provided is increased the problem of history diminishes. That is, if the same effect is observed on 51 different occasions it is unlikely to be explained by extraneous events.
Maturation: Where the application of the stimulus is relatively short lived aging isn’t a problem. If the experiment were carried out over a long enough time period it might become a problem. If the durations of periods when the stimulus is provided and not provided are the same, then fatigue, etc., shouldn’t be a factor.
Testing: There is no pretest
Instrumentation: Again, the same observations are made during the periods when the stimulus is being applied and when it isn’t being applied; there is no obvious systematic bias from, say, observer fatigue.
Regression towards the mean. Suppose the stimulus is applied to employees selected because of poor performance. Regression toward the mean would cause their scores to go up when they are observed after X1. But since the same group is consecutively studied random negative effects should be consecutively averaged out. This should not be a problem
Selection: There is no control group, so differential selection into the two groups cannot be a problem.
Mortality: A given work group under study is likely to lose current members and gain new ones. But if the stimulus is applied multiple times and the effect is present each time, and absent when the stimulus is not applied, mortality can’t be a problem
External Validity
There may well be reactive effects of testing. That is to say, the provision of repeated stimuli may modify the reactions to subsequent stimuli.
Where a natural group is selected – a work group, a class in a school, say - there is always a possibility that any effect of the stimulus may only be produced for subjects of the kind of group being studied. There is likely to be, then, an interactionbetween selection biases and the experimental variable.
Providing and withdrawing music (the example we considered) is a highly artificial experience. There may, then, be reactive effects of experimental arrangement
Depict this regression : number of children = 7.85-0.35(years of educatio
-7.85 is the intercept (the point where the line crosses the Y axis;
–0.35 is the reduction in the number of children associated with one extra year of education. That is, on average, each extra year of education reduced the number of children by about a third.
- Evidently, this is simply the formula for a straight line.
- The relationship is negative – hence the minus
What are advantages of self-administered surveys? (4)
Modest cost – printing and mailing expenses, not hiring or training interviewers.
People may feel more comfortable giving truthful answers to sensitive questions. But this isn’t clear in the data reviewed earlier.
The characteristics of an interviewer will not affect the respondent’s answer.
It allows more time to give well thought-out answers.
Consider the interval validity of equivalant time sample experiment.
History: As the number of occasions on which the stimulus is provided is increased the problem of history diminishes. That is, if the same effect is observed on 51 different occasions it is unlikely to be explained by extraneous events.
Maturation: Where the application of the stimulus is relatively short lived aging isn’t a problem. If the experiment were carried out over a long enough time period it might become a problem. If the durations of periods when the stimulus is provided and not provided are the same, then fatigue, etc., shouldn’t be a factor.
Testing: There is no pretest
Instrumentation: Again, the same observations are made during the periods when the stimulus is being applied and when it isn’t being applied; there is no obvious systematic bias from, say, observer fatigue.
Regression towards the mean. Suppose the stimulus is applied to employees selected because of poor performance. Regression toward the mean would cause their scores to go up when they are observed after X1. But since the same group is consecutively studied random negative effects should be consecutively averaged out. This should not be a proble
Selection: There is no control group, so differential selection into the two groups cannot be a problem.
Mortality: A given work group under study is likely to lose current members and gain new ones. But if the stimulus is applied multiple times and the effect is present each time, and absent when the stimulus is not applied, mortality can’t be a problem.
Distinguish open and focussed coding.
Open coding, whether used with grounded theory or not:try to identify as many themes as possible that emerge from the data; In doing so, delay considerations of connections between themes until later.
Focussed coding:
- having previously identified topics/categories of interest read through the data to note occurrences;
- discard codes that don’t yield interesting information
In a contengency table , how do you % ? what do you ask yourself?
Want to % on the independent variable.
Do you % by row or by column
What is the tuskegee experiment ? what are its ethical issues ?
The Tuskegee experiment
A study of the evolution of syphilis among a sample of Afro-American sharecroppers in Tuskegee, Alabama, began in 1932.
The study was conducted by the U.S. Public Health Service.
There were 399 subjects, all of whom had previously contracted syphilis. They were offered free medical care, meals, and burial insurance. The men were not told that they had syphilis. There were 201 non syphilitic controls.Subject consent to spinal taps (seeking evidence of neurosyphilis) was secured by describing the procedure as a free treatment.The study continued to follow the development of the disease in the men until 1972.2
ethical issues
Deception was used – for example, the spinal tap procedure.
At the point the study began there was no reliable treatment for syphilis; so simply following the disease implied no harm for subjects. (Before the 1940s treatments for syphilis involved the use of some highly toxic substances.)
In the 1940s penicillin, an effective treatment for syphilis, was discovered. This treatment was not made available to the study subjects.
During WWII 250 of the subjects registered for the draft, were diagnosed with syphilis, and ordered to get treatment. The researchers prevented them from getting treatment, preventing them from serving in the military and benefitting from the provisions of the G.I. Bill, passed after the war
From 1946 to 1948 there was an extension of the experiment in Guatemala. There, 696 men and women – prisoners, soldiers, mental hospital patients – were deliberately infected with syphilis and, in some cases, gonorrhea. After infection they were provided antibiotic treatmen
Depict and demonstrate regression/correlation measure
Estimate a regression line = line that best fits this data
Line that minimize the distance between the observations and the line estimated
Squared values have a large effect on lines
Effect of square = Increasing the weight of extreme values
So that line is influenced by extreme values if there are any
What methodical practices does that suggest one should adopt ?
Looking at the scatter plot allows us to see extreme valuables. Before you decide to drop cases, you need to look at the scatter plot. Implications of the results you get.
When you start to look at data , look at the scatter plots
What are true experimetnal desings?
pretest-posttest control group design
posttest-only control group design
What is the elaboration model ?
Adding a 3rd variable
What happen to associate when we had a 3rd variable
The model specifies the ways the relationship may change
how does posttest-only control group design deals with internal validity?
History: not a problem. Anything that happens before X will happen to both
Maturation : not a problem.you have the same conditions under two groups
Testing: not a problem. because there is not pre-test.
Instrumentation: not a problem , you have the same conditions under two groups.
Regression towards the mean : not a problem.
Selection: not a problem, we use randomization
Mortality: maybe but its not due to the variable
What are semistructured interviews? Depict them.
- Evidently, these do not use the simple, predetermined, categories that dominate questionnaire design.
- Rather, the interviewer has a list of topics which can be addressed in no particular order. The flow of conversation with the subject including, the advantage of developing an empathetic relationship with that subject, will determine the order of the questions.
- In addition, subjects not anticipated may be explored in response to the things said by the subject during the interview. This capacity to seize on and explore issues raised during an interview is a considerable strength of this method. This is a major advantage of this method.
- Usually all questions/issues are address
When do you use a scatter plot , when do you use a table ?
Tables : nominal or ordinal data
Scatter : interval or ratio data (super important because you will see something you would not understand)
How do phone surveys work ?
This is a very common method of collecting survey data.
The interviewer calls the respondent (often at home), asks questions and records the answers.
Respondents sampled from lists, phone directories or random digital dialling.
Computer assisted telephone interviewing (CATI) can reliably move the interviewer through complex questionnaires involving contingent questions and skips.
Explain the difference between the mean, the median, and the mode?
Mean: statistical average, found by adding up all given data and dividing by number of entries. Median: the middle number, found by ordering all data points and selecting the one that falls in the middle
Mode: The most frequent number.
What are challenges to interval validity in research design (6)? Explain them.
History: Might events likely to influence the dependent variable but unrelated to the experiment have occurred in addition to the experimental variable?
Maturation: Do the subjects change over time, simply as a function of the passage of time – for example, aging, getting hungry, getting tired?
Testing: Might the experience of a first test change responses at a second test?
Instrumentation: Might the measuring instrument , or people doing the measuring, change in ways likely to influence measurements? For example, do observations become less accurate when observers are tired
Statistical regression (regression towards the mean): Suppose a sample is deliberately drawn from part of a distribution – say, for example, all those who scored poorly on a test. Did random factors influence those scores, random factors that are unlikely to be (as) present for the group selected in a retest? This was identified as a problem in educational research where poor performers on tests were selected to receive some sort of remedial training. It also observed in cases of financial advisers.
Differential selection of respondents to comparison groups: Where there is an experimental and a control group, might the method of assignment of respondents to groups cause the groups to differ in ways that might bias our experimental results?
Experimental mortality: Where there is an experimental and a control group, is there differential attrition from the control and the experimental group
Illustrate and depict Selective coding
In the marijuana legalization group study the researcher went through her notes and identified marginalization as a major theme.
This was partly because she observed that both partying and distrust were linked to marginalization, but not to each other.
Having identified marginalization as a core theme she went through her notes to look for explicit links between marginalization and her other coded characteristics.
She found that under the category distrust, group members talked about encounters with police, views on conspiracy theories, clashes with family members, experiences of harassment by the public.
She concluded that there was mutual causation between distrust and marginalization
