Research designs and quality Flashcards
Please mention the different mixed-method research designs and their characteristics (Greene et al.)?
Mixed method designs(From lecture)
- Triangulation (Simultaneous, Same phenomenon, aim = use different methods is to arrive at the same findings to increase the validity)
- Complementary (Simultaneous, Different facets of the same phenomenon, studied in different ways in order to get a greater and broader understanding of the phenomenon. We would like to enhance our results)
- Development (Sequentially linked, start with one of the methods and ends off with the other)
- Initiation (Sequentially linked, where we via diverging methods try to find shortcomings and contradictions, should not find the first finding invalid, however, should outline what delimit the first finding)
- Expansion Loosely linked, questions informative for one another = main purpose is to extend scope of project
Explain eight key markers of quality in qualitative research:
(Eight ‘Big-tent’ criteria - (cf. Tracy))
Please comment on validity in regards to quantitative research
- Internal validity* relates mainly to the issue of causality where we change one variable that causes a change in other variables. The golden standard is experimenting where we can control the variables.
- External validity* entails generalizing findings beyond the specific research context to other populations or times. High in surveys.
- Construct validity* is the degree to which a study measures what it claims to measure - high in experiments
- Ecological validity* whether the findings are applicable to people’s everyday life - high in field studies
- Statistical conclusion validity* is the degree to which conclusions about the relationship among variables based on the data are correct - high in sample survey
What are the principles of case study?
- Eisenhardt: Case studies are rich, empirical descriptions of particular instances of a phenomenon that are typically based on a variety of data sources. Case studies emphasize the rich, real-world context in which the phenomena occur.
- Case studies can accommodate a rich variety of data sources, including interviews, archival data, survey data, ethnographies, and observations.
- Interviews are a highly efficient way to gather rich, empirical data, especially when the phenomenon of interest is highly episodic and infrequent.
- Abstractly, we can define a case as a phenomenon of some sort occurring in a bounded context. The case is, in effect, your unit of analysis.
- In-depth in one single case but typically a lot of observations and other types of data to gain deep insight into a phenomena. A case study investigates a phenomenon within its real-life context, especially when the boundaries between the phenomenon and context are not clearly evident. We do case study because of:
- Accuracy (context): faithful to everyday realities - what is actually going on
- Comprehensiveness: allows the researcher to maintain holistic and meaningful characteristics
- Richness and depth: complexity
- Dynamic/process: follow developments over time
- What case studies do: describe (new phenomena), explore (new phenomena/perspectives), explain (operational links over time), test a theory (falsify) and tell compelling stories (puts you in the situation)
- På dansk: Et case studier referer til et studie, hvis indsigter ikke formelt antages at gælde for andre end den undersøgte gruppe. I modsætning til studier med sandsynlighedssampling, hvis indsigter antages at kunne generaliseres fra det undersøgte sample til andre personer fra samme population, vil det være op til den enkelte forsker at vurdere om indsigter fra et case studie kan generaliseres til en andre kontekster og/eller andre personer ud over den undersøgte gruppe..
Please explain the principles of phenomenology
- Tends to look at data thematically to extract essences and essentials of participant meanings.
- Study the phenomena; the appearances of things, or things as they appear in our experience. The purpose is to clarify and enlighten how people understand and comprehend certain phenomena.
På dansk: Fænomenologiens ærinde er at beskrive fænomener fra menneskets livsverden. Fænomenet er det, som fremtræder i vores bevidsthed i form af oplevelser, begivenheder eller noget materielt. I fænomenologien forstås den menneskelige bevidsthed som en kropsligt forankret bevidsthed. Fænomenologien bidrager til en dybere forståelse af menneskers levede erfaringer ved at søge bag om de opfattelser, idéer eller stereotype billeder, vi normalt tillægger fænomener i vores livsverden.
Livsverden skal forstås som den hverdagsverden, vi lever vores liv i uden nærmere refleksioner. I livsverdenen deler mennesker omgivelser, sprog og betydning, og det er her, vi gør vores erfaringer (1,2). Disse levede erfaringer er udgangspunktet for fænomenologisk forskning (3). Livsverdenen er således fyldt med mening og betydning, der er tavs, og denne tavse betydning kan bringes frem gennem en fænomenologisk analyse. En sådan analyse beskriver fænomeners betydninger, deres substans og fællestræk, dvs. essensen af de levede erfaringer, uden at bygge på teorier, hypoteser eller forskerens egen forståelse.
What are the principles of grounded theory?
Grounded theory, for example, uses a series of cumulative coding cycles and reflective analytic memoing to develop major categories for theory generation.
The theory is grounded in the actual data, which means the analysis and development of theories happens after you have collected the data (Data collection = Iterative process) – inductive
- ‘A grounded theory is one that is inductively derived from the study of the phenomenon it represents. That is, it is discovered, developed, and provisionally verified through systematic data collection and analysis of data pertaining to that phenomenon’
- In grounded theory research, as in other forms of qualitative inquiry, the investigator is the primary instrument of data collection and analysis. As such, the researcher/analyst relies on skills as well as intuition and filters data through an interpretive lens. (Bowen)
Metode guiden (AU): Tilgangen er induktiv, dvs. en teori konstrueres på baggrund af observationer (i modsætning til deduktiv, hvor observationer fortolkes i lyset af en teori). Formålet med grounded theory er at lade observationer tale for sig selv. Under dataindsamlingen udledes en teori om det undersøgte fænomen. Grounded theory vil i praksis være en dynamisk proces, hvor nye indsigter hele tiden ændrer og optimerer forståelsen for fænomenet. I sin rene form, vil forskeren ikke have noget teoretisk udgangspunkt før dataindsamlingen, som de indsamlede data fortolkes ud fra.
Please explain the principles of ethnography?
- Tend toward the descriptive of peoples and cultures with customs, habits and mutual differences. Thus, observe people in their real-life environment.
- Stays close to the naturalist form of inquiry: that is, (a) extended contact within a given community; (b) concern for mundane, day-to-day events as well as for unusual ones; (c) direct or indirect participation in local activities, with particular care given to the description of local particularities; (d) a focus on individuals’ perspectives and interpretations of their world; (e) relatively little pre-structured instrumentation, but often a wider use of audio and video recordings; and (f) more purposeful observation than in other research traditions.
- Dansk: er en kvalitativ, beskrivende forskningsmetode, udviklet inden for antropologien til at beskrive kulturforskelle blandt samfund og folkeslag. Igennem sin udvikling i amerikansk og europæisk antropologi og etnologi har den etnografiske metode benyttet sig altovervejende af feltarbejde og deltagerobservation som måder til at indsamle data om forskellige samfunds og folkeslags kultur, samfundsorganisation og levemåder. Etnografi er en holistisk metode baseret på det princip, at elementerne eller delene af et samfund ikke nødvendigvis kan forstås uafhængigt af hinanden.
What is a research design?
Paper by Yin
A research design is a logical plan for getting from here (x) to there (y), where here may be defined as the initial set of questions to be answered, and there is some set of conclusions (answers) about these questions. Between here and there may be found a number of major steps, including the collection and analysis of relevant data.
Another way of thinking about a research design is as a “blueprint” for your research, dealing with at least four problems: 1) what questions to study, 2) what data are relevant, 3) what data to collect, 4) and how to analyze the results
What are the characteristics of experimental design
- The experimental/treatment group, receives the treatment, and is compared against a control group, which does not.
- The dependent variable is measured before and after the experimental manipulation so that a before-and-after analysis can be conducted.
- Moreover, the groups, from the same population, are assigned randomly to the two groups
- Law of big numbers: as number of subjects increases, the average of the sample will approach the average of the population
Pro
- Tend to be very strong in internal validity due to high control of treatment settings
- Trustworthiness and strong causal findings due to a high degree of control
- Easy to replicate
Con
- Poor at ecological validity because we do not know how well the findings are applicable to the real world and everyday life
- Low in external validity – Lab settings might not represent the real-world that well
- Possibility for selection bias which can influence the results
What are the characteristics of a quasi-experiment?
Randomization from predetermined groups.
Or
Get to randomization as close as possible through for example matching (trying to get as close to the first group by matching on certain characteristics
Pro
- When randomization is not possible because of constraints
Con
- Not true experiments and the lack of control over assignments of participants
What are the characteristics of a field experiment?
- Have some of the features of a real experiment, but are lacking at some others.
- Occur in real life-settings
- Uses some kind of random assignment but manipulation has to be relevant to working adult participants.
Pro
- Natural everyday environment → high ecological validity
Con
- Difficult to randomly assign people to groups
- Low degree of control compared to lab-experiment (internal validity)
- It can also be low on generalizability to the population (with the study population is not representative of the target population).
What are the characteristics of a cross-sectional design?
Entails the collection of data on more than one case (usually quite a lot more than one) and at a single point in time in order to collect a body of quantitative or quantifiable data in connection with two or more variables (usually many more than two), which are then examined to detect patterns of association.
Pro
- Construct validity can be high if the scales are tested
- High external validity if the sampling is randomly selected from the population
Con
- Poor internal validity because it is hard to establish causal direction from the results.
- Ecological validity is low due to the way of collecting data
What are the characteristics of a longitudinal design?
- Multiple observations on multiple individuals (or any other unit) over time. It is common to distinguish two types of longitudinal design: the panel study (investigates same cases multiple times) and the cohort study (study participants who share common characteristics)
- Survey or secondary data
Pro
- Allows the researcher to look at changes over time
Con
- Time consuming
- Require a large sample size
- More expensive
What are the principles of MaxMinCon?
Investigator attempts (1) to maximize the variance of the variable or variables of his substantive research hypothesis, (2) to
control the variance of extraneous or “unwanted” variables that may have an effect on his experimental outcomes, but in which he is not interested, and (3) to minimize the error or random variance, including so-called errors of measurement..
The principle: The statistical principle behind this mechanism, as stated earlier is: Maximize systematic variance, control extraneous systematic variance, and minimize error variance. In other words, we must control variance.
- We seek to maximize systematic variance by e.g. collecting data from students with different teachers.
We want to make sure we have as much variance caused by/associated with the interesting independent variables as possible. Here, we make sure the treatments are different. -
To control the extraneous systematic variance of the variables that may have an effect but is not of interest
-
How to deal with it
- Choose homogeneous subjects that are ”close” to each other”
- Randomization Both groups and who that gets the treatment
- Build it into the research design Add it as a control variable
-
How to deal with it
-
To minimize the error or random variance by e.g. collecting data at the same time (after lunch break).
- Error variance is the variability of measures due to random fluctuations whose basic characteristic is that they are self-compensating, varying now this way, now that way, now positive, now negative, now up, now down. Random errors tend to balance each other so that their mean is zero, but systematic variance is in essence predictable. Error variance is unpredictable.
- To increase the reliability of measures is to reduce the error variance. Pending fuller discussion later in the book, reliability can be taken to be the accuracy of a set of scores. To the extent that scores do not fluctuate randomly, to this extent they are reliable.
- Another reason for reducing error variance as much as possible is to give systematic variances a chance to show their significance — if they are significant We cannot do this if the error variance, and thus the error term, is too large.
General: Explain descriptive research
- Overall design is rigid
- Research process is structured
- Probability sampling
- Pre-planned design for analysis
