2. Research design and questions Flashcards
What is research?
A collection of information
-Evidence based conclusions about real world problems
-Starts with a research question
Research design - plan on how to answer RQ, plan for sampling
How to find a research question?
- Own interest
- Observation
- Theoretically motivated- do your literature review!
- Gap in our existing knowledge
- Contradicting existing theories
- Empirical evidence contradicting theory
- Boundaries for existing theory
What is a research question?
Exact interrogative statement, aim of the overall study, giving it direction,
relevance and coherence (boundaries).
Success of the project- did you actually answer your research question?
What is a good RQ?
- Gives you a reason to conduct the research
- Specific enough- mentions units of analysis, points to the data needed and
analysis, limit research area/theories - Gives you a starting point to think and write the entire paper as an
argument, with grand finale- answering your RQ - Gives you the terminology that limits your audience and literature
Good and bad RD?
Quality criteria- generalizability, validity, replicability …
* The match between RQ and design determines priorities
* If causal RQ- causality important for RD
* If RQ about population- generalizability important
Bad research design- not thinking through all the choices in the research
process, lacking in allignment
* Patching- I did a mistake in my survey (missing a variable) so now I am
trying to do quick interviews in order to cover it up
* Starting from the wrong end-
What is practices?
Ways to collect and analyze data
Name two methods of collecting data
Quantitative and qualitative
Name 6 research designs
Survey
Interview
Experiment
Observation - naturalistic and participant
Archival - data and textual archives
Combination
What is a survey design
Sample survey- information is gathered from a representative group of people (census- whole population)
When to use survey
Subjective attitudes, values, opinions, intentions
* Social desirability bias- big problem when asking about sensitive topics (although anonymity helps)
* Can understand and remember!
* Objective information that is too complex to access
* Best- combination with objective secondary data
Interview design
- Research interview- detailed information is gathered from a selected individual (representativeness
is important here too, but less so in comparison to the survey) - Aimed more at understanding a phenomenon rather than generalizing
- Frequently combined with different designs
When to use interview?
- Subjective attitudes, values, opinions, feelings- just like survey, but the complexity matters here
- Would take way too many questions in a survey to get the complexity you are looking for?
- Do you have expectations for the answers?
- Internal or external phenomena?
- Surveys often ask multiple respondents about the same external phenomena (to find ”objective” truth)
- Interviews better at capturing internal phenomena
What choices should be made when making an interview?
- Degree of structure
- Types of questions asked- exploratory, descriptive, explanatory, confirmatory…
- Group interviews
- Different interview modes
- Formal/informal site
Experiment design
Randomized control trials- random selection and assignment of participants/cases into
experimental grous, controlled treatment, so the differences in outcomes can be only
due to treatment
When to use experiment design?
Is the research question focusing on causality?
* Is the RQ about cause and effect- asking whether a relationship exists? Or is it asking about a
process- how a relationship unfolds?
* Causality will come at a price- trade-off
* To achieve generalization to entire population through experimentation is difficult and
expensive!
* To avoid self-selection is difficult
* To simulate the real-life condition in an experiment is difficult
* To make people behave naturally is difficult
Which choices can be made for an experiment design?
- Blinding- subjects don’t know in which group they are (single, double, tripe)
- Between and within-subjects desings
- Crossed or nested design
- Laboratory experiment vs. Field experiment, Quasi- and Natural experiment
- Pretesting, registration of hypothesis, treatment fidelity, manipulation checks…
- Is it a control variable or manipulation/treatment?
- Is it a continous variable or categorical and can you change it?
What is mixed design?
Carefully planned utilization of multiple designs in order to answer complex (multipart) research questions
How to combine designs?
How to combine the desings?
* One pre-dominant other supporting
* Not necessarily mix of qual and quantmethods but often happens
* Possibility to triangulate- data, methods, research
* Sequence- one design informs another
* Interaction/iterations- repetitive processes
Observation design
- Observing processes (events, phenomena) as they unfold without interfering
- Takes long time
When to use observation design?
- Usually opened and exploratory- good for developing new theory and finding/refining constructs
- Thick, contextualized, detailed description
- Possibility to uncover very complex, causal, interacting relationships
Archival design
- Analysing existing (secondary) data- can be both text and numbers
When to use archival design?
When you are interested in ”objective” information, cannot get information from respondents
* Good for large scale research, where surveys and interviews would be too difficult/expensive
* Proxy measures exit for sensitive topics
* Interested in historical processes (development over time)
