Methods Flashcards
Clinical studies: Def & Types
A detailed story, often involving a combination of qualitative data, anecdotal evidence, and sometimes quantitative data, about a specific situation or phenomenon Types: - Teaching case - In-depth clinical papers - Empirical large-sample papers
Clinical Studies: Purpose
- To tell theorist what to model
- To guide on how to model these things
- To asses possibility theorems
- To provide evidence of a certain type for testing theory
Areas for Conern
Representativeness, Bias, Replicability
What is insider econometrics?
Insider econometrics is an empirical research strategy for investigating these questions about the effects of management practices on productivity and the determinants of the choice of management practices
What is insider econometrics? (Questions)
1.Why do firms in the same industry adopt different management practices? 2.Does the adoption of a new management practice raise productivity? 3.If so, why does the new management practice raise productivity?
Insider econometrics (Broken Down Defintion)
Insider refers to the use of micro-level data on workers or workgroups inside firms that share a common production function. Also refers to the use of insights from insiders (from managers or employees ) that inform what’s happening inside the firm to contribute to the research
Econometrics refers to the use of statistical tests. (Average treatments effects, treatment of the treated effect, treatment of the non-treated effect, etc)
How to do an Insider Econometrics Research? (1/2)
1) Estimate a productivity regression in which
productivity is a function of some management practice.
2) Identify why management practices raise
productivity and where the practice has
larger and smaller effects on productivity.
3) Model the adoption of the management practices.
How to do an Insider Econometrics Research? (2/2)
4) Analyze micro-level data on production units
5) Formulate testable hypotheses and interpret results.
Treatment Effects and the Econometrics of Insider
Studies : Two common options
1) First Differences
Estimates the treatment of the treated using data on only the treated group, before and after the treatment
Treatment Effects and the Econometrics of Insider
Studies : Two common options
2) Difference-in-Differences Estimator:
Estimates the average gain in productivity if the treatment were applied randomly across all i, using data on both the treated group, before and after the treatment, and on the non-treated group
Analytical Challenges of Insider Econometric Research
Key Research Design Decisions for Insider Econometric Studies (1-3)
1) Identify a Treatment and Measure It Accurately
2) Test a Generalizable Principle by Modeling Economic Behavior
3) Balance Homogeneity and Heterogeneity in the Sample
Key Research Design Decisions for Insider Econometric Studies (4-5)
4) Collect Information on Why the Treatment Was Adopted
5) Collect information on how behavior changes and why the treatment was successful
What do Experimental Economics do?
Ask human subjects respond to incentives in a controlled laboratory environment which represents the essential economic features of a theoretical or naturally
When Experimental Economics can be useful?
- To test theory (Under precisely controlled and/or measured conditions that are typically unavailable in field data.)
- To evaluate assumptions
Other: To identify stylized facts / To compare institutional designs / To explain newly observed regularities
Experimental Economics: A Time-Line
1) Formulate a research question
2) Address the research question
3) Prepare an experimental outline
4) Seek funding
5) Ethical approval
Details: Prior the experiment
- Write instructions for each treatment and an overall “script”
- Prepare a questionnaire (useful for controls). If computerized, make sure the software is capable
- Organize money(when/how to pay)
- Recruit for pilot
- Run pilot experiment
Details: During the experiment
- Improve the design/instructions based on the pilot experience
- Recruit subjects
- Run the experiment
- Analyze data and write the paper
Experimental Economics: Other practical matters
- Make sure you limit the things that can react with your treatment
- Eliminate distractions and enable privacy
- Deception is not allowed
- Preserve anonymity
Experimental Economics: Preserving Anonymity
Single blind: no one other than the researchers will be able to ever identify subjects’ actions or subjects’ payoffs.
Double blind: no one can link subjects’ decisions to their identity even the researchers
Abandoning anonymity: if abandoning anonymity is important for the experiment, then subjects should know and have the option to withdraw if they want
Field vs Laboratory
Field:
- More external validity built-in
- The subject pool is spot-on
- Test if a change have a sizeable effect
- More bias
Laboratory:
- More control over the incentives
- Easier to get strict instructions regarding time and effort
- More transparency and more replicable
- Easier to overcome ethical issues
- More expensive to obtain large samples
What makes an experiment organizational?
In an organizational experiment, two or more subjects engage in a productive enterprise where the impact of the structure of the organization and incentives on productivity are the variables of central interest.
What makes an organizational experiment good?
1) Control over all relevant factors in the experimental environment
2) Internal validity
3) Statistical integrity
4) Generalizability
Four areas of interest to organizational studies
1) Incentives and Worker Effort
2) Coordination
3) Voluntary Effort and Reciprocity
4) Leadership and Hierarchy
Organizational experiments: Voluntary effort and reciprocity in organizations
- Results are heterogeneous. Do not directly refute or confirm the theory.
- In some cases, are highly supportive and encouraging, while in others not.
- For example, leadership over larges firms does not guarantee achieving an efficient equilibrium.
What is a RCT ?
An experiment designed to isolate the influence that a certain intervention or variable has on an outcome or event
Questions RCT
- How would individuals who participated in a program behave in the absence of the program?
- How would those who were not exposed to the program have behave in the presence of the program?
- > No answer to these questions
Impact of the program or pre-existing differences
Treatment individuals may have had different outcomes on average even if they had not been treated
Sample size, design, and the power of experiments
Power of the design: probability of reject the null hypothesis when in fact is false
Should one randomize over individuals or some larger group?
The choice of the level at which to randomize is very context specific, depends on the nature of the intervention as well as the nature of the interactions between the individuals to be treated.
- Some factor could be taken into account:
1) Spillovers from treatment to comparison groups can bias the estimation of treatment effects. In such case, the randomization should occur at a level that captures these effects
2) Randomization at the group level may some times be much easier from the implementation point of view, even if it requires larger sample sizes
What is the value of a baseline survey?
1) Generate control variables that will reduce the variability in final outcomes and therefore reduces sample size requirements, reducing costs
2) Make it possible to examine interactions between initial conditions and the impact of the program, to verify that the randomization was conducted apppropriately and to test and refine data collection procedures.
Field Experiments with Firms: Motivation
- Firms operate in complex environments
- Some characteristics are hard to measure or uncertain
- Implies huge challenges at uncovering casual relationships when understating the behavior of firms based on observational data
How do they do?
Do firm choices maximize profits subject to constraints?
If so, which constraints bind and inform decision-making in firms? If not, why are firms operating inside the frontier?
- Experiments within firms
- Experiments between firms
Field Experiments within Firms how can firms solve agency problems and motivate their employees?
- Monitoring
- Paying for performance
- Increasing motivation through better social relations or status rewards
Field Experiments within Firms how can firms solve agency problems and motivate their employees?
- Monitoring and Shirking
Employees decide their level of effort considering marginal costs and marginal benefits of shirking and firms choose compensation and monitoring policies.
Field Experiments within Firms how can firms solve agency problems and motivate their employees?
Monetary Incentives and the Social Organization of the Workplace.
- Productivity is 50% higher under piece rates
Field Experiments within Firms how can firms solve agency problems and motivate their employees?
Increasing motivation through better social relations or status rewards
Field Experiments across Firms
Field experiments that take the firm as the unit of observation often seek to exogenously vary the availability of key inputs and in this way seek to uncover the constraints faced by firms
Identifying constraints faced by firms allows understanding the development process
Conclusion field experiments
The design of fields experiments ought to be supported by economic theory and hypothesis must be testable
- Selection of unit over the experimentଵ (static vs dynamic)
- Methodology: the utility of difference-in-differences estimator.
- Field experiments are a powerful tool of empirical studies. They allow a better understanding of how firms works and how agents react to incentives.
