Empirical Studies of End-User Programmers Flashcards
Identify the difference between theory building and theory testing experimental design goals Identify subjective measures and objective measures Identify threats to validity when given a study design Identify which kind of study design that would be appropriate to answer a specific experimental question Identify when study subjects may be at risk
Observational Case Studies: classify
Theory-building
Controlled Experiments: classify
Usually theory-testing, but either
Rational Reconstruction: classify
Usually theory-testing, additional analysis allowing for exploration
Historical Data Collection: classify
Theory-testing
Construct validity
Concepts being studied are being measured correctly
Internal Validity
Establish a causal relationship and distinguish spurious relationships
External Validity
Can results be generalized? Tasks and subjects representative?
Empirical Reliability
Can results be reproduced?
Phases of convincing yourself you’ve got a good language
- Prior Knowledge
- Observe
- Theorize
- Design
- Experiment
- repeat from 2
Informed consent
how the data from a study will be used, and what the risks are to the participant
Participants must be given… (2 things)
- INFORMED CONSENT 2. freedom to refuse to participate
Observational Case Studies: describe
- ask someone to perform task & watch what they do
- collect quantitative and qualitative data
- usually exploratory
Controlled Experiments: describe
- comparing 2 languages for given task - separate user groups
- quantitative (e.g. time, lines of code)
- qualitative (user confidence, elegance)
Rational Reconstruction: describe
- comparing 2 languages for given task - you do it yourself
- quantitative (e.g. time, lines of code)
- qualitative (user confidence, elegance)
Historical Data Collection: desribe
- e.g. logs from commits, logs from typing what is added to code
