hci_machenzie_chapter_5 Flashcards
Experimental research
https://www.questionpro.com/blog/experimental-research/
Independent variable
An independent variable is a circumstance or characteristic that is manipulated or systematically controlled to a change in a human response while the user is interacting with a computer. An independent variable is also called a factor.
Factorial experiment
Experiments designed with independent variables are often called factorial experiments.
Dependent variable
A dependent variable is a measured human behavior. In HCI the most common dependent variables relate to speed and accuracy, with speed often reported in its reciprocal form, time—task completion time. Accuracy is often reported as the percentage of trials or other actions performed correctly or incorrectly. In the latter case, accuracy is called errors or error rate. The dependent in dependent variable refers to the variable being dependent on the human.
Control variable
There are many circumstances or factors that (a) might influence a dependent variable but (b) are not under investigation. These need to be accommodated in some manner. One way is to control them to treat them as control variables. Examples include room lighting, room temperature, background noise, display size, mouse shape, mouse cursor speed, keyboard angle, chair height, and so on.
Random variable
Instead of controlling all circumstances or factors, some might be allowed to vary
randomly. Such circumstances are random variables. There is a cost since more
variability is introduced in the measures, but there is a benefit since results are more
generalizable.
Confounding variable
Any circumstance or condition that changes systematically with an independent
variable is a confounding variable. Unlike control or random variables, confounding variables are usually problematic in experimental research. Is the effect observed due to the independent variable or to the confounding variable? Researchers must attune to the possible presence of a confounding variable and eliminate it, adjust for it, or consider it in some way. Otherwise, the effects observed may be incorrectly interpreted. As an example, consider an experiment seeking to determine if there is an effect of camera distance on human performance using an eye tracker for computer control. In the experiment, camera distance—the independent variable—has two levels, near and far. For the near condition, a small camera (A) is mounted on a bracket attached to the user’s eye glasses. For the far condition, an expensive eye tracking system is used with the camera (B) positioned above the system’s display. Here, camera is a confounding variable since it varies systematically across the levels of the independent variable: camera A for the near condition and camera B for the far condition.
Task
A good task is representative of the activities people do with the interface. A task that is similar to actual or expected usage will improve the external validity of the research–the ability to generalize results to other people and other situations. A good task is also one that can discriminate the test conditions. Obviously, there is something in the interaction that differentiates the test conditions, otherwise there is no research to conduct. A good task must attune to the points of differentiation in order to elicit behavioral responses that expose benefits or problems among the test conditions. This should surface as a difference in the measured responses across the test conditions. A difference might occur if the interfaces or interaction techniques are sufficiently distinct in the way the task is performed.
Usability evaluation or usability test
In HCI, we often hear of researchers doing usability evaluation or usability testing. These exercises often seek to assess a prototype system with users to determine problems with the interface. Such evaluations are typically not organized as factorial experiments. So the question of how many participants is not relevant in
a statistical sense. In usability evaluations, it is known that a small number of participants is sufficient to expose a high percentage of the problems in an interface. There is evidence that about five participants (often usability experts) are sufficient to expose about 80 percent of the usability problems (Lewis, 1994; Nielsen, 1994).
Questionnaire
Ordinal data are inherently lower quality than ratio-scale data, since it is not possible to compute the mean or standard deviation.
Within-subjects and between-subjects
The administering of test conditions (levels of a factor) is either within-subjects or between-subjects. If each participant is tested on each level, the assignment is within-subjects. Within-subjects is also called repeated measures, because the measurements on each test condition are repeated for each participant. If each participant is tested on only one level, the assignment is between-subjects. For a between-subjects design, a separate group of participants is used for each test condition.
Order
Practice, then, is a confounding variable, because the amount of practice increases systematically from one condition to the next. This is referred to as a practice effect or a learning effect. Although less common in HCI experiments, it is also possible that performance will worsen on conditions that follow other conditions. This may follow from mental or physical fatigue—a fatigue effect. In a general sense, then, the phenomenon is an order effect or sequence effect and may surface either as improved performance or degraded performance, depending on the nature of the task, the inherent properties of the test conditions, and the order of testing conditions in a within-subjects design.
In the simplest case of a factor with two levels, say, A and B, participants are divided into two groups. If there are 12 participants overall, then Group 1 has 6 participants and Group 2 has 6 participants. Group 1 is tested first on condition A, then on condition B. Group 2 is given the test conditions in the reverse order. This is the simplest case of a Latin square . In general, a Latin square is an n Å~ n table filled with n different symbols (e.g., A, B, C, and so on) positioned such that each symbol occurs exactly once in each row and each column.9 Some examples of Latin square tables are shown in Figure 5.7 . Look carefully and the pattern is easily seen.
A deficiency in Latin squares of order 3 and higher is that conditions precede and follow other conditions an unequal number of times. In the 4 Å~ 4 Latin square, for example, B follows A three times, but A follows B only once. Thus an A-B sequence effect, if present, is not fully compensated for. The compensatory ordering of test conditions to offset practice effects is called counterbalancing.
another way to offset learning effects is to randomize the order of conditions. This is most appropriate where (a) the task is very brief, (b) there are many repetitions of the task, and (c) there are many test conditions.
Group effect
If the learning effect is the same from condition to condition in a within-subjects design, then the group means on a dependent variable should be approximately equal. A group effect is typically due to asymmetric skill transfer differences in the amount of improvement, depending on the order of testing.
The simplest way to avoid asymmetric skill transfer is to use a between-subjects design.
Longitudinal studies
The preceding discussion focused on the confounding influence of learning in experiments where an independent variable is assigned within-subjects. Learning effects more generally, order effects—are problematic and must be accommodated in some way, such as counterbalancing. However, sometimes the research has a particular interest in learning, or the acquisition of skill. In this case, the experimental procedure involves testing users over a prolonged period while their improvement in performance is measured. Instead of eliminating learning, the research seeks to observe it and measure it. An experimental evaluation where participants practice over a prolonged period is called a longitudinal study.
In a longitudinal study, “amount of practice” is an independent variable. Participants perform the task over multiple units of testing while their improvement with practice is observed and measured.
Block
A portion of an experiment distinguished by breaks for subjects, shifts in procedure, and so on. Typically, a block is a set of trials. Often, blocks serve as units of analysis.
