Chapter 4: Analysis & Validation Flashcards
What are the four nested levels of visualization design. Explain how they relate to each other.
- The domain situation level
- The what-and-why task and data abstraction level
- The visual encoding and interaction idiom level (How)
- The Algorithmic level
Explain the domain situation level, as well as the outcomes of this level.
The domain situation level considers the details of a particular application domain for vis. This encompasses a group of target users, their domain of interest, their questions and their data. At this level, situation blocks are identified: the outcome of the design process is an understanding that the designer reaches about the needs of a user, in particular, it is a detailed set of questions asked about or action carried out by the target users.
Explain the task and data abstraction level. Also explain why this is useful to do and what the goals of this level are.
The abstraction level introduces the what-why, where you map the domain specific problems and data into forms that are independent of the domain. Abstracting into the domain-independent vocabulary allows you to realize how domain situation blocks that are described using very different language might have similar reasons why the user needs the vis tool and what data it shows. Your goal is to determine which data type would support a visual representation of it that addresses the user’s problem.
Explain the how-level of design.
The following how-level, the visual encoding and interaction idiom, specifies the approach to visual encoding and interaction. You decide on the specific way to create and manipulate the visual representation of the abstract data block that you chose at the previous level, guided by the abstract tasks that you also identified at that level.
What is meant with an ‘Idiom’?
Each distinct possible approach to visualizing data is called an idiom.
What is the innermost level of the visualization design?
The algorithm level: the goal is to efficiently handle the visual encoding and interaction idioms that you chose in the previous level.
Which two angles of attack are there for visualization design?
There are two common angles of attack for vis design: top down or bottom up. With problem-driven work, you start at the top domain situation level and work your way down. The goal is to grapple with the problems of real-world users and attempt to design a solution that helps them work more efficiently. This is also called a design study.
In technique driven work, you work at one of the bottom two levels, idiom or algorithm design where your goal is to invent new idioms that better support existing abstractions, or new algorithms that better support existing idioms.
What is meant with a threat to validity of your design?
different fundamental reasons why you might have made the wrong choices.
What is the primary threat to validity at the domain situation level. Name one immediate, and one downstream validation method.
At the top level, when characterizing the domain situation, a visualization designer is asserting that particular problems of the target audience would benefit from vis tool support. The primary threat is that the problem is mischaracterized: the target users do not in fact have these problems.
An immediate form of validation is to interview and observe the target audience to verify the characterization, as opposed to relying on assumptions or conjectures. A common approach for this case is a field study by questionnaires.
One downstream form of validation is to report the rate at which the tool has been adopted by the target audience.
What is the primary threat to validity at the task/data abstraction level. Name one downstream validation method.
At the abstraction level, the threat is that the identified task abstraction blocks and designed data abstraction blocks do not solve the characterized problems of the target audience. The key aspect of validation against this threat is that the system must be tested by target users doing their own work, rather than doing an abstract task specified by the designers of the vis system.
A downstream form of validation is to have a member of the target user community try the tool, in hopes of collecting anecdotal evidence that the tool is in fact useful.
What is the primary threat to validity at the idiom level. Name one immediate, and one downstream validation method.
At the visual encoding and interaction idiom level, the threat is that the chosen idioms are not effective at communicating the desired abstraction to the person using the system.
One immediate validation approach is to carefully justify the design of the idiom with respect to known perceptual and cognitive principles.
A downstream approach to validate is to carry out a lab study: a controlled experiment in a laboratory setting. This method is appropriate for teasing out the impact of specific idiom design choices by measuring human performance on abstract tasks that were chosen by the study designers.
What is the primary threat to validity at the algorithm level. Name one immediate, and one downstream validation method.
At the algorithm level, the primary threat is that the algorithm is suboptimal in terms of time or memory performance.
An immediate form of validation is to analyze the computational complexity of the algorithm.
The downstream form of validation is to measure the wall-clock time and memory performance of the implemented algorithm. Benchmarks datasets can be used to test the algorithm in a standardized way, so it is also comparable to other algorithms.
