Module 7 - Validation Flashcards
Upstream vs Downstream Validation
Downstream = during design (going from outer to inner levels)
Upstream = final visualisation implemented, validate design (going from inner to outer)
Algorithm Validation
Downstream: Analyze computational complexity
Upstream: Experimental Study
Visual Encoding Validation
Downstream: Justification of choices (Directly derives from task abstraction)
Upstream: Informal Usability Study and Lab Study
- Informal Usability Study
using qualitative (user experience, opinion) or quantitative data (measurable performance indicators
-> ICET method
- assessing value to a visualisation (insight,time,essence,confidence)
1. Enroll 5 participants
* Visualization users, not necessarily domain experts
* Must be familiar with the data used in the evaluation
* Arrangement with individual sessions
2. Set up the visualization(s) with some dataset(s)
* This method can also be used to compare
different visualizations
* “Teach” each participant the visualization
3. Participants rate each visualisation based on heuristics
？Few users
？Responses might be biased towards a positive outcome
？Subjective interpretation of the heuristics
- Lab Study
Quantitative: Performance/Accuracy of users - Laboratory experiment in controlled conditions
- Allow strong conclusions
- Requires careful formulation of hypotheses
Qualitative: User experience and/or opinions
Data/Task Abstraction Validation
Downstream = Wrong data/task abstraction
Upstream = Field Study and Case Study
- Field Study
deploy tool into real world setting and ask experts to integrate in their workflow, ask if they can solve tasks using our tool and if workflow changed in any way - Case Study
finding out whether can make meaningful insights with our visualisation, test on target users and collect evidence
Domain Validation
Downstream = Observe target users
Upstream = Measure adoption
