Uncertainty Visualization Study Group Notes – 09/17/2012

Discussion

---

Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty Boukhelifa et al., 2012 link

• Undefined uncertainty in the paper
  • Is it qualitative? quantifiable?
  • 2 categories of uncertainty: epistemic / epistemological (can be decreased with knowledge) vs aleatoric (exists on principle / does not decrease with knowledge)
  • Does uncertainty need to be defined before it can be depicted?
  • This paper does not make an attempt to fully operationalize uncertainty.
• User-study end of the paper: issues/comments on the conclusions of the tests
  • Did they overreach? Certainly exhibited strong inclination to approve sketchiness as a potential representation of uncertainty
  • Methodological issues with some of the experiments; however, multiple attempts at examination of sketchiness’ effectiveness as a representation of uncertainty
• Sketchiness increasing interaction by to nature (early design, wireframes) or simply b/c it’s novel?
  • People’s reaction to a new approach certainly providing benefit
• Sketchiness “unprofessional” as opposed to “objective” and “authoritative”?
• Take away: No intuitive encoding? In a real-world scenario, you will necessarily be given some context.
  • Is asking for intuition without context fair?
  • Do we want to give people a user study communicating uncertainty explicitly?
  • Intuitive understanding may be more useful when accompanied by an explicit legend
  • Expressive rather than intuitive encoding channel for uncertainty?
• Different visualization methods for different kinds of uncertainty
  • finer partitioning than epistemological/aleatoric?
  • ordinal vs. categorical, geospatial vs. numeric, etc.
• Uncertainty as a separate variable: misleading, because it is often dependent on another variable
  • Use primary attribute for data, secondary attribute for uncertainty
• Testable: extrinsic vs. intrinsic representations of uncertainty, and how effectively they communicate data
  • Extrinsic: assumes that people choose hierarchically; intrinsic assumes a more simultaneous approach
• Geospatial representations are limiting: avenues of representation are already used
• 2D single representations may work better for some kinds of uncertainty data: data where primary concern is borders, e.g. weather pattern uncertainty
• Mechanical Turk: is it usable? Do they use it properly?
  • There are some issues with data quality (reliability of self-reported demographics)
  • More traditional methods safer in terms of reception, at least on the cognitive side
    • less unknowns, more controlled
    • rooted heavily in tradition where students required to participate in research
  • Mechanical Turk is useful for gathering large numbers of subjects, in very short tasks
  • Online studies using the participant pool vs. in-person studies with the participant pool have produced similar results.
  • We can make our own online studies as well
  • Should we look at the literature evaluating Mechanical Turk?
• Uncertainty quantification has been a topic of investigation in the mathematical community for 15 years; less investigated and recognized by the visualization community

Administrative Info

---

Next Week: Paper on 1D and 2D representations of uncertainty + short paper on extrinsic vs. intrinsic representation (if we have time)

A separate weekly meeting (immediately after this one?) will happen each week to generate a methodology for an experiment.