EuroVis 2012 was not only accompanied by EuroVA but also by a new workshop, the EuroRVVV: EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization. The workshop was co-organized by Britta Weber (Zuse Institute Berlin, Germany) and Paul Rosenthal (Chemnitz University of Technology). It featured a series of invited talks and a panel discussion.
Tuesday, June 5:
09:00 – 09:10 Opening
09:10 – 10:20 Talks
- Managing Uncertainty in Visualization (Slides, Video)
- Visualizing data reduces the precision of the displayed data because of the limited capabilities of both the output devices used and our visual perception. Understanding how and where uncertainty occurs in this process, and how to control it, are key issues to make visualization reliable and reproducible. I will discuss recent work on how to understand and control uncertainty, and present what I believe to be the key next steps towards managed uncertainty in visualization.
- Visual Correctness or “How do we measure the quality of a visualization?” (Slides, Video)
- In this talk, we will attend to the question of how to quantify and evaluate the quality of our work, i.e., visualizations of scientific data. Is it enough to provide a valid implementation or do we have to take further aspects such as human vision and cognition, the users visualization literacy, or measures for visual encoding into account? Can we quantify visual correctness automatically and where do we need user studies?
- Cognitive Visual Interpretation – An Outside Look (Slides, Video)
- One of the main reasons to compute abstract data into visual representations is to discover new ways for interpretation, analysis, and conclusions. In order to achieve this, a fundamental understanding of the human cognitive system, its limits, and known errors with respect to the interpretation of visual stimuli is of particular importance. We will give an introduction in the relevant aspects of the broad field of cognition and especially discuss the challenges for designing experiments and developing new algorithms.
10:20 – 10:45 Coffee Break
10:45 – 12:15 Panel: “Reproducibility in Visualization” or “Do we want a Badge of Honor?”
- Britta Weber (Zuse Institute Berlin) (Slides, Video)
- Rosane Minghim (University of São Paulo) (Slides, Video)
- Helmut Doleisch (SimVis GmbH) (Slides, Video)
- Min Chen (Oxford University) (Video)
For readers of papers, verification and validation of results are often only possible if data and software for a published method are available. However, the barriers for authors to release data and binaries or even code are high. Testing and documenting the highly specialized prototypes typically developed for conference papers and transferring them to different platforms is seldom an easy task. Even though the long-term benefits of giving out code and data for authors and the community are obvious, they are still outweigh in the short run by the time lost for the software release process. To lower this barrier, the SIGMOD conference has introduced a “badge of honor” for authors making their binaries and data available to reviewers. Authors are awarded if results in papers can be repeated by the reviewers. Participating in the evaluation is optional, yet there is a growing number of authors submitting their work to the repeatability committee. We propose to introduce a similar optional evaluation of papers at Vis conferences. We will review the efforts and results at SIGMOD and then discuss the general requirements for establishing such an evaluation. In the panel discussion we will collect feedback from the community regarding the feasibility of this initiation and the pros and cons of different viewpoints