We had a very inspiring and productive meetup about directions for further rewarding reproducibility in visualization. The audience was very diverse and discussed from very different viewpoints the possible directions in this regard. A collection of the findings and opinions of the meetup was collected in a common document. The organizers and a group of committed people will pursue the found directions and try to establish concrete methods for rewards in the future.
Today we will have a meetup at IEEE Vis to develop a common sense of good reproducibility in our community with the title “MeetRV3: Towards a System for Rewarding Reproducibility in Visualization”. The meetup will take place in SALON 2 on the third floor at 12:30 pm. The goal of the meetup is to review and discuss efforts of other communities to promote reproducibility to come up with a common direction for the visualization community. We hope to find a consentient way to honor well-reproducible work and strengthen the community in its effort to establish a common culture of reproducibility.
There will be a meetup at this year’s IEEE Vis in Chicago, following the developments at EuroRV³ regarding reproducibility in visualization. The meetup will be held on Tuesday, October 27, at 12:30pm in SALON 2 and will facilitate discussions towards a system for rewarding reproducibility in visualization. We are looking forward to your input. All updates will be shared at eurorvvv.org.
The co-chairs of EuroRV³ 2015 thank all contributors and visitors of the workshop for making the event a great success with inspiring talks and long, intense, and productive discussions. This effort was a major step into establishing a common culture of reproducibility in the visualization community. All papers can be downloaded from now on in the Eurographics Digital Library.
We are happy to announce that the program of this year’s EuroRV³ has been finalized. We have put together two exciting half days for you, including 9 paper presentations, one keynote with the title “Multidimensional Projection: from Validation to Layout Enrichment” from Luis Gustavo Nonato, one capstone with the title “The Need for Reproducibility of Computer Science in Ophthalmology” from Rico Großjohann, and a concluding panel to discuss the current state and future directions of reproducibility, verification, and validation in visualization.
We are happy to inform you that this year’s EuroRV³ will feature nine contributions focusing on the special topic “Reproducibility in Practice”. The contributions are in detail:
- Nadia Boukhelifa, Anastasia Bezerianos, Evelyne Lutton: A Mixed Approach for the Evaluation of a Guided Exploratory Visualization System
- Juliana Freire: Reproducibility Made Easy
- Jane Hoffswell, Arvind Satyanarayan, Jeffrey Heer: Debugging Vega through Inspection of the Data Flow Graph
- Simone Kriglstein, Margit Pohl: Choosing the Right Sample? Experiences of Selecting Participants for Visualization Evaluation
- Kai Lawonn, Michael Krone, Thomas Ertl, Bernhard Preim: On the Reproducibility of Line Integral Convolution for Real-Time Illustration of Molecular Surface Shape and Salient Regions
- Nicholas Hugo Müller, Linda Pfeiffer, Peter Ohler, Paul Rosenthal: On the Reproducibility of VisRuption: A Tool for Intuitive and Efﬁcient Visualization of Airline Disruption Data
- Katrin Scharnowski, Michael Krone, Guido Reina, Thomas Ertl: On the Reproducibility of our Biomolecular Visualization
- Michael Smuc, Günther Schreder, Eva Mayr, Florian Windhager: Should we Dream the Impossible Dream of Reproducibility in Visual Analytics Evaluation?
- Viktor Vad, Balázs Csébfalvi, Peter Rautek, Eduard Gröller: Reproducibility, Veriﬁcation, and Validation of Experiments on the Marschner-Lobb Test Signal
In addition, we would like to remind you that the registration for EuroRV³, EuroVis and all other co-located events is open and early-bird deadline is Thursday, April 30th, 2015. You can find the registration details here.
The EuroRV3: EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization focuses on the need for reliable visualization. We concentrate on recent and future developments related to the reproducibility of newly-developed visualization approaches, the verification of visualization and data-processing systems, the validation of visualization paradigms, and the effective incorporation of each of these into modern visualizations.
The outcome of the two last EuroRV3 workshops was that the visualization community is clearly aware of the need for reproducibility and a vast majority agrees that the topic should be developed much more. However, it can be observed that, in current publications, providing reproducibility is still often limited to the fraction that is needed for the paper to be accepted to a venue. In addition, the variety of possible characteristics and levels of reproducibility, together with the very different paper types in our community, makes it very hard to come up with a strategy to further encourage good reproducibility. In this workshop with the topic “Reproducibility in Practice” we want to develop a good understanding of why there are still papers providing quite limited reproducibility, how these papers differ from papers that achieve much higher reproducibility and how conditions could be changed to overcome complications and the various different pitfalls in this regard. In addition, we want to showcase how good reproducibility can be achieved and what efforts this takes.
This year’s workshop invites contributions of authors from recent visualization publications about how the topic reproducibility was tackled by the authors in the context of the specific project or paper and why. Contributors should indicate their interest of submission via mail to Paul Rosenthal. All accepted contributors submit extended abstracts (up to 2 pages without references, 3 pages including references) following the submission guidelines, which will undergo a one-stage review cycle. All accepted final papers will be published in the Eurographics Digital Library.
- Indication of Contribution: February 20, 2015
- Submission of Extended Abstract: March 20, 2015
- Submission of Camera Ready Version: April 03, 2015
We are very happy to announce that EuroRV³, the international EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization continues in 2015. The third edition will be co-located with EuroVis 2015 in Cagliari, Italy and take place on May 25-26, 2015. This year’s edition with the topic “Reproducibility in Practice” will concentrate on developing a good understanding of the value of reproducibility “in the wild” and why reproducibility is still often neglected in visualization research. We will discuss ways to achieve higher reproducibility and how conditions could be improved to overcome complications and pitfalls in this regard. In addition, it should be showcased how good reproducibility can be achieved and what efforts this takes. We will soon publish a call for contributions, so stay tuned and check our website EuroRVVV.org.
Finally, EuroRV³ starts today at 2pm with a Keynote by Torsten Möller. We hope you all safely arrive in Leipzig. From Hauptbahnhof take the Tram line 3 until the stop “Torgauer/Permoserstraße” or a taxi directly to Leipzig Kubus. We have packed an interesting program for the upcoming two days and are hoping for many fruitful discussions.