Reproducibility in Transportation Research: A Hands-on Tutorial

Details

• When: 13:30-17:30 (MDT, UTC-6) | Tuesday, September 24, 2024
• Where: Room Salon 5 | Edmonton Convention Center | Edmonton, Canada
• 👉 Participants should bring laptops with installed required software to participate in hands-on sessions.

Overview

This tutorial is dedicated to reproducible research (RR) in transportation. As transportation researchers, it has been our experience that research in transportation is hard to reproduce. Needless to say, this holds back the scientific progress of the field; every time a student needs to re-implement another paper or collect a similar dataset, that is time that could have been spent on new research. Fortunately, tools and best practices supporting RR are maturing, so it is the perfect time for the ITS community to engage with RR. We hope that this tutorial will help to move the needle on reproducibility in transportation, so that our research collectively achieves greater impact.

This is the first of hopefully many tutorials on RR in transportation; as such, we want your feedback on your RR needs and interests! Please do not hesitate to get in touch.

Background

Reproducibility is a cornerstone of scientific research, providing the foundation for validating results and advancing knowledge. In research fields where computation-based scientific publication is pervasive, a credibility crisis has been warned. RR is gaining extensive attention in various fields, such as remote sensing, medicine, and data science, to ensure the validity and reliability of scientific findings.

In the field of traffic and transportation, Intelligent Transportation Systems (ITS) represent the most computationally intensive, fast-growing area of research with significant implications of outcomes for the general population. In ITS, the rapid evolution of technologies and methodologies has emphasized the need for RR practices. Reproducibility can refer to computational reproducibility (the focus of this tutorial), ensuring that the same data and analysis steps produce consistent results. However, when it is infeasible to replicate an entire scientific study, achieving reproducibility sets a minimum standard of scientific rigor by allowing others to validate and build upon the findings. With ITS being inherently interdisciplinary and data-driven, reproducibility forms the backbone for credible, reliable research outputs.

The objectives of the tutorial

• An introduction to the fundamental concepts and importance of RR to the domain of ITS
• Provide hands-on training in the use of tools and software that facilitate reproducibility
• Guide researchers on how to properly document and organize data and project outcomes to foster open science
• Encourage collaboration among tutorial participants and the broader ITS community to foster a community that values and practices reproducible research

Topics we will cover

• Foundations of reproducible research (RR): What does it mean for research to be reproducible? Why is it important? Why is it worthwhile to engage in RR? What makes reproducibility hard? Even when provided the exact dataset and code of another research paper?!
• RR challenges in ITS: What are challenges of RR in the field of ITS? Are they general across many fields or specific to this one? What are examples of (non-)reproducibility in ITS research?
• The state of RR in ITS: Through a live participant survey, we will get a taste of the RR attitudes and needs of the ITS community.
• Documenting code and data for RR: What kinds of missing metadata can be responsible for reproducibility failures? How can version control tools like git and github be used for RR? How can project files be organized for readability?
• Hands-on activities: Through two hands-on activities, participants will attempt to create small reproducible projects. Another participant will then try to reproduce it. Will they succeed? The first activity will focus on a simple report. The second activity will use a small project with code and data.

Participant requirements

Participants should bring laptops with installed required software to participate in hands-on sessions.

Tentative Schedule

Citing the tutorial

If you use this tutorial in your work, you are encouraged to cite us:

C. Wu, B. Ghosh, Z. Zheng, and I. Martínez, “Reproducibility in transportation research: a hands-on tutorial.” IEEE International Conference on Intelligent Transportation Systems (ITSC), 2024. [Online]. Available: https://www.rerite.org/itsc24-rr-tutorial/

@misc{wu2024reproducibility,
  title = {Reproducibility in Transportation Research: A Hands-on Tutorial},
  author = {Wu, Cathy and Ghosh, Bidisha and Zheng, Zuduo and Mart{\'i}nez, Irene},
  year = {2024},
  publisher = {IEEE International Conference on Intelligent Transportation Systems (ITSC)},
  url = {https://www.rerite.org/itsc24-rr-tutorial/}
}

Acknowledgements

We are grateful to the REROUTE project funded by Horizon Europe Marie Skłodowska-Curie Actions (MSCA) for financial support. We thank our early testers of the material: Cathy Wu’s research group, Xiaoyi Wu, Junyi Ji, Kate Sanborn, and Bidisha Ghosh’s research group. Thank you to Vindula Jayawardana and Jung-Hoon Cho for feedback and contributing materials, to Nicholas Saunier for brainstorming, the RERITE working group for moral support, the ITSS Educational Activities Committee for dissemination, and, last but not least, to ITSC 2024 for giving us a platform and a deadline. :)