ICMR Reproducibility (ICMR Repro) Overview and Guidelines here. The page gives the instructions for submitting reproducibility companion papers for the ACM ICMR 2025.

Dates

The important dates along the timeline for submitting the companion paper and the associated artifacts are:

March 30, 2025: submission due date for the paper and reproducibility archive
April 7, 2025: notification of acceptance/rejection
April 24, 2025: camera ready due date for final version of the paper and artifacts (same deadline as for regular papers)

Program Committee

Chairs:

Bo Wu, MIT-IBM Watson AI Lab, USA
Qiushi Huang, University of Surrey, United Kingdom

Submission

A submission has essentially two parts:

Companion Paper (ICMR latest format of PDF file): The companion paper is 2–3.5 pages in length (with an optional page for references). It contains a overview description of the contributions and an introduction of the experiments carried out in the original paper and that are implemented in the submitted archive. It must follow the standard ACM style format, double column. It has to involve a majority of the authors of the original paper and provide in the clear their names and affiliations.
Archive (github repository and a compressed zip file): Contains all the artifacts and information (code, scripts, datasets, and protocols, examples, along with detailed README with instructions) for reproducing the results. The final submission should be finalized, packaged, ready to download, and reproduce the results in a one-click execution.

Submission System

Registration: Please register at ICMR 2025 CMT, select Reproducibility on the left top option bar to enter the ICMR Repro 2025 Submission Webpage.

Submission: Please select Reproducibility Track for submission.
Preparation: For best practices in preparing your submission, please refer to:Best Practices

Submission Guidelines

Authors that have a regular long/short paper published at top academic conferences during 2024-2025 are invited to submit a short reproducibility companion paper to ICMR Reproducibility of ACM ICMR 2025.

That companion paper typically focuses on the technical details of what you published at previous publications. The reproducibility companion paper and the associated artifacts will undergo a reproducibility review, which will result in an accept or a reject decision. Rejected papers are not disclosed.

If accepted, the corresponding ACM Paper Badges will be added to the original paper and to the companion paper, which are both stored in the ACM Digital Library, together with the artifacts. A badged companion paper will appear in the ACM ICMR 2025 proceedings. The reviewers of the badged companion paper add a section documenting their efforts and become co-authors of the paper. The final version may be up to 4 pages (with an optional page for references). If, during the evaluation, a serious flaw invalidating the scientific results published in the original contribution is discovered, then the companion paper is rejected, and the authors are encouraged to publish an errata.

Note: If your original paper turns out to be especially challenging to reproduce, the committee may also recommend that the companion paper be published in the next year of ACM ICMR, instead of ACM ICMR 2025. If replication cannot be completed in time for the next year of ACM ICMR too, then the paper cannot be accepted.

Reproducibility Certification

Paper Badge

We are committed to setting the standard high for reproducibility at ACM ICMR, and the Reproducibility Committee asks authors to target a top-quality badge for the companion papers they submit. Here is an example of the Results Reproduced badge:

We recommend that three separate badges related to artifact review be associated with research articles in ACM publications: Artifacts Evaluated, Artifacts Available and Results Validated. These badges are considered independent and any one, two or all three can be applied to any given paper depending on review procedures developed by the journal or conference.
» ACM DL. Read it here.

Artifacts

We suggest that you take a look at the ACM Digital Library’s webpage entitled Software and Data Artifacts in the ACM Digital Library, which provides information on motivations for reproducibility and a look into how reproducibility is supported and continues to evolve in the ACM Digital library.

Badged Paper Examples

Here are a few links to papers with badges inside the ACM DL. They have been picked because they nicely illustrate what is described in this page, not due to their scientific value. They come from multiple domains, giving you a broad view of what colleagues have done.

The first two papers from ACM MM 2019 comply with the guidelines specified on this page. The latter two do not necessarily comply with our rules. For example, sometimes there is no companion paper, sometimes the companion paper is not always a 2-4 page paper, ACM style. Furthermore, papers may target other types of badges that are about the availability of the artifacts, not solely about replicability or reproducibility of results.

Reproducibility Companion Paper: Knowledge Enhanced Neural Fashion Trend Forecasting. Yunshan Ma, Yujuan Ding, Xun Yang, Lizi Liao, Wai Keung Wong, Tat-Seng Chua, Jinyoung Moon, Hong-Han Shuai. ACM ICMR 2021. Paper’s DOI. The artifacts are on Zenodo.
Reproducibility Companion Paper: Human Object Interaction Detection via Multi-level Conditioned Network, Yunqing He, Xu Sun, Hui Jiang, Tongwei Ren, Gangshan Wu, Maria Sinziana Astefanoaei, Andreas Leibetseder. ACM ICMR 2022. Paper’s DOI. The artifacts are on GitHub.
Reproducible Experiments on Adaptive Discriminative Region Discovery for Scene Recognition, Z. Zhao, Z. Liu, M. Larson, A. Iscen, N. Nitta. ACM MM 2019. . Paper’s DOI. The artifacts are on GitHub.
Generating Preview Tables for Entity Graphs, N. Yan, S. Hasani, A. Asudeh and C. Li. Winner of the 2017 most reproducible papers from SIGMOD 2016. . Paper’s DOI.

References

Researchers in the area of databases started explicitly emphasizing reproducibility early on. Much of what we propose to implement for ACM Multimedia Retrieval is inspired from their DOs and DON’Ts.

A good source of information can be found in the ICDE 2008 tutorial by Ioana Manolescu and Stefan Manegold. The tutorial includes a road map of tips and tricks on how to organize and present code that performs experiments so that an outsider can repeat them. A discussion about reproducibility in research including guidelines and a review of existing tools can be found in the SIGMOD 2012 tutorial by Juliana Freire, Philippe Bonnet, and Dennis Shasha.

Closer to multimedia, ACM MMSys and ACM MM have led the multimedia research community in explicitly emphasizing reproducibility and Gwendal Simon chaired the reproducibility track at MMSys 2019. Emailing Gwendal or reading what he wrote about reproducibility in his blog can be helpful. Take also a look at ACM MM’s reproducibility efforts that are documented here.