"Although computation plays a key and ever-larger part in science,
scientific articles rarely include their underlying code, Rougier says.
Even when they do, it can be difficult for others to execute it, and
even the original authors might encounter problems some time later.
Programming languages evolve, as do the computing environments in which
they run, and code that works flawlessly one day can fail the next.
In 2015, Rougier and Hinsen launched ReScience C, a journal that
documents researchers’ attempts to replicate computational methods
published by other authors, based only on the original articles and
their own freshly written open-source code. Reviewers then vet the code
to ensure it works. But even under those idealized circumstances — with
reproducibility-minded authors, computationally savvy reviewers and
fresh code — the process can prove difficult.
The Ten Years Reproducibility Challenge aims “to find out which of the
ten-year-old techniques for writing and publishing code are good enough
to make it work a decade later”, Hinsen says. It was timed to coincide
with the 1 January 2020 ‘sunset’ date for Python 2, a popular language
in the scientific community, after 20 years of support. (Development
continues in Python 3, launched in 2008, but the two versions are
sufficiently different that code written in one might not work in the
Via Esther Schindler.
*** Xanni ***
Chief Scientist, Xanadu
Partner, Glass Wings
Manager, Serious Cybernetics