Reproducibility Measure

From GM-RKB
Jump to navigation Jump to search

A Reproducibility Measure is a similarity measure in agreement between results of measurements of the same measurand carried out by the same operating conditions over a time period, or by different observers or under changed conditions of measurement.



References

2019a

  • (Wikipedia, 2019) ⇒ https://en.wikipedia.org/wiki/Reproducibility Retrieved:2019-10-19.
    • Reproducibility is the closeness of the agreement between the results of measurements of the same measurand carried out with same methodology described in the corresponding scientific evidence (e.g. a publication in a peer-reviewed journal). Reproducibility can also be applied under changed conditions of measurement for the same measurand - to check that the results are not an artefact of the measurement procedures [1] [2]. A related concept is replication, which is the ability to independently achieve non-identical conclusions that are at least similar, when differences in sampling, research procedures and data analysis methods may exist. Reproducibility and replicability together are among the main tools of "the scientific method" — with the concrete expressions of the ideal of such a method varying considerably across research disciplines and fields of study. The study of reproducibility is an important topic in metascience.

2019b

2018

Goodman Claerbout ACM
Repeatability
Methods Reproducibility Reproducibility Replicability
Results Reproducibility Replicability Reproducibility
Inferential Reproducibility

Table 1: Table 1. Comparison of terminologies. See text for details.

2016a

The concepts of repeatability and reproducibility are taken directly from the VIM. Repeatability is something we expect of any well-controlled experiment. Results that are not repeatable are rarely suitable for publication. The proposed intermediate concept of replicability stems from the unique properties of computational experiments, i.e., that the measurement procedure/system, being virtual, is more easily portable, enabling inspection and exercise by others. While reproducibility is the ultimate goal, this initiative seeks to take an intermediate step, that is, to promote practices that lead to better replicability. We fully acknowledge that simple replication of results using author-supplied artifacts is a weak form of reproducibility. Nevertheless, it is an important first step, and the auditing processes that go well beyond traditional refereeing will begin to raise the bar for experimental research in computing.

2016b

2015

2008

1994