2017 TheMLTestScoreARubricforMlProdu
- (Breck et al., 2017) ⇒ Eric Breck, Shanqing Cai, Eric Nielsen, Michael Salib, and D. Sculley. (2017). “The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction.” In: The Proceedings of the 2017 IEEE International Conference on Big Data (Big Data).
Subject Headings: Production-level Machine Learning System.
Notes
Cited By
Quotes
Abstract
Creating reliable, production-level machine learning systems brings on a host of concerns not found in small toy examples or even large offline research experiments. Testing and monitoring are key considerations for ensuring the production-readiness of an ML system, and for reducing technical debt of ML systems. But it can be difficult to formulate specific tests, given that the actual prediction behavior of any given model is difficult to specify a priori. In this paper, we present 28 specific tests and monitoring needs, drawn from experience with a wide range of production ML systems to help quantify these issues and present an easy to follow road-map to improve production readiness and pay down ML technical debt.
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2017 TheMLTestScoreARubricforMlProdu | D. Sculley Eric Breck Shanqing Cai Eric Nielsen Michael Salib | The Ml Test Score: A Rubric for Ml Production Readiness and Technical Debt Reduction |