2013 AGeneralBootstrapPerformanceDia

From GM-RKB
Jump to navigation Jump to search

Subject Headings:

Notes

Cited By

Quotes

Author Keywords

Abstract

As datasets become larger, more complex, and more available to diverse groups of analysts, it would be quite useful to be able to automatically and generically assess the quality of estimates, much as we are able to automatically train and evaluate predictive models such as classifiers. However, despite the fundamental importance of estimator quality assessment in data analysis, this task has eluded highly automatic solutions. While the bootstrap provides perhaps the most promising step in this direction, its level of automation is limited by the difficulty of evaluating its finite sample performance and even its asymptotic consistency. Thus, we present here a general diagnostic procedure which directly and automatically evaluates the accuracy of the bootstrap's outputs, determining whether or not the bootstrap is performing satisfactorily when applied to a given dataset and estimator. We show that our proposed diagnostic is effective via an extensive empirical evaluation on a variety of estimators and simulated and real datasets, including a real-world query workload from Conviva, Inc. involving 1.7 TB of data (i.e., approximately 0.5 billion data points).

References

;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2013 AGeneralBootstrapPerformanceDiaAriel Kleiner
Ameet Talwalkar
Sameer Agarwal
Ion Stoica
Michael I. Jordan
A General Bootstrap Performance Diagnostic10.1145/2487575.24876502013