TPC-C Benchmark

References

http://www.tpc.org/tpch/default.asp
- TPC-C is an on-line transaction processing benchmark.
  The current version of the TPC-C benchmark is Version 5.11. The TPC-C has continued to evolve its benchmarks to remain as representative of current practice as possible. The TPC-C benchmark continues to be a popular yardstick for comparing OLTP performance on various hardware and software configurations. For a short period after initial release of Version 5, upgrades from previous benchmark runs were allowed but required new Full Disclosure Reports and an auditor's review of the upgrade. Those results can be discerned in the table as having availability dates prior to the submitted dates.
  Compared to the previous versions, pricing changes included reducing maintenance support pricing to 3 years down from 5 years, 24x7 maintenance up from 8x5, removing terminal network pricing (hubs, switches), and allowing pricing quotes from web pages and print materials. Runtime changes included reducing the disk space requirements to 60 days from 180 days, increasing the measurement interval to 2 hours up from 20 minutes, reporting checkpoint durations, and reporting the number of lost connections of users during the measurement interval. Because these revisions changed the price/performance metric and reporting requirements, Version 5 results may not be compared to results from earlier versions.

(Nambiar & Poess, 2010) ⇒ Raghunath Nambiar, and Meikel Poess. (2010). “Transaction Performance Vs. Moore's Law: A Trend Analysis.” In: Proceedings of the Second TPC technology conference on Performance evaluation, measurement and characterization of complex systems. doi:10.1007/978-3-642-18206-8_9
- QUOTE: For over two decades the Transaction Processing Performance Council (TPC) has been very successful in disseminating objective and verifiable performance data to the industry. During this period the TPC's flagship benchmark, TPC-C, which simulates Online Transaction Processing (OLTP) Systems has produced over 750 benchmark publications across a wide range of hardware and software platforms representing the evolution of transaction processing systems. TPC-C results have been published by over two dozen unique vendors and over a dozen database platforms, some of them exist, others went under or were acquired. But TPC-C survived. Using this large benchmark result set, we discuss a comparison of TPC-C performance and price-performance to Moore's Law.