Benchmarking the Dependability of Different OLTP Systems



On-Line Transaction Processing (OLTP) systems constitute the kernel of the information systems used today to support the daily operations of most organizations. Although these systems comprise the best examples of complex business-critical systems, no practical way has been proposed so far to characterize the impact of faults in such systems or to compare alternative solutions concerning dependability features. This paper presents a practical example of benchmarking key dependability features of four different transactional systems using a first proposal of dependability benchmark for OLTP application environments. This dependability benchmark is an extension to the TPC-C standard performance benchmark, and specifies the measures and all the steps required to evaluate both the performance and dependability features of OLTP systems. Two different versions of the Oracle transactional engine running over two different operating systems were evaluated and compared. The results show that dependability benchmarking can be successfully applied to OLTP application environments.


Dependability benchmarking, transactional systems, experimental dependability evaluation


Dependability benchmarking

Related Project

DBench - Dependability Benchmarking


International Conference on Dependable Systems and Networks, DSN-DCC 2003, June 2003

Cited by

Year 2013 : 1 citations

 Anna Lanzaro, Roberto Natella, Ricardo Barbosa, "Tools for Injecting Software Faults at the Binary and Source-Code Level", Innovative Technologies for Dependable OTS-Based Critical Systems, 2013.

Year 2011 : 1 citations

 Roberto Natella, “Achieving Representative Faultloads in Software Fault Injection”, PhD Thesis, Universita' Degli Studi di Napoli Federico II, Italy, November 2011.

Year 2009 : 1 citations

 1. Young Jun Cho, Yun Chan Cho, Jae Wook Jeon, "D-Bus based user device driver framework design for Linux mobile software platform", IEEE International Symposium on Industrial Electronics (ISIE 2009), Seoul, Korea, July 2009.

Year 2008 : 3 citations

 1. Gustavo M. D. Vieira, Willy Zwaenepoel, Luiz E. Buzato, “Dynamic Content Web Applications: Crash, Failover, and Recovery Analysis”, Technical Report - IC-08-34, Instituto de Computacão, University of Campinas, Campinas, Brazil, December 2008.

 2. Arnaud Albinet, Jean Arlat, Jean-Charles Fabre, “Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel”, Dependability Benchmarking for Computer Systems, Lisa Spainhower & Karama Kanoun Eds., August 2008.

 Kamalendu Majumdar, "Designing of Bibliographical Data-Base Management System: An Object Oriented Approach", PhD Thesis, Guru Ghasidas University, Bilaspur(CG), India, 2008.

Year 2006 : 1 citations

 1. L. Strigini et al., “Resilience-Building Technologies: State of Knowledge – ReSIST NoE Deliverable D12”, deliverable of the project: ReSIST: Resilience for Survivability in IST - A European Network of Excellence (ReSIST: Contract Number: 026764), September 2006.

Year 2005 : 4 citations

 1. Ricardo Gimenes, “Avaliação da Confiabilidade da Informação Utilizada em Sistemas de Informação e em Sistemas Críticos”, MSc Thesis, University of São Paulo, São Paulo, Brazil, 2005.

 2. Ali Kalakech, “Étalonnage de la sûreté de fonctionnement des systèmes d’exploitation – Spécifications et mise en œuvre”, PhD Thesis, LAAS-CNRS, Toulouse, France, August 2005.

 3. Richard Elling, “Availability Benchmarking”, RAS Engineering Group, Performance and Availability Engineering, Sun Microsystems, Inc., 2005.

 4. Cristian Constantinescu, “Dependability Benchmarking Using Environmental Test Tools”, Proceedings of IEEE Reliability and Maintainability Symposium, RAMS 2005, January 24-27, 2005.

Year 2004 : 4 citations

 1. Ali Kalakech, Tahar Jarboui, Jean Arlat, Yves Crouzet, Karama Kanoun, “Benchmarking Operating System Dependability: Windows 2000 as a Case Study”, 10th IEEE Pacific Rim International Symposium on Dependable Computing, PRDC2004, Papeete, Tahiti, French Polynesia, March 3-5, 2004.

 2. Arnaud Albinet, Jean Arlat, Jean-Charles Fabre, “Characterization of the Impact of Faulty Drivers on the Robustness of the Linux Kernel”, IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2004, Florence, Italy, June 28-July 1, 2004.

 3. Dong Tang, Dileep Kumar, Sreeram Duvur, Oystein Torbjornsen, “Availability Measurement and Modelling for An Application Server”, IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2004, Florence, Italy, June 28-July 1, 2004.

 4. A. Romanovsky et al., “CaberNet Vision of Research and Technology Development in Distributed and Dependable Systems”, Network of Excellence in Distributed and Dependable Computing Systems, IST Contract No. IST-2000-25088, January 2004.

Year 2003 : 1 citations

 1. Aaron Brown, “A Recovery-Oriented Approach to Dependable Services: Repairing Past Errors with System-Wide Undo”, PhD Thesis, EECS Computer Science Division - University of California, Berkeley, California, USA, December 2003.