Joint Evaluation of Performance and Robustness of a COTS DBMS Through Fault-Injection



This paper presents and discusses observed failure modes of a common-off-the-shelf (COTS) Database Management System (DBMS) under the presence of transient operational faults induced by SWIFI. The standard Transaction Processing Performance Council (TPC) TPC-C benchmark and associated environment is used here together with fault-injection technology, building a framework that discloses both dependability and performance figures. Over 1600 faults were injected in the database server of a Client/Server computing environment built upon Oracle 8.1.5 database engine and Windows NT running on COTS machines with Intel Pentium processors. A macroscopic view on the impact of faults revealed that: 1) A large majority of the faults caused no observable abnormal impact in the database server; In 96% of hardware faults and 80% of software faults the database server behaved normally. 2) Software faults are more prone to let the database server hanging or cause abnormal terminations. 3) Up to 51% of software faults lead to observable failures in the client processes.


Experimental COTS Evaluation


Dependable Systems and Networks Conference, DSN-2000, June 2000

Cited by

Year 2008 : 1 citations

 Gabriella Carrozza, "Software Faults Diagnosis in Complex, OTS-Based, Critical Systems?, PhD Thesis, "FEDERICO II? University of Naples, Naples, Italy, November, 2008.

Year 2007 : 1 citations

 Salvatore Orlando, "Software Aging Analysis of Off The Shelf Software Items?, PhD Thesis Salvatore Orlando, "FEDERICO II? University of Naples, Naples, Italy, October, 2007.

Year 2005 : 1 citations

 Denise Eng, Dimitri Mostinski, and Martin Talbot, "Feasibility and Practical Considerations of Database Self-Managing Technologies?, CS848 Final Project, University of Waterloo, Waterloo, ON, Canada, 2005 (

Year 2003 : 3 citations

 Kiran Nagaraja, Neeraj Krishnan, Ricardo Bianchini, Richard P. Martin, Thu D. Nguyen, "Quantifying and Improving the Availability of High-Performance Cluster-Based Internet Services?, In Proceedings of SC-2003, Phoenix, AZ, November, 2003

 Regina Moraes, "Strategy for testing of an object oriented database using software fault injection", Supplemental volume of the IEEE/IFIP International Conference on Dependable Systems and Networks, Dependable Computing and Communications, DSN-DCC 2003, San Francisco, CA, USA, pp. A-43-A-45, June 22-25, 2003.

 Kiran Nagaraja, Neeraj Krishnan, Ricardo Bianchini, Richard P. Martin, Thu D. Nguyen, "Quantifying and Improving the Availability of Cooperative Cluster-Based Internet Service", DCS-TR-517, Technical Report, Rutgers State University, NJ, USA, 2003.

Year 2002 : 2 citations

 Ravishankar K. Iyer, Zbigniew Kalbarczyk, "Hardware and Software Error Detection", Book Chapter (44 pages), used in classes at the Center for Reliable and High-Performance Computing, Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, 1308 W. Main Street, Urbana, IL 61801, 2002.

 Alberto Arazo, "Surete de Fonctionnement de Systemes Logiciels Critiques : Validation Experimentale d"un Systeme de Contrôle-Comande du Commerce?, tese de doutoramento, LAAS-CNRS, Toulouse, França, Julho de 2002.

Year 2001 : 1 citations

 Maitrayi Sabaratnam, "Improving Dependability in Database Management Systems?, tese de doutoramento de Maitrayi Sabaratnam, Novembro de 2001, Universidade de Trondheim, Trondheim, Noruega.

Year 2000 : 2 citations

 Subhachandra Chandra "An Evaluation of the Recovery-Related Properties of Software Faults" PhD Thesis, University of Michigan, USA, 2000.

 P. Chevochot and I. Puaut. Experimental evaluation of the fail-silent behavior of a distributed real-time run-time support built from COTS components. Technical Report 1370, IRISA, Oct. 2000