Proactive Re-optimization



Traditional query optimizers rely on the accuracy of estimated
statistics to choose good execution plans. This design often leads to
suboptimal plan choices for complex queries, since errors in
estimates for intermediate subexpressions grow exponentially in the
presence of skewed and correlated data distributions. Reoptimization
is a promising technique to cope with such mistakes.
Current re-optimizers first use a traditional optimizer to pick a
plan, and then react to estimation errors and resulting
suboptimalities detected in the plan during execution. The
effectiveness of this approach is limited because traditional
optimizers choose plans unaware of issues affecting reoptimization.
We address this problem using proactive reoptimization,
a new approach that incorporates three techniques:
i) the uncertainty in estimates of statistics is computed in the form of
bounding boxes around these estimates, ii) these bounding boxes are
used to pick plans that are robust to deviations of actual values from
their estimates, and iii) accurate measurements of statistics are
collected quickly and efficiently during query execution. We present
an extensive evaluation of these techniques using a prototype
proactive re-optimizer named Rio. In our experiments Rio
outperforms current re-optimizers by up to a factor of three.

PDF File

Cited by

Year 2008 : 3 citations

 Robustness in automatic physical database design
Kareem El Gebaly, Ashraf Aboulnaga
Proceedings of the 11th international conference on Extending database technology: Advances in database technology (EDBT'2008)

 Large Scale Data Management in Grid Systems: a Survey
Hameurlain, A.; Morvan, F.; El Samad, M.
Information and Communication Technologies: From Theory to Applications, 2008. ICTTA 2008

 Proactive and reactive multi-dimensional histogram maintenance for selectivity estimation
Zhen He, Byung Suk Lee, and X. Sean Wang
Journal of Systems and Software, Volume 81, Issue 3, March 2008, Pages 414-430
Selected Papers from the 2006 Brazilian Symposia on Databases and on Software Engineering

Year 2007 : 13 citations

 Sequential Design of Experiments via Linear Programming
Sudipto Guha, Kamesh Munagala
ACM Symposium on Theory of Computing (STOC)

 Robust Plans through Plan Diagram Reduction
Harish D., Pooja N. Darera, Jayant R. Haritsa
Technical Report TR-2007-02
Database Systems Lab, Supercomputer Education and Research Centre
Indian Institute of Science

 Robust Placement of Mobile Relational Operators for Large Scale Distributed Query Optimization
Belgin Ergenc, Franck Morvan, Abdelkader Hameurlain
Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies, 2007. PDCAT '07.

 Reduction of Query Optimizer Plan Diagrams
Pooja N. Darera
MSc Thesis
Supercomputer Education and Research Centre, INDIAN INSTITUTE OF SCIENCE, INDIA

 Adaptively Reordering Joins during Query Execution
Q Li, M Shao, V Markl, K Beyer, L Colby, G Lohman
ICDE 2007

 A Lightweight Online Framework For Query Progress Indicators
C Mishra, N Koudas
ICDE 2007

 Collecting and Maintaining Just-in-Time Statistics
A El-Helw, IF Ilyas, W Lau, V Markl, C Zuzarte
ICDE 2007

 Model-driven optimization using adaptive probes
S Guha, K Munagala
Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, 2007

 Progressive optimization in a shared-nothing parallel database
WS Han, J Ng, V Markl, H Kache, M Kandil
Proceedings of the 2007 ACM SIGMOD

 Automation Everywhere: Autonomics and Data Management
NW Paton
Lecture Notes in Computer Science
Springer Berlin / Heidelberg
Data Management. Data, Data Everywhere
Volume 4587/2007, Pages 3-12, August 19, 2007

 Robust Plans through Plan Diagram Reduction
Harish D. Pooja N. Darera Jayant R. Haritsa
Technical Report TR-2007-02, Indian Institute of Science - Bangalore, India

 HybMig: A Hybrid Approach to Dynamic Plan Migration for Continuous Queries
Y Yang, J Kramer, D Papadias, B Seeger
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2007

 Robustness in Automatic Physical Database Design
Kareem El Gebaly
Technical Report CS-2007-29
David R. Cheriton School of Computer Science University of Waterloo

Year 2006 : 8 citations

 How to Probe for an Extreme Value
Ashish Goel, Sudipto Guha, Kamesh Munagala
ACM SIGACT-SIGMOD-SIGARTSymposium on Principles of Database Systems (PODS), 2006

 Autonomic Query Parallelization using Non-dedicated Computers: An Evaluation of Adaptivity Options
N.W. Paton, V. Raman, G. Swart, I. Narang
2006 IEEE International Conference on Autonomic Computing pp. 221-230

 Asking the Right Questions: Model-driven Optimization using Probes
A Goel, S Guha, K Munagala
PODS 2006

 A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing
Kwanchai Eurviriyanukul, Alvaro A. A. Fernandes and Norman W. Paton
Lecture Notes in Computer Science
Current Trends in Database Technology - EDBT 2006
Springer Berlin / Heidelberg
Volume 4254/2006, pages 589-600, October 17, 2006

 Jürgen Krämer, Yin Yang, Michael Cammert, Bernhard Seeger and Dimitris Papadias
Lecture Notes in Computer Science
Current Trends in Database Technology - EDBT 2006
Springer Berlin / Heidelberg
Volume 4254/2006, pages 497-516, October 17, 2006

 Autonomic Query Parallelization using Non-dedicated Computers: An Evaluation of Adaptivity Options
NW Paton, V Raman, G Swart, I Narang
Proc. 3rd Intl. Conference on Autonomic Computing (ICAC 2006)

 Adaptive Query Processing
A Deshpande, Z Ives, V Raman
Foundations and Trends® in Databases: Vol. 1: No 1, pp 1-140, 2006

 Recursive SQL Query Optimization with k-Iteration Lookahead
Ahmad Ghazal1 Contact Information, Alain Crolotte1 Contact Information and Dawit Seid
Lecture Notes in Computer Science
Database and Expert Systems Applications
Springer Berlin / Heidelberg
Volume 4080/2006, Pages 348-357, September 21, 2006

Year 2005 : 2 citations

 Query Processing using Negative Tuples in Stream Query Engines
T. M. Ghanem, et al
Technical Report 04-040, Purdue University, April 2005.

 Data-Driven Batch Scheduling
John Bent
PhD Thesis, University of Wisconsin - Madison, 2005