Hybrid Honey Bee Mating and Deep Reinforcement Learning-Based Adaptive Query Plan Optimization

Abstract

Query plan optimization is a crucial element in the architecture of relational database management systems (DBMS), seeking to identify an ideal execution plan by reducing the overall execution time of queries. Our study employs a novel paradigm, specifically deep reinforcement learning (Deep RL), a subfield of machine learning that integrates reinforcement learning (RL) with deep learning to enhance query optimization methods, which constitute a complete NP problem. In this work , our contribution consists in combining the Honey-bee Mating optimization (HBMO) algorithm to efficiently explore the search space of query plans (stochastic search inspired by the behavior of bees) and adapt DRL algorithms to demonstrate their efficacy. The idea of our approch is to break the problem down into two complementary phases: the Exploration (Global Search) phase uses the Honey Bee Mating Optimization (HBMO) algorithm to find a number of "good" candidate plans by widely exploring the space of possible query plans. The Learning and Adaptation phase uses a Deep Reinforcement Learning (DRL) agent to learn how to choose the best plan among the candidates suggested by HBMO in the first phase.Synergy is produced by hybridization: DRL contributes adaptive intelligence that HBMO alone cannot, while HBMO shrinks the vast research space for DRL. We employ the Proximal Policy Optimization (PPO) method as a model-free approach. Our modest experiments on the IMDB benchmark revealed a progressive gain in Latency (15 to 25%) and a drastic reduction in physical I/O (30-40%), demonstrate that the DRL agent learns to prioritize the most effective plans.