A Join Optimizer Based on Asynchronous Dueling DQN and Plan Latency Prediction Network

GAO Rui-wei; QIAO Shao-jie; HAN Nan; MIN Sheng-jie; LI He; QIN Xiao; ZHANG Tao; YUAN Chang-an

doi:10.12263/DZXB.20220490

您当前的位置：

首页 >

文章列表页 >

A Join Optimizer Based on Asynchronous Dueling DQN and Plan Latency Prediction Network

PAPERS | 更新时间：2025-12-08

- A Join Optimizer Based on Asynchronous Dueling DQN and Plan Latency Prediction Network
- ACTA ELECTRONICA SINICA Vol. 51, Issue 7, Pages: 1868-1874(2023)
- 作者机构：
  
  1.成都信息工程大学软件工程学院,四川成都 610225
  2.成都信息工程大学管理学院,四川成都 610103
  3.中电智元数据科技有限公司,北京 100081
  4.西安电子科技大学计算机科学与技术学院,陕西西安 710126
  5.南宁师范大学,广西南宁 530001
  6.宜宾学院,四川宜宾 644000
  7.广西科学院,广西南宁 530007
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62272066;61962006);Sichuan Science and Technology Program(2021JDJQ0021;2022YFG0186);Planning Foundation for Humanities and Social Sciences of Ministry of Education of China(22YJAZH088);Chengdu ‘Take the Lead’ Science and Technology Project(2022-JB00-00002-GX;2021-JB00-00025-GX);Chengdu Technology Innovation and Research and Development Project(2021-YF05-02413-GX;2021-YF05-02414-GX);The 54th Research Institute of China Electronics Technology Group Corporation-University Cooperation Project(SKX212010057);Science and Technology Innovation Capability Improvement Project of Chengdu University of Information Technology(KYTD202222)
- DOI：10.12263/DZXB.20220490
  CLC： TP311
- Received：05 May 2022，
  
  Revised：2023-02-18，
  
  Published：25 July 2023
- 稿件说明：
移动端阅览
高瑞玮,乔少杰,韩楠等.基于异步Dueling DQN和计划时间预测网络的连接优化器[J].电子学报,2023,51(07):1868-1874.

GAO Rui-wei,QIAO Shao-jie,HAN Nan,et al.A Join Optimizer Based on Asynchronous Dueling DQN and Plan Latency Prediction Network[J].ACTA ELECTRONICA SINICA,2023,51(07):1868-1874.
高瑞玮,乔少杰,韩楠等.基于异步Dueling DQN和计划时间预测网络的连接优化器[J].电子学报,2023,51(07):1868-1874. DOI： 10.12263/DZXB.20220490.

GAO Rui-wei,QIAO Shao-jie,HAN Nan,et al.A Join Optimizer Based on Asynchronous Dueling DQN and Plan Latency Prediction Network[J].ACTA ELECTRONICA SINICA,2023,51(07):1868-1874. DOI： 10.12263/DZXB.20220490.

摘要

连接顺序选择是查询优化领域中极具挑战性的研究方向，对于数据库管理系统获得良好的查询性能至关重要.然而，传统优化方法和现有智能优化方法均存在着不足，如规划时间过长、容易得到质量较差的连接计划、编码未考虑结构特征、依赖基数估计和代价估计使得连接计划无法反映真实的执行时间等.针对上述问题，提出了一种新型基于异步Dueling DQN（Deep Q-network）和计划时间预测网络的连接优化器：ADP-Join（Asynchronous Dueling DQN and Plan Latency Prediction Network for Join Order Selection）.ADP-Join集成了一种新的编码方法，能够区分不同结构的连接计划.ADP-Join设计了计划时间预测网络PLN（Plan Latency Prediction Network）来改善现有基于强化学习优化器的奖励机制.再者，提出异步更新机制改进Dueling DQN模型来提升训练性能和减少训练时间.大量的实验结果表明，在TPC-H和JOB真实数据集上ADP-Join的性能优于现有的智能优化器.

Abstract

Join order selection is a challenging research topic in the field of query optimization

and it is very important for database management system to obtain good query performance. However

both traditional optimization methods and existing intelligent optimization methods have disadvantages such as long planning time

easily to obtain poor quality join plans

encoding without considering structural characteristics

making join plans unable to reflect the real execution time due to dependency on cardinality estimation and cost estimation. In order to solve the above problems

a new join optimizer ADP-Join (Asynchronous Dueling DQN and Plan latency prediction network for Join order selection) is proposed. ADP-Join integrates a new encoding method that can distinguish join plans of different structures. ADP-Join designs a plan latency prediction network to improve the reward mechanism of existing reinforcement learning-based optimizers.Furthermore

the asynchronous update mechanism is proposed to improve the Dueling DQN model to improve the training performance and reduce the training time. Extensive experimental results show that ADP-Join outperforms existing intelligent optimizers on real TPC-H and JOB datasets.

关键词

Keywords

references

QIAO S J , YANG G P , HAN N , et al . Cardinality estimator: Processing SQL with a vertical scanning convolutional neural network [J]. Journal of Computer Science and Technology , 2021 , 36 ( 4 ): 762 - 777 .

LI G L , ZHOU H X , GAO L . AI meets database: AI4DB and DB4AI [C]// Proceedings of SIGMOD Conference . New York : ACM , 2021 : 2859 - 2866 .

LI G L , ZHOU X H , SUN J . openGauss: Anautonomous database system [J]. Proceedings of VLDB Endowment , 2021 , 14 ( 12 ): 3028 - 3041 .

MARCUS R , PAPAEMMANOUIL O . Towards a hands-free query optimizer through deep learning [EB/OL]. ( 2018 )[2022]. https://api.semanticscholar.org/CorpusID:52880119 https://api.semanticscholar.org/CorpusID:52880119 .

YU X , LI G L , CHAI C L . Reinforcement learning with tree-LSTM for join order selection [C]// Proceedings of 36th International Conference on Data Engineering . Washington : IEEE , 2020 : 1297 - 1308 .

ZHOU H X , LI G L , CHAI C L . A learned query rewrite system using Monte Carlo tree search [J]. Proceedings of VLDB Endowment , 2021 , 15 ( 1 ): 46 - 58 .

KAOUDI Z . ML-based cross-platform query optimization [C]// Proceedings of the 36th International Conference on Data Engineering . Washington : IEEE , 2020 : 1489 - 1500 .

MARCUS R . Deep reinforcement learning for join order enumeration [C]// Proceedings of the First International Workshop on Exploiting Artificial Intelligence Techniques for Data Management . New York : ACM , 2018 : 31 - 34 .

KRISHNAN S , YANG Z H , GOLDBERG K , et al . Learning to optimize join queries with deep reinforcement learning [EB/OL].( 2018 )[2022]. https://api.semanticscholar.org/CorpusID:51952349 https://api.semanticscholar.org/CorpusID:51952349 .

WANG J Y , CHAI C L , LIU J B . FACE: A normalizing flow based cardinality estimator [J]. Proceedings of VLDB Endowment , 2021 , 15 ( 1 ): 72 - 84 .

MARCUS R , NEGI P , MAO H Z . Neo: A learned query optimizer [J]. Proceedings of VLDB Endowment , 2019 , 12 ( 11 ): 1705 - 1718 .

MNIH V , KAVUKCUOGLU K , SILVER D . Human-level control through deep reinforcement learning [J]. Nature , 2015 , 518 ( 7540 ): 529 - 533 .

DEY R , SALEM F M . Gated recurrent unit neural networks gate-variants [C]// Proceedings of 60th International Midwest Symposium on Circuits and Systems . Washington : IEEE , 2017 : 1597 - 1600 .

POSS M , FLOYD C . New TPC benchmarks for decision support and web commerce [J]. SIGMOD Record , 2000 , 29 ( 4 ): 64 - 71 .

LEIS V , GUBICHEV A , MIRCHEV A . How good are query optimizers, really? [J]. Proceedings of VLDB Endowment , 2015 , 9 ( 3 ): 204 - 215 .

ZHANG J , ABEDJAN Z , HOSE K . AlphaJoin: Join order selection à la AlphaGo [C]// Proceedings of the VLDB 2020 PhD Workshop Co-located with the 46th International Conference on Very Large Databases . Trondheim : VLDB Endowment , 2020 : 5 - 8 .

WASS F . Join order selection [C]// Proceedings of British National Conference on Databases . Berlin : Springer , 2000 : 256 - 265 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

CS-ROMF:Efficient Community Search Based on Graph Combinatorial Optimization

Knowledge-Based and Data-Driven Integrating Design Methodology for Air Combat Strategy in Multi-Opponent Adversarial Game

Reinforcement Learning Based Tuning-free Plug-and-Play Image Reconstruction Method for Single Photon Imaging

Optimal Directed Control of Discrete Event Systems Based on Reinforcement Learning

Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning

Related Author

ZHANG An-ran

WANG Xing-fen

ZHAO Yu-han

LI Li-bo

FENG Jin-yuan

CHEN Min

LI Jun-ying

CHEN Jia-le

Related Institution

Beijing Information Science and Technology University

Hong Kong Baptist University, Hongkong

Institute of Automation, Chinese Academy of Sciences

School of Artificial Intelligence, University of Chinese Academy of Sciences

School of Computer Science, Nanjing University of Information Science & Technology

⁰