一种基于Q-Learning策略的自适应移动物联网路由新算法

doi:10.3969/j.issn.0372-2112.2018.10.003

电子学报 ›› 2018, Vol. 46 ›› Issue (10): 2325-2332.DOI: 10.3969/j.issn.0372-2112.2018.10.003

一种基于Q-Learning策略的自适应移动物联网路由新算法

张德干^1,2, 葛辉^1,2, 刘晓欢^1,2, 张晓丹³, 李文斌^1,2

1. 天津理工大学天津市智能计算及软件新技术重点实验室, 天津 300384;
2. 天津理工大学计算机视觉与系统省部共建教育部重点实验室, 天津 300384;
3. 中国科学技术信息研究所, 北京 100038

收稿日期:2017-08-16 修回日期:2017-12-09 出版日期:2018-10-25
- 通讯作者:
- 刘晓欢
- 作者简介:
- 张德干,男,1970年1月出生,湖北黄冈英山县人.博士,教授,博导.主要研究方向为物联网、移动计算、网络通信、智能控制等.E-mail:zhangdegan@tsinghua.org.cn;葛辉,男,1993年3月出生,山东济南人.天津理工大学硕士研究生.主要研究方向为物联网、网络通信等.E-mail:1464736574@qq.com;张晓丹,女,1972年5月出生,吉林通化人.中国科学技术信息研究所研究员.研究方向为机器学习等.E-mail:183650594@qq.com;李文斌,男,1991年6月出生,天津市人.天津理工大学硕士研究生.主要研究方向为网络通信.E-mail:549411835@qq.com
- 基金资助:
- 国家自然科学基金 (No.61571328）; 天津市重大科技专项 (No.15ZXDSGX00050，No.16ZXFWGX00010）; 天津市科技支撑重点项目 (No.17YFZCGX00360）; 天津市自然科学基金 (No.15JCYBJC46500）; 天津市科技创新和131人才团队 (No.TD12-5016，2015-23，No.TD13-5025）

A Kind of New Routing Algorithm with Adaptivity for Mobile IOT Based on Q-Learning

ZHANG De-gan^1,2, GE Hui^1,2, LIU Xiao-huan^1,2, ZHANG Xiao-dan³, LI Wen-bin^1,2

1.Tianjin Key Lab of Intelligent Computing & Novel Software Technology, Tianjin University of Technology, Tianjin 300384, China;
2.Key Laboratory of Computer Vision and System (Tianjin University of Technology), Ministry of Education, Tianjin University of Technology, Tianjin 300384, China;
3.Institute of Institute of Scientific and Technical Information of China, Beijing 100038, China

Received:2017-08-16 Revised:2017-12-09 Online:2018-10-25 Published:2018-10-25
- Corresponding author:
- LIU Xiao-huan
- Supported by:
- National Natural Science Foundation of China (No.61571328); Key Science and Technology Project of Tianjin Municipal (No.15ZXDSGX00050, No.16ZXFWGX00010); Key Program of Tianjin science and technology support project (No.17YFZCGX00360); Natural Science Foundation of Tianjin Municipality, China (No.15JCYBJC46500); Tianjin Science and Technology Innovation and 131 Talent Team (No.TD12-5016, 2015-23, No.TD13-5025)

摘要/Abstract

摘要： 针对移动物（车）联网的路由问题，通过对车辆的运动特点及造成链路断裂的原因进行的详细分析，我们建立了链路维持时间模型，并将维持时间作为设计路由算法的重要参数.Q-Learning作为一种启发式机器学习策略，能够通过与周围环境交互来动态地调整路由路径.基于此，我们设计了一种自适应的路由新算法.它将学习任务分散在每一个车辆节点中，通过周期性的与周围节点交换信标信息来维护可靠的路由路径.利用NS-2模拟器对该算法的性能进行了评估，结果表明，在不同的网络场景中，该算法在递交率、端到端的延时以及平均跳数等方面均表现出很好的效果.

关键词: 机器学习, 移动物联网, 拓扑, 动态, 路由

Abstract: In order to solve the routing problem of mobile IOT (IOV), based on our analyzing the details about motion characteristics of the vehicle and the reasons that cause links down, we set up link model of the duration time and using the duration time as key parameter to design the new routing method. Q-Learning as a kind of heuristic machine learning strategy is able to dynamically adjust the routing path through interaction with the surrounding environment. So a kind of new routing algorithm with adaptivity for mobile IOT based on Q-learning has been presented in this paper. It distributes the learning task into each vehicle node and maintains the reliable routing path by continuously exchanging the beacon information with the neighbor nodes. With the NS-2 simulator, the performance of the algorithm is tested. The results show that it has better performances on delivery, end-to-end delay and average hops in many mobile applications.

Key words: machine learning, mobile IOT, Topology, dynamic, routing

中图分类号:

TP393

张德干, 葛辉, 刘晓欢, 等. 一种基于Q-Learning策略的自适应移动物联网路由新算法[J]. 电子学报, 2018, 46(10): 2325-2332.

ZHANG De-gan, GE Hui, LIU Xiao-huan, et al. A Kind of New Routing Algorithm with Adaptivity for Mobile IOT Based on Q-Learning[J]. Acta Electronica Sinica, 2018, 46(10): 2325-2332.

参考文献

[1] Namboodiri V,Gao L.Prediction-based routing for vehicular ad hoc networks[J].IEEE Transactions on Vehicular Technology,2007,56(4):2332-2345.
[2] Zhang D G.A new approach and system for attentive mobile learning based on seamless migration[J].Applied Intelligence,2012,36(1):75-89.
[3] Liu S.Novel unequal clustering routing protocol considering energy balancing based on network partition & distance for mobile education[J].Journal of Network and Computer Applications,2017,88(15):1-9.
[4] Ma Z.A novel compressive sensing method based on SVD sparse random measurement matrix in wireless sensor network[J].Engineering Computations,2016,33(8):2448-2462.
[5] Zhang X D.Design and implementation of embedded un-interruptible power supply system (EUPSS) for web-based mobile application[J].Enterprise Information Systems,2012,6(4):473-489.
[6] Song X D.Extended AODV routing method based on distributed minimum transmission (DMT) for WSN[J].International Journal of Electronics and Communications,2015,69(1):371-381.
[7] Zhao C P.A new medium access control protocol based on perceived data reliability and spatial correlation in wireless sensor network[J].Computers & Electrical Engineering,2012,38(3):694-702.
[8] Zhang D G,Liang Y P.A kind of novel method of service-aware computing for uncertain mobile applications[J].Mathematical and Computer Modelling,2013,57(3-4):344-356.
[9] Zhang D G,Li G.An energy-balanced routing method based on forward-aware factor for wireless sensor network[J].IEEE Transactions on Industrial Informatics,2014,10(1):766-773.
[10] Ma Z.Shadow detection of moving objects based on multisource information in internet of things[J].Journal of Experimental & Theoretical Artificial Intelligence,2017,29(3):649-661.
[11] Zhang D G.A novel image de-noising method based on spherical coordinates system[J].EURASIP Journal on Advances in Signal Processing,2012,2012(110):1-10.DOI:10.1186/1687-6180-2012-110.
[12] Eiza M H.An evolving graph-based reliable routing scheme for VANETs[J].IEEE Transactions on Vehicular Technology,2013,62(4):1493-1504.
[13] Yan G J,Olariu S.A probabilistic analysis of link duration in vehicular ad hoc networks[J].IEEE Transactions on Intelligent Transportation Systems,2011,12(5):41227-41236.
[14] 张德干.基于局域世界的WSN拓扑加权演化模型[J].电子学报,2012,40(5):1000-1004. ZHANG De-gan.Local-world weighted topology evolving model for wireless sensor networks[J].Acta Electronica Sinica,2012,40(5):1000-1004.(in Chinese)
[15] 钟远.一种不可信环境下的匿名位置辅助路由激励机制[J].电子学报,2013,41(3):475-480. ZHONG Yuan.An anonymous incentive mechanism for location-aided routing in untrusted MANET scenarios[J].Acta Electronica Sinica,2013,41(3):475-480.(in Chinese)
[16] 张德干.一种基于前向感知因子的WSN能量均衡路由方法[J].电子学报,2014,42(1):113-118. ZHANG De-gan.A kind of routing protocol based on forward-aware factor for energy balance[J].Acta Electronica Sinica,2014,42(1):113-118.(in Chinese)
[17] Zhang D G.A novel approach to mapped correlation of ID for RFID anti-collision[J].IEEE Transactions on Services Computing,2014,7(4):741-748.
[18] Zhu Y N.A new constructing approach for a weighted topology of wireless sensor networks based on local-world theory for the internet of things (IOT)[J].Computers & Mathematics with Applications,2012,64(5):1044-1055.
[19] Song X D.New agent-based proactive migration method and system for big data environment (BDE)[J].Engineering Computations,2015,32(8):2443-2466.
[20] Zhang D G.A kind of novel VPF-based energy-balanced routing strategy for wireless mesh network[J].International Journal of Communication Systems,2017,30(6):1-15.
[21] Wang X.New clustering routing method based on PECE for WSN[J].EURASIP Journal on Wireless Communications and Networking,2015,2015(162):1-13.DOI:10.1186/s13638-015-0399-x.
[22] Zheng K.Novel quick start (QS) method for optimization of TCP[J].Wireless Networks,2016,22(1):211-222.
[23] Zheng K.A novel multicast routing method with minimum transmission for WSN of cloud computing service[J].Soft Computing,2015,19(7):1817-1827.

一种基于Q-Learning策略的自适应移动物联网路由新算法

A Kind of New Routing Algorithm with Adaptivity for Mobile IOT Based on Q-Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	吴军君, 王涛, 王英楷, 王星辉. 基于迁移学习的LiPON制备工艺模拟优化[J]. 电子学报, 2023, 51(3): 687-693.
[2]	王娜, 王俊平, 朱俊辉. 基于矩阵偏序关系的形态学算子[J]. 电子学报, 2023, 51(1): 213-221.
[3]	闫梦宇, 李金海, 刘文奇, 张文修. 带对象结构信息形式背景的概念知识发现与演化[J]. 电子学报, 2023, 51(1): 11-17.
[4]	田晖, 严艳, 汤莉莉, 吴俊彦, 王慧东, 全韩彧. 基于分数基音延迟动态搜索的语音隐写算法[J]. 电子学报, 2023, 51(1): 67-75.
[5]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[6]	刘群, 谭洪胜, 张优敏, 王国胤. 基于元路径的动态异质网络表示学习[J]. 电子学报, 2022, 50(8): 1830-1839.
[7]	王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936.
[8]	赵耿, 马英杰, 陈磊, 董有恒, 侯艳丽. 基于扰动时空混沌系统的动态S盒设计[J]. 电子学报, 2022, 50(8): 2037-2042.
[9]	麻文刚, 张亚东, 郭进. 基于景深先验引导与环境光优化的图像去雾[J]. 电子学报, 2022, 50(7): 1708-1721.
[10]	高雅, 吴洪博. 闭元确定的拓扑系统中闭包元及其应用[J]. 电子学报, 2022, 50(5): 1270-1276.
[11]	史芳静, 樊养余, 王鑫圆, 康博超, 陈博, 高永胜. 基于PDM-DPMZM的大动态范围微波光子I/Q下变频系统[J]. 电子学报, 2022, 50(4): 782-788.
[12]	郭春, 罗迪, 申国伟, 崔允贺, 平源. 一种基于诱导机制的间谍软件检测方法[J]. 电子学报, 2022, 50(4): 1014-1024.
[13]	陈书仪, 刘亚丽, 林昌露, 李涛, 董永权. 面向物联网的轻量级可验证群组认证方案[J]. 电子学报, 2022, 50(4): 990-1001.
[14]	黄璐, 蔚保国, 李宏生, 李隽, 贾浩男, 程建强, 李雅宁. GNSS拒止环境下的伪卫星指纹定位方法[J]. 电子学报, 2022, 50(4): 811-822.
[15]	崔亚奇, 何友, 唐田田, 熊伟. 一种深度学习航迹关联方法[J]. 电子学报, 2022, 50(3): 759-763.