A Study on Scheduling Method of Hadoop Yarn

LI Yuan-zhen; YANG Qun; LAI Shang-qi; LI Bo-han

doi:10.3969/j.issn.0372-2112.2016.05.002

您当前的位置：

首页 >

文章列表页 >

A Study on Scheduling Method of Hadoop Yarn

更新时间：2025-07-16

- A Study on Scheduling Method of Hadoop Yarn
- Acta Electronica Sinica Vol. 44, Issue 5, Pages: 1017-1024(2016)
- 作者机构：
  
  1. 南京航空航天大学计算机科学与技术学院,江苏,南京,210016
  2. 香港大学计算机科学与技术系,香港
  3. 南京航空航天大学计算机科学与技术学院,江苏,南京,210016
  4. 香港大学计算机科学与技术系,香港
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.0372-2112.2016.05.002
  CLC： TP301.6
- Published Online：25 May 2016，
  
  Published：2016
- 稿件说明：
移动端阅览
LI Yuan-zhen, YANG Qun, LAI Shang-qi, et al. A Study on Scheduling Method of Hadoop Yarn[J]. Acta Electronica Sinica, 2016, 44(5): 1017-1024.
DOI：

LI Yuan-zhen, YANG Qun, LAI Shang-qi, et al. A Study on Scheduling Method of Hadoop Yarn[J]. Acta Electronica Sinica, 2016, 44(5): 1017-1024. DOI： 10.3969/j.issn.0372-2112.2016.05.002.

摘要

针对Hadoop Yarn资源调度问题

为提高集群作业执行效率

提出一种基于蚁群算法与粒子群算法的自适应Hadoop资源调度算法SRSAPH.SRSAPH中

通过Hadoop Yarn跳通信机制获取负载、内存、CPU速度等属性信息初始化信息素矩阵;同时

将粒子群算法的自我认知能力与社会认知能力引入到蚁群算法

提高算法的收敛速度;此外

根据蚁群算法全局最优解的波动趋势动态调整信息素挥发系数

提高解的精度.实验表明

采用SRSAPH进行资源调度

集群的作业执行时间缩短至少10%.

Abstract

In view of the resource scheduling problem of Hadoop Yarn

to improve the execution efficiency of the cluster job

we propose a Self-adapt Resource Scheduling algorithm based on Ant Colony Algorithm and Particle Swarm Algorithm in Hadoop (SRSAPH).In SRSAPH

we initialize the pheromone matrix of SRSAPH by using the attribute information of load

memory

and CPU speed obtained through the heartbeat message transfer mechanism.Meanwhile

we introduce the self-cognitive ability and social cognition ability of particle swarm algorithm into the ant colony algorithm to speed up the rate of convergence of the algorithm.Moreover

we dynamically adjust the pheromone evaporation rate based on the fluctuation trends of global optimal solution to enhance the accuracy of the solutions.Experimental result shows that by using SRSAPH in resource scheduling

the execution time of cluster job is shorten by 10%.

关键词

Keywords

references

Views

762

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Utility Optimization-Based Joint User Association and Resource Allocation Algorithm for Integrated Satellite-Terrestrial Network

ISAR Resource Scheduling of Multiple Micro-Motion Targets Based on Genetic-Heuristic Algorithm

Task Offloading Scheme Based on Comprehensive Trust Evaluation in Edge Computing

Resource Scheduling Algorithm of Rotating Phased Array Radar Based on Generalized Time Window

Graph Learning Based Transmission Resources Scheduling in Dense Space Networks

Related Author

CHAI Rong

CHEN Mi-ling

LI Jin-hong

WANG Yue

HUANG Lu

QI Hao-fan

TIAN Xu-dong

BAI Xue-ru

Related Institution

School of Communications and Information Engineering， Chongqing University of Posts and Telecommunications

National Laboratory of Radar Signal Processing， Xidian University

Beijing Institute of Tracking and Communication Technology

Key Laboratory of Electronic Information Countermeasure and Simulation Technology， Ministry of Education， Xidian University

College of Science, Jiangxi University of Science and Technology

⁰