Repair Pipelining for Erasure-Coded Storage Based on Load-Balanced

JIANG Xiao-yu; LI Gui-yang; ZHOU Yue; HU Jing-ping; LI Hui

doi:10.3969/j.issn.0372-2112.2020.05.013

您当前的位置：

首页 >

文章列表页 >

Repair Pipelining for Erasure-Coded Storage Based on Load-Balanced

更新时间：2025-07-16

- Repair Pipelining for Erasure-Coded Storage Based on Load-Balanced
- Acta Electronica Sinica Vol. 48, Issue 5, Pages: 930-936(2020)
- 作者机构：
  
  四川师范大学计算机科学学院,四川,成都,610101
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China (No.61701331)
- DOI：10.3969/j.issn.0372-2112.2020.05.013
  CLC： TP333
- Published Online：25 May 2020，
  
  Published：2020
- 稿件说明：
移动端阅览
JIANG Xiao-yu, LI Gui-yang, ZHOU Yue, et al. Repair Pipelining for Erasure-Coded Storage Based on Load-Balanced[J]. Acta Electronica Sinica, 2020, 48(5): 930-936.
DOI：

JIANG Xiao-yu, LI Gui-yang, ZHOU Yue, et al. Repair Pipelining for Erasure-Coded Storage Based on Load-Balanced[J]. Acta Electronica Sinica, 2020, 48(5): 930-936. DOI： 10.3969/j.issn.0372-2112.2020.05.013.

摘要

大数据分布式存储系统中，修复流水线（Repair Pipelining，RP）减少90%的修复时间，有效地解决由于修复时间开销较大，纠删码不适用于存储热数据的问题.然而，现有的RP存在节点负载不均衡的问题，导致系统性能下降.通过研究后，设计节点负载均衡的纠删码修复流水线（Node Load Balancing-based Repair Pipelining，NLB-RP），并根据性能评价指标提出计算节点负载的算法和计算修复时间的算法.理论分析及实验结果表明，在没有引入额外修复代价的情况下，NLB-RP从局部到整体有效地平衡并减少节点的负载.相比RP，NLB-RP的节点负载方差为0，即每个节点的负载相同.因此，NLB-RP具有最优的负载均衡性.

Abstract

In distributed storage system for big data

the repair pipelining (RP) reduces repair time by 90%

which effectively solves the problem that erasure code is not suitable for storing hot data due to the heavy overhead of repair time. However

the existing RP has the problem of unbalanced load among nodes

which leads to the degradation of system performance. In this paper

a repair pipelining based on node load balancing (NLB-RP) is designed

and then the algorithms for calculating the load of nodes and repair time according to the evaluation index of performance are proposed. Theoretical analysis and experimental results both show that

from local to global

the NLB-RP can effectively balance and reduce the load of nodes without introducing extra repair cost. Compared with the RP

the load variance of the NLB-RP is zero

that is

the load of each node is the same. Thus

the proposed NLB-RP has the optimal load balance.

关键词

Keywords

references

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An Erasure Coding Technology Supporting Near-Data Processing in Distributed Storage Systems

First Filling Strategy-Based Partitioning Method to Balance Data in Spark

Optimization Study of the Load Balancing Algorithm in the Multi-Layer Lattice Boltzmann Method

Diverse User Service Requirement-Oriented Dynamic Resource Allocation Algorithm for Multi-Beam Satellite Systems

BSS: A Storage Optimization Scheme for the Permissioned Blockchain

Related Author

LI Hao-ran

HUANG Zhi-jie

SHI Yu-long

ZHAO Cheng-jia

ZHAO Nan-nan

ZHANG Xiao

HE Yu-lin

WU Dong-tong

Related Institution

School of Computer Science, Northwestern Polytechnical University

Guangdong Laboratory of Artificial Intelligence and Digital Economy

College of Computer Science and Software Engineering, Shenzhen University

School of Computer Engineering and Science, Shanghai University

School of Mechanics and Engineering Science, Shanghai University

⁰