云计算环境下基于用户体验的成本最优存储策略研究

doi:10.3969/j.issn.0372-2112.2014.01.004

电子学报 ›› 2014, Vol. 42 ›› Issue (1): 20-27.DOI: 10.3969/j.issn.0372-2112.2014.01.004

云计算环境下基于用户体验的成本最优存储策略研究

王宁^1,2, 杨扬¹, 孟坤³, 陈宇¹, 王磊¹, 季青¹

1. 北京科技大学计算机与通信工程学院, 北京 100083;
2. 烟台工程职业技术学院机电工程系, 山东烟台 264006;
3. 清华大学计算机科学与技术系, 北京 100084

收稿日期:2013-01-23 修回日期:2013-05-02 出版日期:2014-01-25
- 作者简介:
- 王宁 女，1978年出生于山东烟台.现为北京科技大学博士生，同时也是烟台工程职业技术学院讲师，主要从事云计算、数据挖掘等方面的研究. E-mail:wangning200658@126.com 杨扬 男，1955年出生于河北承德.教授、博士生导师.主要研究领域为云计算、智能控制、多媒体通信等. E-mail:yyang@ustb.edu.cn 孟坤 男，1980年生于河南项城.博士，现为清华大学博士后，主要研究领域为网络安全、性能评价等. 陈宇 男，1989年生于北京.现为北京科技大学硕士生，主要研究领域为云计算. 王磊 女，1982年生于河北张家口.现为北京科技大学博士生，主演研究领域为移动自组织网络和服务组合. 季青 女，1978年生于江苏淮安，现为北京科技大学博士生，主要研究领域为云计算.
- 基金资助:
- 国家自然科学基金 (No.61070182，No.61170209，No.61272528，No.61202432）

A Customer Experience-Based Cost Minimization Strategy of Storing Data in Cloud Computing

WANG Ning^1,2, YANG Yang¹, MENG Kun³, CHEN Yu¹, WANG Lei¹, JI Qing¹

1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China;
2. Department of Mechanical and Electrical Engineering, Yantai Engineering & Technology College, Yantai, Shandong 264006, China;
3. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

Received:2013-01-23 Revised:2013-05-02 Online:2014-01-25 Published:2014-01-25

摘要/Abstract

摘要： 为了提高大数据文件的存取效率，满足各类用户的需求，通常采用对该文件进行分块、冗余副本等机制进行存储，关于设置块大小、副本个数和块部署等存储机制的研究一直是该领域研究重点.根据用户对内容块兴趣需求，我们定义了数据块的热度并提出了一种满足用户需求的存储数据块的最小服务成本策略（MCSB）.在成本矩阵的基础上，通过引入与数据块热度相关的成本矩阵调整因子，使得热度较低的数据块被优先部署，实现了在不改变存储数据块的最小服务总成本的情况下，内容存取服务性能的提高.基于该策略，以Hadoop中的缺省数据块存储策略为控制组，通过在HDFS系统中实现MCSB，并对MCSB进行了较为详细的分析.实验结果显示MCSB策略能够在满足最小服务成本的情况下，具有更短的系统平均响应时间.进一步考虑到数据存储节点由服务器集群承担的事实，对基于不同负载下的数据存储策略进行了深入探讨，在分析现有机制对性能影响的基础上，给出了一种自适应的数据节点内的存储数据块的最小服务成本策略AMCSB，实验表明，本文所提出的AMCSB策略能够在降低服务成本的同时，有利于系统的负载均衡，并提高该系统的服务性能.

关键词: 云存储, 数据块, MCSB, AMCSB, 大数据

Abstract: Splitting a whole large file into some small blocks and distributedly deploying these block replications on severs can improve their application efficiency and meet users' demands.How to split a file,how many replications of a block should be created,and how to deploy these replications become the most critical issures in this field.In this paper,considering client's demands for each block,we introduce block-popularity and present a block deployment strategy according to it,called minimum cost of storing blocks (MCSB),which can satisfy users' requirement and minimum storage cost.In our strategy,through introducing cost adjustment factors of blocks,not only that lower hot blocks are assured to deploy earlier,but also the practical total storage cost are not exceed than the original case.To maintain the performance of service system,we study resource management methods in-depth among severs with different workloads and propose an adaptive MCSB strategy (AMCSB).We implement the MCSB (and AMCSB) in Hadoop and compare their performance with other related strategies.The experiment results show that MCSB fulfills minimum service cost and shortens average response time,and AMCSB realizes load balance among target servers.

Key words: cloud storage, data block, MCSB, AMCSB, big data

中图分类号:

TP393

王宁, 杨扬, 孟坤, 陈宇, 王磊, 季青. 云计算环境下基于用户体验的成本最优存储策略研究[J]. 电子学报, 2014, 42(1): 20-27.

WANG Ning, YANG Yang, MENG Kun, CHEN Yu, WANG Lei, JI Qing. A Customer Experience-Based Cost Minimization Strategy of Storing Data in Cloud Computing[J]. Acta Electronica Sinica, 2014, 42(1): 20-27.

参考文献

[1] Spillner Josef,Muller Johannes,Schill Alexander.Creating optimal cloud storage systems[J].Future Generation Computer Systems,2013,29(4):1062-1072.
[2] 李建江,崔健,王聃,等.MapReduce并行编程模型研究综述[J].电子学报,2011,39(11):2636-2643. Li Jianjiang,Cui Jian,Wan Dan,et al.Survey of MaqReduce parallel programming model[J].Acta Electronica Sinica,2011,39(11):2636-2643.(in Chinese)
[3] Zhang Dawei,Sun Fuquan,Cheng xu,et al.Research on Hadoop-based enterprise file cloud storage system [A].Proceedings of 2011 3rd International Conference on Awareness Science and Technology,iCAST 2011 [C].Dalian,China:IEEE Computer Society,2011.434-437.
[4] Wei Qingsong,Veeravalli Bharadwaj,Gong Bozhao,et al.CDRM:a cost-effective dynamic replication management scheme for cloud storage cluster [A].Proceedings of 2010 IEEE International Conference on Cluster Computing [C].Heraklion,Crete,Greece: Institute of Electrical and Electronics Engineers Inc,2010.188-196.
[5] He Dian,Liang Ying,Hang Zhi.Replicate distribution method of minimum cost in cloud storage for internet of things [A].Proceedingso of 2011 International Conference on Network Computing and Information Security,NCIS 2011 [C].Guilin,Guangxi,China:IEEE Computer Society,2011.89-92.
[6] Yuan Dong,Yang Yun,Liu Xiao,et al.A cost-effective strategy for intermediate data storage in scientific cloud workflow systems [A].Proceedings of the 2010 IEEE International Symposium on Parallel and Distributed Processing,IPDPS 2010 [C].Atlanta,GA,United States:IEEE Computer Society,2010.1-12.
[7] Li Wenhao,Yang Yun,Yuan Dong.A novel cost-effective dynamic data replication strategy for reliability in cloud data centres [A].Proceedings of 2011 Ninth IEEE International Conference on Dependable,Autonomic and Secure Computing [C].Sydney,Australia:IEEE Computer Society,2011.496-502.
[8] Chang Chiawei,Liu Pangfeng,Wu Janjan.Probability-based cloud storage providers selection algorithms with maximum availability [A].Proceedings of the International Conference on Parallel Processing [C].Pittsburgh,United States:Institute of Electrical and Electronics Engineers Inc,2012.199-208.
[9] Agarwala Sandip,Jadav Divyesh,Bathen Luis A.iCostale:Adaptive cost optimization for storage clouds [A].Proceedings of 2011 IEEE 4th International Conference on Cloud Computing,CLOUD 2011 [C].Washington,DC,United States:IEEE Computer Society,2011.436-443.
[10] Kim Shin-Gyu,Han Hyuck,Eom Hyeonsang,et al.Toward a cost-effective cloud storage service [A].Proceedings of 12th International Conference on Advanced Communication Technology:ICT for Green Growth and Sustainable Development,ICACT 2010 [C].Seoul,Korea: Institute of Electrical and Electronics Engineers Inc,2010.99-102.
[11] 孙大为,常桂然,李凤云,等.一种基于免疫克隆的偏好多维QoS云资源调度优化算法[J].电子学报,2011,39(8):1824-1831. Sun Da-wei,Chang Gui-ran,Li Feng-Yun,et al.Optimizing multi-dimensional QoS cloud resource scheduling by immune clonal with preference[J].Acta Electronica Sinica,2011,39(8):1824-1831.(in Chinese)
[12] 吴吉义,傅建庆,平玲娣,等.一种对等结构的云存储系统研究[J].电子学报,2011,39(5):1100-1107. Wu Ji-yi,Fu Jian-Qing,Ping Ling-di,et al.Study on the P2P cloud storage system[J].Acta Electronica Sinica,2011,39(5):1100-1107.(in Chinese)
[13] 范玉梅,许尔,周汉良.数学规划及其应用[M].北京: 冶金工业出版社,2004.70-80.
[14] 李建敦,彭俊杰,张武.云存储中一种基于布局的虚拟磁盘节能调度方法[J].电子学报, 2012,40(11):2247-2254. Li Jian-dun,Peng Jun-jie,Zhang Wu.A layout-based energy-aware approach for virtual disk scheduling in cloud storage[J].Acta Electronica Sinica,2012,40(11):2247-2254.(in Chinese)

[1]	任正伟, 李雪婷, 王丽娜, 童言, 徐士伟, 丁炜. 云存储中外包数据确定性删除研究综述[J]. 电子学报, 2022, 50(10): 2542-2560.
[2]	侯秀聪, 赖晓平, 曹九稳. 正则化超限学习机的最大分划广义交替方向乘子法[J]. 电子学报, 2021, 49(4): 625-630.
[3]	杨洋. 改进帕累托算法求解超大规模多选择背包问题[J]. 电子学报, 2020, 48(6): 1205-1212.
[4]	陆佳炜, 吴涵, 陈烘, 张元鸣, 梁倩卉, 肖刚. 一种基于动态拓扑的流计算性能优化方法及其在Storm中的实现[J]. 电子学报, 2020, 48(5): 878-890.
[5]	徐超, 陈勇, 葛红美, 何炎祥. 基于大数据的审计技术研究[J]. 电子学报, 2020, 48(5): 1003-1017.
[6]	江小玉, 李贵洋, 周悦, 胡金平, 李慧. 基于负载均衡的纠删码修复流水线[J]. 电子学报, 2020, 48(5): 930-936.
[7]	王亚文, 郭云飞, 刘文彦, 霍树民, 杨超. 面向云存储安全的可自愈拜占庭Quorum系统[J]. 电子学报, 2020, 48(4): 675-681.
[8]	杜瑞忠, 石朋亮, 田俊峰. 基于DDCT表的多副本完整性审计方案[J]. 电子学报, 2020, 48(1): 164-171.
[9]	张鹏, 叶剑, 张鹏. 基于概念漂移检测的大数据交易过程模型优化方法[J]. 电子学报, 2019, 47(7): 1465-1474.
[10]	李建江, 马占宁, 张凯. 一种基于内容分块的层次化去冗优化策略[J]. 电子学报, 2019, 47(5): 1094-1100.
[11]	魏波, 吴瑞峰, 张文生, 吕敬钦, 王莹莹, 夏学文. 具有双重稀疏机制的在线学习算法[J]. 电子学报, 2019, 47(10): 2202-2210.
[12]	金宏桥, 董一鸿, 陈华辉, 钱江波. 分布式环境下基于马尔科夫链的图流三角近似计算[J]. 电子学报, 2018, 46(9): 2139-2148.
[13]	郭艺, 叶剑, 张鹏. 基于偏差约减的大数据交易模型分析与修复方法[J]. 电子学报, 2018, 46(7): 1754-1761.
[14]	尹鑫, 田有亮, 王海龙. 面向大数据定价的委托拍卖方案[J]. 电子学报, 2018, 46(5): 1113-1120.
[15]	丁琳琳, 崔子强, 尹显坤, 王俊陆, 宋宝燕. 基于MapReduce的增广动态Skyline查询处理方法[J]. 电子学报, 2018, 46(5): 1062-1070.

云计算环境下基于用户体验的成本最优存储策略研究

A Customer Experience-Based Cost Minimization Strategy of Storing Data in Cloud Computing

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics