基于数据流的网页内容分析技术研究

doi:10.3969/j.issn.0372-2112.2013.04.021

电子学报 ›› 2013, Vol. 41 ›› Issue (4): 751-756.DOI: 10.3969/j.issn.0372-2112.2013.04.021

基于数据流的网页内容分析技术研究

王佰玲^1,2, 曲芸¹, 张永铮³, 田志宏¹

1. 哈尔滨工业大学计算机科学与技术学院,黑龙江哈尔滨 150001;
2. 北京大学信息科学与技术学院,北京 100871;
3. 中国科学院计算技术研究所,北京 100190

收稿日期:2011-09-10 修回日期:2012-09-20 出版日期:2013-04-25
- 作者简介:
- 王佰玲 博士,副教授,硕士生导师;哈尔滨工业大学(威海)网络技术研究所负责人;哈尔滨工业大学(威海)计算机学院院长助理;北京大学信息科学与技术学院博士后;北京邮电大学兼职副教授;计算机协会高级会员.研究领域包括计算机网络安全、信息内容安全、网络攻防对抗、信息穿透对抗、网络舆情技术等. E-mail:wbl@hit.edu.cn 张永铮 博士,副研究员,硕士生导师.研究领域包括计算机网络安全、信息内容安全、物联网技术等. 田志宏 博士,副研究员,硕士生导师.研究领域包括信息内容安全、网络信息搜索技术等.
- 基金资助:
- 国家自然科学基金 (No.61170262)

Research on Network-Traffic Based Web Traffic Computing Technology

WANG Bai-ling^1,2, QU Yun¹, ZHANG Yong-zheng³, TIAN Zhi-hong¹

1. School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China;
2. School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China;
3. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China

Received:2011-09-10 Revised:2012-09-20 Online:2013-04-25 Published:2013-04-25
- Supported by:
- National Natural Science Foundation of China (No.61170262)

摘要/Abstract

摘要： 提出针对网络数据流中活跃信息进行话题相关数据采集与分析方法.首先给出面向论坛话题的定义;然后对网络数据流进行分析、对用户访问行为进行分类;并给出基于数据流的用户行为识别方法及话题相关数据抽取、存储算法;最后给出实验分析,结果表明,所提出的基于数据流的论坛话题数据采集方法能够很好地反映用户行为,并对基于数据流的网络舆情热点话题发现、突发事件检测与实时跟踪等应用提供有利的数据资源.

关键词: 网络舆情, 热点话题, 突发事件, 网络数据流

Abstract: In this paper,a network-traffic based topic extracting and analyzing method is introduced.The new topic definition for web2.0 and the classification of user behavior is given;the detecting method of user behavior,topic extracting method,and data storage algorithm is also proposed.At last,a prototype of topic collector based on network traffic is implemented;the testing results show that the user behavior and the hot topic can be collected and detected effectively and correctly,and the new method provides a new data channel for analyzing public opinion.

Key words: public opinion, hot topic, emergent event, network traffic

中图分类号:

TP301

王佰玲, 曲芸, 张永铮, 等. 基于数据流的网页内容分析技术研究[J]. 电子学报, 2013, 41(4): 751-756.

WANG Bai-ling, QU Yun, ZHANG Yong-zheng, et al. Research on Network-Traffic Based Web Traffic Computing Technology[J]. Acta Electronica Sinica, 2013, 41(4): 751-756.

参考文献

[1] Gupta S,Kaiser G,Stolfo S.Extracting context to improve accuracy for HTML conten t extraction[A].Ellis A,Tatsuya H,eds Proc of the 14th Intl Conf.on World Wide Web—Special Interest Tracks and Posters[C].New York:ACM Press,2005.1114-1115 .
[2] 王佰玲,方滨兴,云晓春.传统报文捕获平台的性能影响因素分析[J].计算机工程与应用, 2003,39(22):151-152. WANG B L,FANG B X,YUN X C.The analysis of the performance factor in traditional packet capture plat[J].Computer Engineering and Applications,2003,39(22):151-1 52.(in Chinese)
[3] 王佰玲,方滨兴,云晓春.零拷贝报文捕获平台的研究与实现[J].计算机学报,2005,28(1):4 6-52. WANG B L,FANG B X,YUN X C.The study and implementation of zero-copy packet capt ure platform[J].Chinese Journal of Computers,2005,28(1):46-52.(in Chinese)
[4] 王佰玲.基于良性蠕虫的网络蠕虫主动遏制技术研究[D].黑龙江哈尔滨:哈尔滨工业大学. 2006.
[5] 王佰玲,田志宏,张永铮.奇异值分解算法优化[J].电子学报,2010,38(10):2234-2239. Wang B L,Tian Z H,Zhang Y Z.Optimization of singular vector decomposition algori thm[J].Acta Electronica Sinica,2010,38(10):2234-2239.(in Chinese)
[6] 欧阳震诤,罗建书,胡东敏,吴泉源.一种不平衡数据流集成分类模型[J].电子学报,2010,38 (1):184-189. OUYANG Z Z,LUO J S,HU D M.An ensemble classifier framework for mining imbalanced data streams[J].Acta Electronica Sinica,2010,38(1):184-189.(in Chinese)
[7] 詹英,吴春明,王宝军.一种与缓冲区紧耦合的环形循环滑动窗口的数据流抽取算法[J].电子学报,2011,39(4):894-898. ZHAN Y,WU C,WANG B.An algorithm for data stream sampling based on ring circular sliding window tightly-coupled with buffer[J].Acta Electronica Sinica,2011,39 (4):894-898.(in Chinese)
[8] MacDonald J.Versioned file archiving,compression,and distribution[OL].http://w ww.cs.berkeley.edu/~jmacd/.UC Berkeley,1999.
[9] Golaxy中科天玑[OL].http://www.golaxy.cn/,2009.
[10] 李连霞.基于多特征的HTML网页内容提取的研究[D].山东济南:山东大学,2008.
[11] 林昌平,郑皎凌.基于DOM规范的网页分析技术研究[J].成都信息工程学院学报,2007,(S1): 113-117.
[12] TSE SourceCode[OL].http://sewm.pku.edu.cn/src/TSE/,2009.
[13] Gomes D,Santos AL,Silva MJ.Webstore:A manager for incremental storage of content s[R].Technical Report,DI/FCUL TR 04-15,Lisbon:University of Lisbon,2004.
[14] Sekiguchi Y,Kawashima H,Okuda H,Oku M.Topic detection from Blog documents using users'interests[A].Aberer K,Hara T,eds Proc of the 7th Intl Conf on Mobile Da ta Management(MDM 2006)[C].Washington:IEEE Computer Society,2006.108-111.
[15] Wang XY,Xiong FY,Ling B,Zhou A.A similarity-based algorithm for topic explorati on and distillation[J].Journal of Software,2003,14(9):1578-585.
[16] McCown F.Dynamic web file format transformations with grace[A].Proc of the 5th Intl Web Archiving Workshop and Digital Preservation[C].2005.22-23.
[17] Lampos C,Eirinaki M,Jevtuchova D,Vazirgiannis M.Archiving the Greek Web[A].Pro c.of the 4th Int'l Web Archiving Workshop[C].2004.

基于数据流的网页内容分析技术研究

Research on Network-Traffic Based Web Traffic Computing Technology

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics