Dynamic Refresh Strategy for Crawler in Search Engine

GAO Kai

您当前的位置：

首页 >

文章列表页 >

Dynamic Refresh Strategy for Crawler in Search Engine

更新时间：2025-07-16

- Dynamic Refresh Strategy for Crawler in Search Engine
- Acta Electronica Sinica Vol. 35, Issue 10, Pages: 1984-1988(2007)
- 作者机构：
  
  1. 河北科技大学信息科学与工程学院计算机科学与技术系,河北,石家庄,050054
  2. 河北科技大学信息科学与工程学院计算机科学与技术系河北石家庄,050054
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP393
- Published：2007
- 稿件说明：
移动端阅览
GAO Kai. Dynamic Refresh Strategy for Crawler in Search Engine[J]. Acta Electronica Sinica, 2007, 35(10): 1984-1988.
DOI：

GAO Kai. Dynamic Refresh Strategy for Crawler in Search Engine[J]. Acta Electronica Sinica, 2007, 35(10): 1984-1988. DOI：

摘要

为了能及时采集到有关网页信息

搜索引擎应根据相应网站及其更新速度

动态调整其信息采集的频度.本文就模型化网页更新过程以及根据相关性动态调整搜索引擎的信息采集频度进行了探讨.一方面使用泊松过程来描述网页更新并分析了搜索引擎如何有效完成信息采集;另一方面采用基于网页从属关系和内容分析的相关性来调节该过程

使得在进行信息采集与数据更新时的针对性更强.实验表明了该方法的有效性.

Abstract

As for a search engine

keeping up with the evolving Web is necessary.We concern about modeling on an effective Web page collecting policy and propose an adaptive refresh strategy based on the relevance

which is used to adjust the process.On one hand

we think the refresh behavior follows the properties of the Poisson process and analyze the strategy on how to crawl the Web effectively.Further

the relevance is on the basis of the affiliation detecting and the contents analysis.It is used to adjust the process.This makes the process more targeted.The experimental results validate the feasibility of the approach.

关键词

Keywords

references

Views

1140

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Measurement Method of Search Engine Retrieve Performance Based User Path Model

A Study on the Ranking Method of Search Engines

An Evaluation Model on Information Coverage of Search Engines

Web User Interest Mining Based on Ontology and Patterns

A Blocking Inverted File Structure for Search Engine

Related Author

YUAN Yu-yu

LUO Xue-chao

HAN Li-xin

MENG Tao

YAN Hong-fei

LI Xiao-ming

SU Xue-yang

ZUO Wan-li

Related Institution

Department of Computer Science,Peking University

Department of Research Center,SOHU

Department of Computer SciencePeking UniversityBeijing 100871China

Department of Research CenterSOHUBeijing 100084China

河海大学计算机科学与技术系

⁰