DOI QR코드

DOI QR Code

Efficient URL Prioritizing Method for Web Crawlers

웹 크롤러를 위한 효율적인 URL 우선순위 할당 기법

  • Md. Hijbul Alam (Division of Computer and Communication Engineering, Korea University) ;
  • Jong-Woo Ha (Division of Computer and Communication Engineering, Korea University) ;
  • Yoon-Ho Cho (Division of Computer and Communication Engineering, Korea University) ;
  • SangKeun Lee (Division of Computer and Communication Engineering, Korea University)
  • ;
  • 하종우 (고려대학교 정보통신대학 컴퓨터통신공학부) ;
  • 조윤호 (고려대학교 정보통신대학 컴퓨터통신공학부) ;
  • 이상근 (고려대학교 정보통신대학 컴퓨터통신공학부)
  • Published : 2008.11.14

Abstract

With the amazing growth of web faster important page crawlers poses great challenge. In this research we proposed fractional PageRank, a variation of PageRank computed during crawl that can able to prioritize the downloading order. Experimental results shows that it outperforms the prior crawler in terms of running time yet provide a well download ordering.

Keywords

Acknowledgement

이 연구에 참여한 연구자는 '2단계 BK21'의 지원비를 받았음