• Proceedings of the Korean Society for Agricultural Machinery Conference
• /
• 2000.11b
• /
• pp.146-153
• /
• 2000

The rubber crawler system for farm machine is composed of driving units such as track rollers, driving sprockets and rubber crawlers. Vibration characteristics of the rubber crawler system varies by driving speed, center of gravity, mass□moment of inertial□location arrangement of track rollers and dynamic parameters such as dynamic spring constant (k) and viscous damping coefficient (c) of a rubber crawler. In general, vibration of the rubber crawler system occurs by reason for mechanical interaction between the rubber crawler and track rollers. Because the dynamic spring constant and viscous damping coefficient vary periodically by mechanical characteristics(deformation characteristics) of the rubber crawler when track rollers drive on the between lugs of the rubber crawler. Therefore, both dynamic parameters k and c were expressed as Fourier series by authors through the shaking test of the rubber crawler and further, vibration characteristics of the rubber crawler system could be simulated analytically. However, actual values of dynamic parameters k and c are different from those obtained by the shaking test because dynamic characteristics of the rubber crawler vary by the effect of variable tension and driving resistance of track rollers. So, actual values of k and c should be identified in the condition of actual driving test. In this study, dynamic parameters such as k and c of the rubber crawler system, which are expressed as Fourier series, were identified using the Gauss-Newton Method. Therefore, validity of identified parameters k and c was discussed through the simulation using experimental data of actual driving test. As a result, in the Fourier series of dynamic parameters of spring constant k and viscous damping coefficient c, excellent parameter convergence and simulation were observed using the Fourier series' zero order and first term of the dynamic model. Furthermore, it was clarified that identification for model parameters which are fitted to actual dynamic motion (vibration) wave of the crawler system was possible by using the time series data observed in vertical and pitching motion of the crawler system.

• Journal of Korea Society of Industrial Information Systems
• /
• v.8 no.3
• /
• pp.12-18
• /
• 2003

A web crawler is used widely in a variety of Internet applications such as search engines. As the Internet continues to grow, high performance web crawlers become more essential. Crawl scheduling which manages the allocation of web pages to each process for downloading documents is one of the important issues. In this paper, we identify issues that are important and challenging in the crawl scheduling. To address the issues, we propose a dynamic owl scheduling framework and subsequently a system architecture for a web crawler subject to the framework. This paper presents the architecture of a web crawler with dynamic scheduling support. The result of our preliminary performance evaluation made to the proposed crawler architecture is also presented.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2002.04b
• /
• pp.1337-1340
• /
• 2002

웹 정보검색에서 웹 문서를 수집하고, 색인을 구축하는 작업에서 Web Crawler 의 역할은 매우 중요하다. 그러나, 웹 문서의 급속한 증가로 인하여 Web Crawler 가 모든 웹 문서를 수집하는 것은 불가능하며, 웹 정보검색의 정확성을 증가시키기 위한 방법으로 특정한 영역의 문서를 수집하는 focused web crawler에 대한 연구가 활발히 진행되어 왔다. 이와 함께, 웹 문서의 link구조를 이용하여 문서의 집합에서 중요한 문서를 찾는 연구들이 많이 진행되었다. 그러나, 기존의 연구에서는 문서의 link 구조에만 초점이 맞추어져 있으며, hypertext 전체의 연결 구조를 알아야 한다는 문제점이 있다. 본 연구에서는 hyperlink의 구조와 hypertext 분류방법을 이용하여 문서에 연결된 다른 문서 중 중요한 문서를 결정하는 방법을 제시하고 이를 이용한 web crawler 를 통하여 특정영역에서 정확한 문서를 수집함을 보였다.

• Journal of Korea Multimedia Society
• /
• v.21 no.2
• /
• pp.199-209
• /
• 2018

Recently, services provided to consumers are increasingly being combined with big data such as low-priced shopping, customized advertisement, and product recommendation. With the increasing importance of big data, the web crawler that collects data from the web has also become important. However, there are two problems with existing web crawlers. First, if the URL is hidden from the link, it can not be accessed by the URL. The second is the inefficiency of fetching more data than the user wants. Therefore, in this paper, through the Casper.js which can control the DOM in the headless brwoser, DOM event is generated by accessing the URL to the hidden link. We also propose an intelligent web crawler system that allows users to make steps to fine-tune both Structured and unstructured data to bring only the data they want. Finally, we show the superiority of the proposed crawler system through the performance evaluation results of the existing web crawler and the proposed web crawler.

• Journal of Digital Contents Society
• /
• v.18 no.5
• /
• pp.933-942
• /
• 2017

In this paper, we propose a web crawler service method for collecting information efficiently about college students and job-seeker's external activities, competition, and scholarship. The proposed web crawler service uses Jsoup tree analysis and Json format data transmission method to avoid problems of duplicated crawling while crawling at high speed. After collecting relevant information for 24 hours, we were able to confirm that the web crawler service is running with an accuracy of 100%. It is expected that the web crawler service can be applied to various web sites in the future to improve the web crawler service.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2011.04a
• /
• pp.1598-1599
• /
• 2011

본 논문에서는 웹페이지의 내용을 분석하고, 또 분석한 정보를 바탕으로 웹페이지의 내용을 활용할 수 있게 해주는 Crawler의 개발에 대해 기술한다. 또한 단순한 정보의 수집뿐만 아니라 Crawler를 통하여 수집된 정보를 활용해 완전한 독립적인 프로그램을 생성할 수 있는 위젯 기능에 대해서도 자세히 기술하고자 한다.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.1
• /
• pp.17-28
• /
• 2017

Recently, a study on data has been actively conducted because the value of the data has become more useful. Web crawler that is program of data collection recently spotlighted because it can take advantage of the various fields. Web crawler can be defined as a tool to analyze the web pages and collects the URL by traversing the web server in an automated manner. For the treatment of Big-data, distributed Web crawler is widely used which is based on the Hadoop MapReduce. But, it is difficult to use and has constraints on the performance. Apache spark that is the In-memory computing platform is an alternative to MapReduce. The search engine which is one of the main purposes of web crawler displays the information you search by keyword gathered by web crawler. If search engines implement a spark-based web crawler instead of traditional MapReduce-based web crawler, it would be a more rapid data collection.

• Journal of Korea Society of Industrial Information Systems
• /
• v.8 no.4
• /
• pp.64-72
• /
• 2003

A Web crawler is an important Internet software technology used in a variety of Internet application software which includes search engines. As Internet continues to grow, implementations of high performance web crawlers are urgently demanded. In this paper, we study how to support dynamic scheduling for a multiprocess-based web crawler. For high peformance, web crawlers are usually based on multiprocess in their implementations. In these systems, crawl scheduling which manages the allocation of web pages to each process for loading is one of the important issues. In this paper, we identify issues which are important and challenging in the crawl scheduling. To address the issue, we propose a dynamic crawl scheduling framework and subsequently a system architecture for a web crawler with dynamic crawl scheduling support. And we analysed the behaviors of Web crawler. Based on the analysis result, we suggest the direction for the design of high performance Web crawler.

• Proceedings of the Korean Society for Agricultural Machinery Conference
• /
• 1993.10a
• /
• pp.1202-1211
• /
• 1993

The authors propose in this research the equation to calculate the velocities, accelerations and penetration angles of the locus of lug motion for the rubber crawler mechanism. In these equations with some values of factors, motion characteristics of all points or faces of the lug in the front-half a rubber crawler. After that we also consider the reactionary force from the soil to the lug by computing the removed soil area for the purpose of understanding a relation between crawler lug and the soil in the terms of estimating trafficability.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.10
• /
• pp.4856-4872
• /
• 2018

The mobile internet has grown ubiquitous across the globe with the widespread use of smart devices. However, the designs of modern mobile operating systems and their applications limit content retrieval with mobile applications. The mobile internet is not as accessible as the traditional web, having more man-made restrictions and lacking a unified approach for crawling and content retrieval. In this study, we propose an automatic and scalable mobile application content crawler, which can recognize the interaction paths of mobile applications, representing them as interaction graphs and automatically collecting content according to the graphs in a parallel manner. The crawler was verified by retrieving content from 50 non-game applications from the Google Play Store using the Android platform. The experiment showed the efficiency and scalability potential of our crawler for large-scale mobile internet content retrieval.