Neural Net Agent for Distributed Information Retrieval

분산 정보 검색을 위한 신경망 에이전트

  • Published : 2001.10.01


Since documents on the Web are naturally partitioned into may document database, the efficient information retrieval process requires identifying the document database that are most likely to provide relevant documents to the query and then querying the identified document database. We propose a neural net agent approach to such an efficient information retrieval. First, we present a neural net agent that learns about underlying document database using the relevance feedbacks obtained from many retrieval experiences. For a given query, the neural net agent, which is sufficiently trained on the basis of the BPN learning mechanism, discovers the document database associated with the relevant documents and retrieves those documents effectively. In the experiment, we introduce a neural net agent based information retrieval system and evaluate its performance by comparing experimental results to those of the conventional well-known approaches.

웹과 같은 분산 정보 검색 환경에서 문서들의 많은 문서 데이터 베이스들에 자연스럽게 분할되어서 존재한다. 그러므로 이러한문서들의효율적인 검색을 위해서는 먼저 질의에 관련되는 문서들을 제공할것으로 판단되는 문서 데이타베이스를 찾아내고 다음으로 그 문서 데이타베이스에 질의를 줌으로써 분산 정보 검색을 수행해야한다. 본 논문에서는 이러한 효율적인 분산 정보 검색을 위한 신경망 에이전트를 제안한다. 신경망 에이전트는 질의 검색 예제들을 통하여 얻어진 질의에 대한 관련도 피드백 정보에 기반하여 역전파 알고리즘으로 분산 정보 검색 지식을 학습한다. 충분히 학습한 후의 신경망 에이전트는 주어진 질의에 대하여 관련 문서 데이타베이스들을 찾아내고 그 문서 데이타베이스들로부터 관련되는 문서들을 검색한다. 실험에서 제안된 신경망 에이전트 시스템을 구현하여 정보 검색 성능을 널리 알려진 기존의 분산 정보 검색 기법을 사용했을때 비교함으로써 신경망 에이전트의 유용성을 예증한다.



  1. D. Clifford Neuman, 'The Prospero File System: A global file system based on the Virtual System model,' Computer Systems, 5(4), 1992. [1]
  2. Michael F. Schwarz, Alan Emtage, Brewster Kahle, and B. Clifford Neuman, 'A Comparison of INTERNET resource discovery approaches,' Computer Systems, 5(4), 1992
  3. B. Kahle and A. Medlar, 'An information system for corporate users: Wide Area Information Servers,' Technical Report TMC199, Thinking Machines Corporation, 1991
  4. G. Salton, The SMART Retrieval System-Experiments in Automatic Document Processing, Prentice-Hall, Inc., Englewood Cliffs NJ, 1971
  5. G. Salton and M. McGill, Introduction to Modem Information Retrieval, MacGraw-Hill, New York NY, 1983
  6. A. Howe and D. Dreilinger, 'Savvy Search: A Meta-Search Engine that Learns Which Search Engines to Query,' AI Magazine, 18(2), 1997
  7. L. Gravano, H. Garcia-Molina, and A. Tomasic, 'The Effectiveness of GlOSS for the Text-Database Discovery Problem,' in Proceedings of ACM SIGMOD, 1994
  8. L. Gravano and H. Garcia-Molina, 'Generalizing GlOSS to Vector-Space Databases and Broker Hierarchies,' in Proceedings of VLDB, 1995
  9. Yong S. Choi and Suk I. Yoo, 'Neural Network Based Web Information Agent,' in Proceedings of ACM CIKM'98 Workshop on Web Information and Data Management, November 1998
  10. Yong S. Choi and Suk I. Yoo, 'Neural Net Agent for Discovering Text Databases on the Web,' in Proceedings of International Conference on Advances in Databases and Information Systems, September 1999
  11. Ian H. Witten, Alistair Moffat, and Timothy C. Bell, Managing Gigabytes: Compressing and Indexing Documents and Images, Von Nostrand Reinhold, New York, 1994
  12. I. Biederman, On the Semantics of a Glance at a Scene, Perceptual Organization, Hillsdale, New-Jersey, Lawrence Erlbaum, 1981
  13. J. A. Freeman and D. M. Skapura, Neural Networks Algorithms, Applications, and programming Techniques, Addison-Wesley, MA, 1992
  14. B.A. LaMacchia, Internet Fish, PhD thesis, MIT, MA, 1996
  15. G. Tesauro and H. Janssens, 'Scaling relationships in back-propagation learning,' Complex Systems, Vol. 6, 1988
  16. M. Minsky and S. Papert, Perceptrons, MlT Press, Cambridge, MA, 1969
  17. Yong S. Choi and Suk I. Yoo, 'Multi-agent Learning Approach to WWW Information Retrieval using Neural Network,' in Proceedings of ACM International Conference on Intelligent User Interfaces, January 1999
  18. Yong S. Choi, Suk I. Yoo, and Jaeho Lee, 'Hierarchically Organized Neural Net. Agents for Distributed Web Information Retrieval,' in Proceedings of 23rd International IEEE Computer Software and Applications Conference, October 1999