Researchers spend a great deal of time combing the Internet for relevant information. This may involve a number of queries to various search engines resident on the Internet. The initial search result will yield information spanning several contexts, leaving the user to either laboriously click through all the results, or adjust the initial query and try, try, again.
If one thinks about the task at hand, a generic searching process can be automated. Once we find a relevant data source, usually one of three things happens. Either the search terminates, the user follows a link embedded in the page, or the user can find all pages pointing to the relevant page if the search engine happens to support this functionality, like Google. Two out of the three functions are traditional functions of web spiders.
Our project will draw upon a body of open source programs to construct a prototype application that can help researchers gather information.
Our project seeks to automate user searches by:
Programs that are used in this project for development include: