This proposal is in the area of automated web data extraction and web data management. The aim of our project is to provide the logical, methodological, and algorithmic foundations for the knowledge-based extraction of structured data from web sites belonging to specific domains, such as estate agents, restaurants, travel agencies, car dealers, and so on. One core part of this will be a comprehensive multi-dimensional logical data model that will be used to simultaneously represent both the content of a large website, its structure, inferred user-interaction patterns and all meta-information and knowledge (factual and rule-based) that is necessary to automatically perform the desired extraction tasks. I envision that, based on these new foundations, we will be able to build extremely powerful systems that autonomously explore websites of a given domain, understand their structure and extract and output richly structured data in formats such as XML or RDF. We aim at systems that take as input a URL of a website in a given domain, automatically explore this site and deliver as output a structured data set containing all the relevant information present on that site. As an example, imagine a system specialized in the real-estate domain, that receives as input the URL of any real-estate agent, explores the site automatically and outputs richly structured records of all properties that are currently advertised for sale or for rent on the many web pages of this site. We plan to develop and implement at least two such systems for two different domains, including the one mentioned. The breakthrough in automatic data extraction that we are striving for would enable a quantum leap for two interrelated technologies which are the hottest next topics in web search: vertical search, that is, web search in specialized domains, and object search, that is, the search for web data objects rather than web pages.
Call for proposal
See other projects for this call