Community Research and Development Information Service - CORDIS

Looking for the 'invisible Web'

There are vast expanses of the Web that are invisible to most search engines, and experts claim this 'invisible Web' is expanding faster than the Web itself. However a new form of search service promises to boldly go where no search engine has gone before.

The 'invisible Web'...
There are vast expanses of the Web that are invisible to most search engines, and experts claim this 'invisible Web' is expanding faster than the Web itself. However a new form of search service promises to boldly go where no search engine has gone before.

The 'invisible Web' is made up of information stored in databases, which most search engines pass by without properly recognising. The information stored in these databases is locked away from search engines, who can record the address, but they don't recognise any of the contents.

A new search service holds the key, and has unlocked the door to thousands of searchable, interactive databases.

An index of search databases with links to more than seven thousand speciality search resources has been developed by Lycos and Intelliseek, maker of BullsEye desktop search utility. Browsers can choose to browse a set of listings, or alternatively Lycos will suggest databases with their own search engines.

Similar search engines, such as formerly Isleuth.com which unveiled a new website on 2 August, are also available. It lets users search multiple engines, Web directories and news databases simultaneously and President and Chief Executive Pat DeMicco claims it is the 'most comprehensive search on the Internet'.

'By bringing together several unique and powerful 'Big' components in one easy to use website, TheBigHub.com will ensure that Internet users never need to go anywhere else', he said.

However 'Search Engine Watch' editor Danny Sullivan claimed Lycos Invisible Web catalogue is better.

'The great thing about Lycos is that it has integrated lots of speciality search engines into its own search. It's not really going into the search engines themselves. It has just catalogued the information, and allows you to access a directory of sites.'

Mr Sullivan also pointed for the need for a common standard across all search engines. He has initiated the Search Engine Standards project to encourage search engines to support basic standard functions, saving site owners the hassle of listing their sites in several different ways.

'We have had a significant amount of interest in this project, and currently we are working on finding common ground for search engine developers, rather than competing on most basic functionality', he said.
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top