What does the term invisible Web refer to?

Answer 1

The "invisible web" is the part of the web that search engines do not have direct access to. It is a vast amount of databases estimated to be 500 times bigger than the visible web that is accessible to search engines.

Answer 2

The 'Deep' web is the part of the web that is not normally accessable to search engines, this is content that is restricted to subscription-only services. The services are normally academic in nature. A web site will allow a web crawler or spiderbot to access specific part of the website, such as the home page, about us pages and contact pages. However academic papers and other content is either on a pay per view or just does not need to be indexed by a search engine (such as administration sections of an ecommerce site). It is this content that is described as being 'The Deep web' because very little of it is actually 'seen' by web crawlers.

A file held within the 'root' web directory called a robots.txt file contains information on what a web crawler can actually index to the search engine it is crawling for.

Hope this helps to answer your questoion.

Ref:

The Deep Web:

http://pipl.com/help/deep-web/

Robots.txt:

http://www.robotstxt.org/

Answer 3

Also referred to as the deep Web, the term refers to either Web pages that cannot be indexed by a typical search engine or Web pages that a search engine purposely does not index, rendering the data "invisible" to the general user.