Expert answers on what your Google searches might be missing.
Q. Does a Google search cover the whole Internet? If not, what does it miss?
aims to index all the world’s data. That’s an ambitious goal, and Google does have the most comprehensive coverage of any search engine. But even Google doesn’t cover all trillion-plus pages on the World Wide Web.
Several factors can prevent a site from being indexed while Google’s spiders “crawl” the Web, following links from one site to another. If that site isn’t linked to other sites, requires a password, or contains a do-not-index code, Google bypasses it. Google reindexes frequently updated Web sites several times per day, but changes in more-static sites might not show up in Google results for months.
In addition, Google typically doesn’t index the “deep Web” or “invisible Web”—that is, searchable databases, such as library catalogs and census indexes, and other pages generated in response to queries. For example, most databases on Ancestry.com
and HeritageQuest Online
aren’t indexed on Google; you need to use those sites’ own search engines. Google also misses some downloadable files. But postings on GenForum
are regular Web pages, so Google indexes them. (To limit your Google search to GenForum, include site:genforum.genealogy.com
in your query.)
Updated from the March 2009 Family Tree Magazine