Search Engine History: From Early Databases to AI

Search engine history reaches back to experiments with remote computers in the 1960s. By the time Yahoo appeared in 1994, people were already searching online databases, looking up files with Archie, browsing Gopher menus, and sending web crawlers out to find pages.

The equipment changed from room-sized computers to browser tabs. Along the way, search acquired a cast of characters that sounds like a Saturday morning cartoon: Archie, Veronica, a gopher, a crawler, and eventually a butler. Each solved a different part of the problem of finding something online.

Key takeaways

• Online document search predates the web. Computer History Museum records describe real-time full-text retrieval experiments in 1963.
• Archie’s development began at McGill in 1989. Gopher, W3Catalog, the Wanderer, and JumpStation followed before Yahoo’s 1994 directory.
• Search brands often shared underlying technology. Yahoo used Google results, later acquired Inktomi, and subsequently signed a search agreement with Microsoft.
• Modern market-share charts measure a particular activity. The comparison tool below shows search referrals by device; AI citations and questions answered without a click need separate measures.

Before the web: searching cards, microfilm, and remote computers

Punched cards and microfilm provided early ways to select matching records. In the 1920s, Emmanuel Goldberg built a machine for retrieving information from microfilm. Computer History Museum, the Bourne collection.

By 1963, Charles Bourne and Lynn Chaitin were working on an experiment that let a user search document text on a remote computer in real time. The query could use Boolean operators such as AND and OR. Full-text online search reached commercial services by the end of the decade.

Services such as Dialog then sold access to organized databases. A researcher could narrow a question over several exchanges with the system. Access charges and database subscriptions made these tools a substantial expense, especially compared with the free search boxes that later filled the web.

1989–1993: Archie finds files, Gopher organizes documents

Archie starts with a system administrator’s chore

At McGill, Alan Emtage had to find free software for the people using the university’s computers. He automated the hunt with Archie. Both McGill, Archie’s origins and the Internet Hall of Fame, Alan Emtage date his initial implementation to 1989.

Archie let people look up filenames available on anonymous FTP servers. A search helped locate a file and the machine hosting it, which saved you from checking one server after another. The contemporary Internet resource guide, RFC 1580 (1994) documents how people used it.

Picture trying to find a utility without an app store, a download site, or a web search engine. Knowing which machine held the file was already a useful answer.

Gopher and Veronica give the network a menu

The University of Minnesota’s Gopher organized distributed documents into menus. Its protocol documentation traces the original specification to 1991. You could browse a hierarchy of subjects and retrieve a document from another server without having to navigate that server yourself. Gopher protocol, RFC 1436.

As Gopher servers multiplied, Veronica provided keyword searches across the titles of their menu items. The University of Nevada developed it, and Minnesota’s 1994 newsletter explains how to choose a server, enter a query, and open the returned items. University of Minnesota, June 1994 newsletter.

The result you searched was often a label attached to a document. That distinction helps explain why the later ability to search a whole web page mattered: the useful phrase might be buried in the text.

WAIS searches distributed databases

Wide Area Information Servers, or WAIS, offered another route to documents on networked computers. Developed at Thinking Machines, it let users search databases through client software. The 1994 internet resource guide describes choosing sources and refining a search using relevant documents. Internet resource guide, RFC 1580 (1994).

The web creates a new collection to search

Tim Berners-Lee proposed the World Wide Web at CERN in 1989. CERN put the web software into the public domain on April 30, 1993, helping people build and connect their own sites. Those linked pages gave search developers a new kind of collection to explore. CERN, the birth of the web.

1993–1994: W3Catalog, the Wanderer, and JumpStation

Several projects tackled different parts of web discovery in 1993. One gathered existing resource lists, another measured the web, and another combined a crawler with a searchable database. That makes “the first search engine” a question about what the system actually did.

W3Catalog reuses the lists people already made

Oscar Nierstrasz’s W3Catalog began on September 2, 1993, at the University of Geneva. It collected and reformatted existing lists of web resources into a searchable catalog. People maintained the source lists; software made it easier to query them. Oscar Nierstrasz, W3 Catalog history.

Nierstrasz’s account also describes the maintenance burden. Resource lists changed format, software needed attention, and larger services arrived. W3Catalog stopped on November 8, 1996, with AltaVista and HotBot among the alternatives named in its farewell.

The Wanderer measures; JumpStation retrieves

Matthew Gray’s World Wide Web Wanderer began in June 1993 with the aim of measuring web growth. Its archived robot record makes that purpose explicit. A crawler could visit pages for counting and research as well as for search. World Wide Web Wanderer, archived robot registry.

Jonathon Fletcher’s JumpStation ran at the University of Stirling in December 1993. It followed links, stored information from websites, and let visitors query that database through a web page. Stirling records its first run on December 12 and its announcement on December 21. University of Stirling, JumpStation history.

Fletcher said the announcement attracted enough visitors to overload the university’s servers. The project eventually struggled to keep pace with the web. Search had become useful enough to create its own capacity problem.

1994–1998: the search-engine names start multiplying

Yahoo sorts, WebCrawler reads, and Lycos grows

Yahoo began in 1994 as Jerry and David’s Guide to the World Wide Web. Jerry Yang and David Filo organized sites into categories, giving visitors a directory they could browse by subject. Computer History Museum’s Yahoo history.

That same year, Brian Pinkerton developed WebCrawler at the University of Washington. It searched the full text of web pages, expanding the words a query could match beyond short descriptions or labels. University of Washington, WebCrawler’s origins.

Michael Mauldin’s Lycos grew out of Carnegie Mellon. In an August 26, 1994 announcement, he reported a production database of 634,000 documents and a dedicated workstation for searches. That number describes the index, not the audience. Michael Mauldin, August 26, 1994 Lycos announcement.

Infoseek’s early service searched magazine articles

An October 3, 1994 review by Adam Engst describes using Infoseek to search part of Ziff-Davis’s Computer Select collection. He tested it through a web browser and explained that the free trial would give way to commercial accounts. Adam Engst’s October 3, 1994 Infoseek review.

That is a useful glimpse of the period: a web interface could search a specific publication collection, while another service searched pages across the web. Two sites with search boxes could offer very different things.

Excite, AltaVista, and HotBot compete on retrieval

Excite reached the internet market in October 1995. Its prospectus described concept-based searching, intended to retrieve relevant documents even when they did not contain the exact query terms. Excite prospectus excerpts in court exhibits.

Digital Equipment Corporation created AltaVista in 1995. It belongs alongside WebCrawler, Lycos, and Excite in the generation of services that made searching a growing web a routine activity. DEC’s records at Computer History Museum.

Inktomi approached the scale problem with clusters of networked computers. Eric Brewer and Paul Gauthier launched the company in 1996 after developing the underlying system at Berkeley. HotBot used Inktomi’s engine. UC Berkeley, Inktomi’s search infrastructure; UC Berkeley, Inktomi and HotBot technical account.

Google adds links to the ranking problem

Larry Page and Sergey Brin’s Stanford project used links between pages to help judge their importance. It was first called Backrub, then Google, and became a company in 1998. Google’s company history.

A page’s words could tell an engine what it discussed. Links supplied another kind of evidence about which pages deserved attention. Anyone who has spent an afternoon trying to earn one good link has inherited part of that history.

1997–2009: language, advertising, and the engine behind the logo

Search develops around different languages

Yandex-Web launched on September 23, 1997. Its history describes Russian morphology, word proximity, and document ranking as parts of the original system. Handling the forms words take in a language was a product feature. Yandex, company history.

South Korea’s Naver launched its search portal in June 1999. Its company chronology places the portal’s debut alongside the formation of Navercom. Naver, company milestones.

Baidu formed in 2000 and launched search services at Baidu.com in September 2001, according to its IPO prospectus. Keeping those dates separate prevents a company’s incorporation from being mistaken for a website’s launch. Baidu, 2005 IPO prospectus.

Country and language therefore belong beside any claim about search popularity. A worldwide total can conceal a very different set of choices in a particular market.

Search becomes something other websites buy

Yahoo chose Google as its default search-results provider on June 26, 2000. Visitors could use Yahoo while Google supplied the results. The joint announcement described the combination of Yahoo’s directory and Google’s search technology.

Google launched self-service AdWords on October 23, 2000. Businesses could buy text advertisements matched to search terms and monitor campaigns themselves. Search queries now gave advertisers a direct way to reach someone expressing a specific need. Google, October 23, 2000 AdWords launch.

Yahoo acquired Inktomi in 2003. Six years later, Yahoo and Microsoft announced an agreement under which Microsoft would power Yahoo search. Those changes make a history of providers as useful as a history of homepages. UC Berkeley, Inktomi company record; Microsoft and Yahoo, July 29, 2009 agreement.

MSN Search had started in 1998, well before the Bing name. Microsoft’s 2004 announcement described a preview of its own search technology; the 2009 Bing launch was another stage in that longer effort. Microsoft, MSN Search history and 2004 technology preview; Bing announcement.

2008–2026: privacy, answers, and AI enter the search box

Privacy and an independent index become selling points

DuckDuckGo dates its privacy-focused search work to 2008. Its appeal gave people another reason to choose a search service: how it handled their information. DuckDuckGo, company history.

Brave Search entered public beta on June 22, 2021 with its own index, while using other providers for some results. In April 2023, Brave announced that it had removed Bing API calls from its web-search results. Brave Search, June 22, 2021 beta launch; Brave, April 27, 2023 index announcement.

Kagi launched on June 1, 2022 with a paid search service. Its approach put the user’s subscription at the center of the business model, offering another way to fund search alongside advertising. Kagi, launch history and product archive.

The distinction is useful when comparing alternatives today. A service can change the privacy policy, interface, or ranking while drawing some results from another provider. Building and maintaining an index adds a separate layer of work.

Answers appear directly on the results page

Google introduced the Knowledge Graph in May 2012, connecting information about people, places, and other entities. Search could display facts about a subject alongside links to pages. Google, May 16, 2012 Knowledge Graph announcement.

That gave a visitor a way to finish some lookups on the results page. A website owner could still appear in search while receiving fewer reasons for someone to click. Gridlok’s guide to rising impressions and falling clicks explains how to investigate that pattern.

AI adds a conversation and generated summaries

Microsoft announced its AI-powered Bing and Edge experience on February 7, 2023, adding chat and generated responses to its search offering. Google followed with the US rollout of AI Overviews in May 2024. Microsoft, February 7, 2023 AI search announcement; Google’s AI Overviews announcement.

ChatGPT search launched on October 31, 2024 with web-connected answers and source links. OpenAI’s announcement described third-party search providers and partner content as inputs to that experience. OpenAI, October 31, 2024 ChatGPT search launch.

The old provider question still applies. The product receiving your question, the service finding documents, and the publisher supplying an answer can be three different organizations.

Ask.com’s closure on May 1, 2026 gives the story a particularly memorable ending for the old butler. Questions in everyday language had been part of its identity long before today’s AI chat interfaces. Its farewell page records the closing date.

You can link directly to the early-search timeline, the 1990s engine table, the provider history, or the answer-search timeline. Each keeps its dates and sources next to the event.

When did Google pull away from Yahoo?

Pew’s surveys give us two clear snapshots. Among US search users, Google was the most-used engine for 47% in 2004 and 83% in 2012. Yahoo moved from 26% to 6%. Pew’s 2012 report.

Source: Pew Research Center. Survey preference among US search users; bars use a 0–100% scale. Percentage-point changes are calculated from the reported figures.

By 2012, the gap was 77 percentage points. For this group, choosing a search engine had become a much less competitive habit. Link to the comparison table.

Which search engines get the clicks now? Try the device comparison

Switch between desktop and mobile and watch Bing’s share change. The source, month, and geography stay fixed: Statcounter, August 2026, worldwide. This gives you a useful comparison without changing several things at once.

Source: Statcounter, all platforms, desktop, and mobile. Selected engines are shown; remaining engines account for the balance. Link to this comparison tool.

Google moves the other way: 86.05% on desktop and 95.59% on mobile. For a site owner, the useful follow-up is to split your own search referrals by device and country before choosing where to spend time.

A worldwide chart is a starting point. Your website’s audience, conversions, and search queries tell you which engines deserve closer attention.

Does your search traffic look different from these charts? Ask Jim to review your search traffic by engine, device, and country.

How does AI change the search-engine scorecard?

Google’s May 2024 announcement put AI Overviews inside its existing search product. An answer can now contain a summary and source links before a visitor reaches a website. Google’s rollout announcement marks that change in the timeline.

The questions become more specific. Did an engine send a visitor? Did an answer name your company? Did it cite your page? Keep those results in separate columns so you can act on them.

Statcounter counts referrals to participating websites. A search that ends without a click produces no referral in that measurement. Keep that definition beside the chart when using it to discuss AI answers.

Gridlok’s study of 407 AI Overview citations looks at the pages appearing as sources. The impressions-up, clicks-down article covers what happens when being seen produces fewer visits.

If your own pages are difficult to read or crawl, fix that before chasing the next search interface. Our HTML-shell and SEO article shows the problem developers need to check. The name on the search box can change; the page still has to deliver something worth opening.

Sources and calculations

The timelines draw on inventor accounts, contemporary documents, university archives, company announcements, and Computer History Museum records. Historical preference figures come from Pew’s US surveys. Current referral figures use Statcounter’s August 2026 worldwide snapshots, checked September 22, 2026.

The charts retain each source’s population and metric. Calculated changes use percentage-point subtraction; the Bing device comparison divides 8.84 by 0.60 and rounds to one decimal place. The on-page tool runs from the fixed figures shown in the table.

Frequently asked questions

References

1. Computer History Museum, Fairchildren (Yahoo origin)
2. Computer History Museum, Digital Equipment Corporation records
3. Google, company history
4. Google and Yahoo, search-provider announcement, June 26, 2000
5. Microsoft, Bing announcement, May 28, 2009
6. Google, AI Overviews rollout, May 14, 2024
7. Ask.com, service closure, May 1, 2026
8. Pew Research Center, Search Engine Use 2012, March 9, 2012
9. Statcounter, worldwide search-engine share, August 2026
10. Statcounter, desktop search-engine share, August 2026
11. Statcounter, mobile search-engine share, August 2026
12. Statcounter, methodology and search-referral definitions
13. Kagi, launch history and product archive
14. Naver, company milestones
15. Computer History Museum, the Bourne collection
16. Internet Hall of Fame, Alan Emtage
17. McGill, Archie’s origins
18. Gopher protocol, RFC 1436
19. University of Minnesota, June 1994 newsletter
20. Internet resource guide, RFC 1580 (1994)
21. CERN, the birth of the web
22. World Wide Web Wanderer, archived robot registry
23. Oscar Nierstrasz, W3 Catalog history
24. University of Stirling, JumpStation history
25. University of Washington, WebCrawler’s origins
26. Michael Mauldin, August 26, 1994 Lycos announcement
27. Adam Engst’s October 3, 1994 Infoseek review
28. Excite prospectus excerpts in court exhibits
29. UC Berkeley, Inktomi’s search infrastructure
30. UC Berkeley, Inktomi and HotBot technical account
31. UC Berkeley, Inktomi company record
32. Yandex, company history
33. Baidu, 2005 IPO prospectus
34. Microsoft, MSN Search history and 2004 technology preview
35. Google, October 23, 2000 AdWords launch
36. Microsoft and Yahoo, July 29, 2009 agreement
37. DuckDuckGo, company history
38. Google, May 16, 2012 Knowledge Graph announcement
39. Brave Search, June 22, 2021 beta launch
40. Brave, April 27, 2023 index announcement
41. Microsoft, February 7, 2023 AI search announcement
42. OpenAI, October 31, 2024 ChatGPT search launch