Google Search Lab
πŸ“ŠOverview
🌐Crawl & Analyse
πŸ“‚Inverted Index
βš™οΈRanking Engine
πŸ›°οΈQuery & Infra
Labs completed: 0 / 4 β€’ Rating estimate: 0.0 / 10
πŸ”Ž

Search Engineer Trainee

Use each lab as an experiment. Follow the mission steps, then observe the system response.

Hint: Google does not scan the whole web for every query. It crawls and analyses pages ahead of time, builds an inverted index, and answers from that index using ranking and distributed infrastructure.[web:19][web:70]
Current mission
  • Overview: watch one query move through all four layers.
  • Crawl: ensure important public docs enter the repository but private/test docs do not.
  • Index: build posting lists and run an intersection query.
  • Rank: adjust signal weights and see which doc becomes #1.
Journey progress0%
πŸ“ŠOverview – Pipeline view
Follow one query through crawl, analysis, index, ranking, and infrastructure.
A modern search engine has four core layers: crawling (discover public documents), document analysis (extract words, links, language), inverted index + ranking (build term β†’ document postings and compute scores), and query + infrastructure (parse the user query and answer from distributed data centers).[web:19][web:70]
Pipeline diagram

Mission: click "Animate" and carefully read each box as it lights up.

1. Crawl

Googlebot fetches URLs.

2. Analyse

Tokenise text, detect language.

3. Inverted index & rank

Build postings and scores.

4. Query & infra

Merge shards and respond.
Steps for this lab

1. Press the button once and track the order of boxes.
2. Say out loud: "crawl β†’ analyse β†’ index & rank β†’ query & infra".
3. Ask: where do my documents live? Where does my query travel?

🌐Crawl & document analysis
Choose which documents are allowed into the system.
Crawlers like Googlebot fetch public URLs. A document analyser then turns each page into structured data: tokenised words, stripped stop‑words, extracted links, and language tags.[web:19][web:78] Private or blocked documents should remain outside the index.
Document set

Mission: allow important docs, block login/test docs, then run the crawl.

Steps for this lab

1. Keep doc101 and doc102 crawlable.
2. Block doc103 (login) and doc104 (robots/test).
3. Decide what to do with doc105 (low‑value parameter page).
4. Click "Run crawl & analyse" and check the log.

πŸ“‚Inverted index builder
Create term β†’ document posting lists.
Google stores a huge dictionary of terms where each term points to a posting list of document IDs. Queries look up one or more terms and intersect or combine those lists to find candidate documents.[web:75][web:70] Here you manually build a tiny version.
Documents and terms

Mission: make sure the key documents appear in postings for "search" and "index".

Steps for this lab

1. Click terms in each document to send doc IDs into postings.
2. Use the term selector to inspect postings.
3. When ready, press "Run sample query" to intersect postings for "search" and "index".

Current term β†’ document IDs
βš™οΈRanking engine
Balance relevance, PageRank and freshness.
For each candidate document, Google computes scores using term statistics, link‑based authority (PageRank), freshness, and many other signals.[web:70][web:75] In this simplified engine you choose which signals matter and see how the winning document changes.
Query: "search engine architecture"

Mission: get the most balanced technical article to rank #1.

Steps for this lab

1. Start with all signals on and note the top document.
2. Turn off PageRank and see what changes.
3. Turn off freshness and compare again.
4. Discuss which combination seems best for this query.

πŸ›°οΈQuery engine & infrastructure
Parse queries and respond from distributed data centers.
The query engine cleans the input (spelling, synonyms), fans out a request to multiple index shards, merges the responses, and sends them from nearby data centers to keep latency low.[web:70][web:40] You will toggle query features and adjust server counts to see both relevance and speed change.
Query configuration

Mission: produce a good candidate set while keeping latency under ~80 ms.

Parsed query & candidate docs
Steps for this lab

1. Choose which query features should be on for "search engine design".
2. Adjust server sliders for India, Europe and US.
3. Run the query and check candidate docs and latency.
4. Try to reach a configuration with both good docs and low latency.