Where Does Google Get Its Information: Architecture from Knowledge Graph to AI Overviews

Google assembles what it knows from several distinct sources, not one index: pages found by its crawler, a Knowledge Graph of entities and facts, licensed data feeds, business profiles, structured data that site owners embed in their pages, and, for AI Overviews, a generative layer that synthesises an answer from the best retrieved passages. Each source has its own freshness, its own verification, and its own place on the results page. This article walks through those sources, the constraints that shape them, how their rankings are combined, and how to diagnose a missing Knowledge Panel or a suppressed AI Overview.

Google does not publish the internals of its ranking systems. Where this article gives formulas or pipeline stages, they are a generic reference design for hybrid retrieval, drawn from public research and from the standards listed in the references, not a description of Google's production code.

From Crawl to AI Overview: The Six Sources Behind a Google Answer

A modern search stack can be pictured as a multi-tier retrieval pipeline. A router collects signals from six primary conduits, and each one feeds a different surface on the results page.

• Web crawling and indexing. Googlebot fetches pages, and an inverted index of postings lists makes lexical lookup fast. This feeds the standard organic results.
• The Knowledge Graph. A directed graph of entities and the facts that connect them. It feeds Knowledge Panels, entity cards and fact boxes.
• Licensed data feeds. Structured feeds delivered by data providers, which power widgets such as weather, stocks, sports and flight status.
• User and business profiles. Business profile data, including identity verification, feeds the local pack and local business panels.
• Structured data markup. Schema.org markup in JSON-LD, extracted when a page is crawled, feeds rich snippets, merchant listings and carousels.
• Generative retrieval. A retrieval-augmented generation (RAG) layer, which Google uses for AI Overviews, retrieves passages and facts and synthesises an answer from them. Such a layer can draw on cached search results and on user-generated content as well as on model weights.

The five-phase sequence below is a useful way to read the rest of this article.

A Five-Phase Reference Pipeline

1. Phase 1: Multi-pipeline ingestion. Signals from the six conduits above are collected and routed within a small time budget.

2. Phase 2: Entity disambiguation and triple reconciliation. The system extracts subject-predicate-object triples $(s, p, o)$ in the data model standardised by W3C RDF 1.1, and reconciles each entity mention against the graph by checking whether independent sources corroborate it. In a design like this, corroboration is what can earn an entity a Knowledge Panel. A simple conceptual confidence score for an entity $e$ over a set of sources $S_i$ with weights $w_i$ is:

   $C(e) = \sum_{i=1}^N w_i \cdot \mathbb{I}(e \in S_i)$

   This is an illustration of the idea that more independent agreement yields more confidence. It is not a published Google formula, and the threshold at which an entity is committed to the graph is not public.

3. Phase 3: Dual-channel retrieval. A sparse channel scores inverted-index candidates with BM25. A dense channel runs approximate nearest-neighbour search over Hierarchical Navigable Small World (HNSW) graphs, often with product quantisation to shrink the vectors. The two candidate lists are merged with Reciprocal Rank Fusion (RRF) with smoothing constant $k = 60$:

   $\text{RRF}(d) = \sum_{m \in \{\text{sparse}, \text{dense}\}} \frac{w_m}{k + r_m(d)}$

4. Phase 4: Click-feedback reranking. A component such as NavBoost reranks the merged candidates using historical click interactions, with an exponential time decay so that older clicks count for less.

5. Phase 5: Generative synthesis and verification. A RAG model drafts an answer from top-ranked passages and knowledge-graph facts. A verification step can measure how consistent several sampled answers are, and fall back to a deterministic result such as a Knowledge Panel when they disagree.

Terms that recur in this area: Knowledge Graph, entity disambiguation, RDF triples, Knowledge Panel, AI Overviews, Retrieval-Augmented Generation, Neural Generative Retrieval, HNSW, product quantisation, NavBoost, entity corroboration, semantic entropy, multi-hop traversal, Reciprocal Rank Fusion and sparse-dense hybrid indexing. How crawling, indexing and ranking fit together is covered in How Search Engines Work, and how entity signals reach the graph is covered in Knowledge Graph Entity Signals.

Operating Limits: Latency, Memory and Freshness Across the Sources

Every tier runs inside physical limits on latency, memory bandwidth and throughput, and the limits explain why each source behaves the way it does.

Memory is the clearest example. A 768-dimensional embedding stored as 32-bit floats occupies 3,072 bytes, and a collection of billions of them would need an impractical amount of RAM. Product quantisation (PQ) compresses a vector by splitting it into sub-vectors and replacing each with a short code. The cost is some quantisation noise, which is why a more precise reranker usually follows. Hierarchical proximity graphs such as HNSW (Malkov and Yashunin, IEEE TPAMI 2020) are a common structure for approximate nearest-neighbour search over such vectors.

Freshness differs by source, and the differences matter when a site owner is deciding where to publish a fact.

Two boundary conditions are worth knowing. First, when an inbound document stream violates HTTP semantics (RFC 9110) or carries inconsistent provenance metadata, a well-designed ingestion pipeline can quarantine the payload so that contaminated triples never reach downstream answers. Second, if a server responds slowly, a crawler may stop before it extracts structured markup from the page, so slow edge responses can cost a site its rich results. RFC 9110 describes the content negotiation and caching mechanisms that help servers respond quickly.

The end-to-end latency of an answer can be written as the slower of the two first-stage channels plus the later stages, all of which must fit a response-time budget $B$:

$L_{\text{total}} = \max(L_{\text{sparse}}, L_{\text{dense}}) + L_{\text{NavBoost}} + L_{\text{synthesis}} \le B$

If the generative stage cannot meet its budget, a search engine can fall back to standard results rather than show a late answer.

Fusing Rankings and Weighting Clicks: The Scoring Model

Sparse BM25 scores and dense cosine scores live on different scales, so adding them directly makes the final ranking unstable. Reciprocal Rank Fusion avoids the problem by ignoring scores and using only rank positions. Each retriever contributes the weighted reciprocal of a document's rank, and the constant $k$ (traditionally 60) damps the influence of the very top positions.

A click-feedback tier adds a behavioural signal. A common way to model it is a sum over past time steps $t$ of click interactions $C(d, q, t)$, each discounted by an exponential decay with rate $\lambda$ and scaled by a weight $\alpha$:

$S_{\text{NavBoost}}(d, q) = S_{\text{prior}}(d, q) + \alpha \sum_{t=0}^{T} e^{-\lambda (t_{\text{now}} - t)} \cdot \log(1 + C(d, q, t))$

Combining the two gives one conceptual ranking score:

$\text{score}(d) = \sum_{m \in \{\text{sparse}, \text{dense}\}} \frac{w_m}{k + r_m(d)} + \alpha \sum_{t=0}^T e^{-\lambda (t_{\text{now}} - t)} \cdot \log(1 + C(d, q, t))$

The weights, the decay rate and the observation window of any real system are not public, so treat the symbols as parameters rather than as known values.

The generative layer needs its own safeguard against false statements. One research technique is discrete semantic entropy. For a query $x$, the system samples several candidate answers and clusters them into $K$ equivalence classes $\{C_1, \dots, C_K\}$ according to whether they entail one another. The entropy of the cluster distribution is:

$H_{\text{semantic}}(Y \mid x) = - \sum_{k=1}^K p(C_k \mid x) \log p(C_k \mid x)$

A high value means the sampled answers disagree about the facts, which is a warning sign for a hallucinated claim. A generative system can use such a signal to withhold speculative text and show a deterministic result instead. Whether and how Google applies it is not public, so it is described here as a mechanism, not a known setting.

Diagnosing Missing Knowledge Panels and Suppressed AI Overviews

Silent visibility failures usually trace back to one of four structural faults in the knowledge pipeline.

• Entity disambiguation collisions. A company or product name that is shared with other things gets conflated with a different entity, for example a different Wikipedia or Wikidata entry. When a portal publishes conflicting descriptions of the same brand across subdomains, the system's confidence in the entity stays low and a Knowledge Panel may not render.
• Semantic entropy divergence. When sampled generative answers about a brand disagree, an AI Overview may be suppressed or may avoid mentioning the brand.
• Click-feedback attenuation. A page that users abandon quickly can lose weight in click-based reranking over time.
• Crawl postponement. Slow edge responses can make a crawler abort before it extracts the page's structured markup.

Remediating an entity collision starts with formal triple validation and explicit disambiguation. Publish fully qualified Schema.org JSON-LD that includes unambiguous sameAs references pointing to the profiles that independently describe the entity, such as an encyclopedia entry, a business registry or an official directory. The aim is to give the graph several independent places that agree.

A conceptual rule for the generative gate says that when observed entropy exceeds a chosen threshold $\tau$, generative output is suppressed in favour of a deterministic result:

$H_{\text{observed}}(Y \mid x) - \tau > 0 \implies \text{suppress generative output}$

Keeping machine-readable provenance, for example with the W3C PROV Data Model, helps because it makes the origin and history of each claim easy to verify. How AI answer engines source content differently from classic search is covered in AI Answer Engines vs Traditional Search.

Standards and Markup That Feed Google's Entity Pipeline

Durable visibility depends on a handful of open standards.

• W3C RDF 1.1 Concepts and Abstract Syntax defines the model of directed multigraphs and typed edges, in which a fact is a subject-predicate-object triple.
• W3C PROV-DM specifies how to describe the origin, attribution and transformation history of information, which lets a consumer verify where a statement came from.
• RFC 9110 defines HTTP semantics, including content negotiation and caching.
• Schema.org with JSON-LD embeds structured semantic graphs in a page, so crawlers and RAG indexers can read an entity's properties without parsing the layout. Google's structured data guidelines explain which markup types it supports for rich results, and valid markup is a precondition for features such as rich snippets and merchant listings.

The example below shows how a TechArticle can declare what it is about and who publishes it, using placeholder values:

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "TechArticle",
      "@id": "https://example.com/architecture#article",
      "headline": "Where Does Google Get Its Information: Architecture from Knowledge Graph to AI Overviews",
      "inLanguage": "en-US",
      "about": [
        {
          "@type": "Thing",
          "name": "Google Knowledge Graph",
          "sameAs": "https://example.com/glossary/knowledge-graph"
        },
        {
          "@type": "Thing",
          "name": "Retrieval-Augmented Generation",
          "sameAs": "https://example.com/glossary/retrieval-augmented-generation"
        }
      ],
      "publisher": {
        "@type": "Organization",
        "@id": "https://example.com/#organization",
        "name": "Example Organization",
        "url": "https://example.com"
      }
    }
  ]
}

A triple belongs in a knowledge graph $\mathcal{G}_{\text{KG}}$ when its parts are valid and its provenance is strong enough, which can be written as:

$E(s, p, o) \in \mathcal{G}_{\text{KG}} \iff \text{val}(s) \land \text{val}(p) \land \text{val}(o) \land \text{prov}(s, p, o) \ge \tau$

References

• W3C RDF 1.1 Concepts and Abstract Syntax — W3C. https://www.w3.org/TR/rdf11-concepts/
• PROV-DM: The PROV Data Model — W3C. https://www.w3.org/TR/prov-dm/
• HTTP Semantics (RFC 9110) — IETF, 2022. https://www.rfc-editor.org/rfc/rfc9110
• Google Search Central Structured Data Guidelines — Google. https://developers.google.com/search/docs/appearance/structured-data
• Efficient and Robust Approximate Nearest Neighbor Using HNSW — Malkov and Yashunin, IEEE TPAMI, 2020. DOI:10.1109/TPAMI.2018.2889473