First-Principles Thinking on AI Supernode Unified Addressing — From Hopper to NVL72, and AI Native Memory Fabric

A Mental Shift: Unified Addressing Is a Philosophy, Not an Implementation

When discussing unified addressing, we traditionally start from CUDA UVA or CPU virtual memory. But for AI supernodes, this misses the point entirely.

Through deep analysis of the Hopper architecture, NVL72 topology, and DeepEP's addressing design, I have come to realize that unified addressing is not a fixed implementation — it is a design philosophy.

The core insight is deceptively simple:

Location Transparency does not require a Global Address Space. The scope of address transparency need not equal the scope of the Fabric — it can equal the scope of the Communication Domain.

This single insight changes everything about how we think about AI supernode architecture.

1. The Core Philosophy: Location Transparency at Any Scope, at Any Time

1.1 What Is the Essence of Unified Addressing?

Strip away the hardware details, and the essence of unified addressing is one thing: enabling a GPU kernel to access remote memory as if it were local.

The kernel should express:

float val = load(ptr);

Not:

float val = load(gpu7, hbm2, offset);

This property — that the kernel does not need to know where data physically resides — is called Location Transparency.

1.2 The RDMA Analogy: Address Capability as a Control-Plane Service

To understand why Location Transparency is a service rather than a given, consider how RDMA works.

A common misconception is that RDMA "natively" accesses remote memory. In reality, before any RDMA Read can execute, the control plane must complete:

1. Memory Region registration — pin the buffer, assign lkey/rkey

2. Queue Pair establishment — create the communication channel

3. Address and permission exchange — share virtual addresses, rkeys, and QP numbers between peers

After this setup, the NIC knows: Remote Virtual Address + R_Key + QP Context. Only then can the data plane execute zero-CPU RDMA operations.

RDMA proves a fundamental point: Address Transparency is not an inherent property of the data plane — it is a capability established by the control plane before communication begins.

The analogy to NVLink is direct: a GPU can execute load(ptr) not because NVLink natively knows remote addresses, but because the system has pre-established an Address Resolution Service that resolves ptr into a fabric-visible peer memory target.

And critically, this service is established per communication relationship, not globally. Each QP connects specific peers; each MR maps specific buffers. The scope of RDMA address transparency is exactly the scope of the communication domain — no more, no less.

1.3 The Design Space: When and Where to Build Transparency

If we accept that unified addressing is fundamentally an Address Resolution Service, then two design axes emerge:

Scope axis — how many GPUs does the transparency cover?

Scope	Description	Cost
All GPUs in cluster	Global Address Space (NVL72 today)	High: page tables, TLB, peer mapping for all
Communication Domain	Only GPUs that actually communicate	Medium: proportional to actual need
Communicator	GPUs within one NCCL communicator	Low: established at library init
Instance	GPUs serving one inference request	Minimal: per-task, released after

Time axis — when is the transparency established?

When	Characteristics
System boot	Static, always available, wasteful if unused
Library init	Per-communicator, moderate overhead
Instance start	Per-task, pay-as-you-go
Runtime (on-demand)	Most flexible, requires runtime support

The current NVL72 approach — establishing Global Address Space at system boot — is one deliberate point in this design space. It trades higher initialization and management cost for the simplest programming model and lowest runtime overhead.

1.4 Why Did Hopper Choose This Point?

This choice stems from HPC's fundamental assumption:

In traditional HPC, any MPI Rank may communicate with any other Rank. The simplest software model is to establish a Global Address Space for the entire Fabric at once.

For static clusters of dozens of GPUs, this is a reasonable trade-off. But it comes with constraints:

Centralized service dependency: NVL72 requires Fabric Manager and NVLSM as rack-wide control-plane services
Scope mismatch: Even if a kernel only accesses 4 GPUs, every GPU's GMMU maintains Peer Mapping for all 72
No dynamic adaptation: Cannot release mappings when communication patterns change

2. Why Do AI Supernodes Need Location Transparency?

2.1 Fusion Operators Have Changed the Nature of Communication

In the past, GPU kernels accessed local HBM, and inter-GPU communication was handled by NCCL and MPI. Compute and communication were two distinct phases.

But today's AI algorithms increasingly adopt Fusion Operator designs:

Fusion Operator	Remote Access Pattern
MoE Fusion Kernel	Directly access parameters on different Expert GPUs
Cross-GPU Attention	Directly read KV Cache on remote GPUs
Engram Memory / Embedding Cache	Massive random remote access inside kernels

Communication has begun to merge into compute itself.

2.2 What Does a Fusion Operator Want to Express?

For a fusion operator, what it truly wants to express is:

// Ideal: location-transparent access
float val = load(ptr);

Not:

// Reality: explicit location breaks compute semantics
float val = load(gpu7, hbm2, offset);

And certainly not the degenerate path:

Fusion Kernel → Suspend → CPU Runtime → Address Lookup → Network → Resume Kernel

Because this means the GPU must exit its execution flow and let the CPU resolve addresses. This not only adds latency — more importantly, it breaks the GPU pipeline, preventing remote memory loads from being issued as continuously as local loads.

2.3 The Location Service Model

A unified addressing system essentially adds a layer of Location Service between GPU and memory:

    AI Object
       │
   Location Service
       │
  Physical Location
       │
    Transport
       │
  Remote Memory

This maps to three independent questions:

Question	Responsibility	Hopper Mapping
Location	Where is the object?	Control Fabric
Translation	Logical address → accessible location	Memory Fabric
Transport	How to deliver the request?	Transport Fabric

3. How Does Hopper Implement the Location Service?

Note on terminology: The following three-layer model (Control Fabric / Memory Fabric / Transport Fabric) is an architectural abstraction I propose to clarify responsibilities. It is not NVIDIA's official terminology. My contribution here is not to explain NVIDIA's implementation, but to re-abstract it into a framework that reveals the underlying design principles.

With this framework, we can see that NVIDIA did not put the Location Service into NVLink — instead, the system is divided into three layers with entirely distinct responsibilities:

               CUDA Runtime
                    │
──────────────────────────────────────
         Control Fabric
──────────────────────────────────────
Fabric Manager / NVLSM / Driver
──────────────────────────────────────
         Memory Fabric
──────────────────────────────────────
GMMU / TLB / Page Table /
Memory Aperture / Peer Mapping
──────────────────────────────────────
        Transport Fabric
──────────────────────────────────────
NVLink / NVSwitch / HBM

3.1 Control Fabric: Manages Topology, Not Memory Pages

The Control Fabric handles GPU Discovery, Topology Discovery, Partition, Routing Programming, and Peer Registration. Fabric Manager and NVLSM maintain the resource topology of the entire NVL72 — they know where GPUs are and how switches are connected, but they know nothing about Tensors, KV Caches, or Experts.

3.2 Memory Fabric: The Concrete Manifestation of the Location Service

When a GPU kernel executes load(ptr), the request enters the GMMU. Through TLB lookup, page table traversal, and aperture mechanisms, the virtual address is resolved into a fabric-visible peer memory target, which ultimately maps to remote HBM.

The Memory Fabric itself has no control logic — it is merely the execution result of the Control Fabric left on the GPU.

3.3 Transport Fabric: Pure Data Plane

NVLink and NVSwitch see only (Destination GPU, HBM Offset, Payload). They have no knowledge of Virtual Addresses. NVLink's responsibility has always been Transport, not Address Translation.

3.4 Complete Data Path

   Control Fabric
         │
   Establish Peer Mapping
         │
   Memory Fabric
   (GMMU / Aperture)
         │
   Resolve Target GPU
         │
   Transport Fabric
      (NVLink)

4. Hopper Server vs. NVL72: From Interconnect to Computer Fabric

Dimension	Hopper Server (8 GPU)	NVL72 (72+ GPU)
Control Scope	Single node	Rack
Control Plane	CUDA Driver	Fabric Manager + NVLSM
Memory Fabric	Node-global Address	Rack-global Address
Peer Mapping	8 peers	72+ peers
TLB Pressure	Low	High
Partition	Not supported	Supported (MIG / Multi-tenant)

In a Hopper Server, 8 GPUs make Peer Mapping tractable — establishing a Global Address Space at boot is reasonable.

But NVL72 is not simply "more GPUs." The fundamental shift is that the Fabric transforms from a Node Interconnect into a Computer Fabric.

Traditional GPU server:

CPU → PCIe → GPU → NVLink → GPU

NVL72:

GPU → NVSwitch Fabric → GPU → GPU Memory

NVL72 is essentially a large-scale GPU NUMA machine — a single logical computer where distributed memory semantics must be maintained across 72 GPUs. This is why NVIDIA introduced rack-wide Fabric Manager and NVLSM: the system needs centralized control not for performance, but for coherence and consistency at scale.

Both employ the same addressing strategy:

      Global Fabric
           │
   Global Memory Fabric
           │
  Global Address Space

The Control Fabric and Memory Fabric share the same scope — the entire NVL72.

5. AI Workloads Have Changed the Assumption

AI workloads no longer follow the "all GPUs may communicate" assumption:

An Attention Kernel may only involve GPU4~GPU7
A MoE Kernel may only access a few Expert GPUs
An inference instance may use only a fraction of NVL72

The fundamental communication unit of AI has shifted from the entire Fabric to the Communication Domain.

5.1 What Is Truly Necessary?

What is truly necessary is not a Global Address Space spanning the entire NVL72, but:

Establishing address-transparent capabilities for the current compute task before communication begins.

Global Address Space is merely one way to achieve Address Transparency — and a deliberate trade-off, not the only option.

Address transparency can be established at different moments:

When	Scope	Use Case	Hopper Support
System boot	Entire NVL72	HPC all-to-all	✓ (current)
NCCL init	Communicator	Collective comms	Partial
Inference instance start	Instance-domain	Inference serving	✗
Fusion operator start	Communication Domain	MoE Fusion	✗

What is truly indispensable is the Address Resolution Service, not the Global Address Space.

6. Hopper's Limitation: Fabric Scope Bound to Address Scope

From this perspective, Hopper's real limitation is not in NVLink, nor in NVSwitch — it is in the scope of the Memory Fabric.

Today, the Control Fabric manages the entire NVL72, and therefore the Memory Fabric also covers the entire NVL72. Based on the architecture of NVL72 and the behavior observed in multi-GPU peer access patterns, the current design appears to maintain Peer Mapping for all GPUs in the fabric regardless of actual communication needs — even if a Fusion Kernel only accesses 4 GPUs, the GMMU infrastructure is provisioned for the entire rack.

Note on evidence: The specific microarchitecture behaviors described below (Peer Mapping scope, TLB pressure characteristics, mapping lifecycle) are inferred from NVIDIA's public architecture whitepapers, the GH200/NVL72 system design documentation, and observed profiling behaviors. NVIDIA has not publicly disclosed the exact implementation details of GMMU peer mapping tables or their TLB replacement policies. These claims should be treated as architectural inferences rather than verified microarchitecture facts, and readers are encouraged to cross-reference with official NVIDIA documentation where available.

Fabric Scope and Address Scope are bound together.

Cost Item	Current State (Inferred)	Potential Problem
Page Table	Covers entire NVL72	Mismatched with actual communication domain
TLB Miss	Global mapping increases pressure	May increase remote access latency
Peer Mapping	Established at boot, persistent	Cannot be released or shrunk on demand
Fabric Manager	Maintains global view	Centralized scalability concern

7. Next: Memory Fabric from Static Resource to Dynamic Service

The future does not require overthrowing Hopper's three-layer architecture. The Control / Memory / Transport division remains sound. What must change is the lifecycle of the Memory Fabric.

    Communication Domain
             │
             ▼
   Address Resolution Service
        (Control Plane)
             │
             ▼
     Address Domain
   (Page Table / Peer Mapping)
             │
             ▼
      Memory Fabric
   (GPU can directly load/store)
             │
             ▼
     Transport Fabric
   (NVLink / UALink / UB)

What is truly dynamic is the Address Resolution Service. The Memory Fabric is merely the concrete manifestation of address-resolution relationships in GPU GMMUs and page tables.

8. Address Resolution Service: Who Owns the Authority?

The dynamic Address Resolution Service raises a critical system design question: Who owns the Address Authority, and how is consistency guaranteed?

Consider a concrete scenario: in a MoE system, an Expert migrates from GPU12 to GPU25. The Address Resolution Service must:

1. Update the mapping: Expert_X → GPU25 (was GPU12)

2. Invalidate stale caches: Any GPU that cached the old mapping must be notified

3. Ensure consistency: No GPU should access the old location during migration

4. Maintain security: Only authorized kernels may access the Expert's memory

This is analogous to DNS:

Domain Name → IP Address  (with TTL, caching, invalidation)

Future AI Object Directory:

Expert Object ID → GPU Location  (with versioning, invalidation, consistency)

The Address Control Plane must provide:

Property	Mechanism
Versioning	Each mapping has a version number; stale versions are rejected
Invalidation	Broadcast or targeted invalidation when mappings change
Caching	Local TLB-like cache with coherence protocol
Consistency	Read-after-write guarantees during migration
Security	Per-domain access control; kernels can only access authorized objects

The Address Resolution Service is not just a lookup table — it is a distributed control plane with consistency, caching, and security guarantees.

9. AI Object Directory: From Memory Fabric to AI Data Fabric

The most forward-looking evolution is from address-based to object-based addressing.

Today's unified addressing manages:

Virtual Address → GPU + HBM Offset

This is still a memory system abstraction — it serves the memory subsystem, not the AI workload.

Future AI supernodes may manage:

AI Object ID → GPU + HBM Offset

This is an AI runtime system abstraction — it serves the fusion operator directly.

The Layered Architecture

        AI Runtime
             │
     Object Directory
     (Expert, KV Block, Embedding)
             │
     Memory Service
     (Address Resolution)
             │
     GPU Memory Fabric
     (GMMU / TLB / Page Table)
             │
     Transport Fabric
     (NVLink / UALink)

Concrete Example: MoE Expert Dispatch

Today (address-based):

token → expert_id → GPU rank → load(remote_ptr)

Future (object-based):

token → Expert Object ID → Object Directory → GPU Memory

The kernel no longer needs to know which GPU holds the Expert. It simply requests the Expert by ID, and the Object Directory resolves the location.

This is the essence of AI Native Memory Fabric: the memory system understands AI objects, not just memory pages.

10. Summary: Unified Addressing Is Not Unified Addresses — It Is Unified Location

Unified addressing is not about building a single address space spanning the entire supernode — it is about establishing location-transparent access semantics for the current compute task, enabling GPUs to access remote memory as if it were local HBM.

Three core conclusions:

1. It is a philosophy, not an implementation: Unified addressing = Location Transparency. The scope can be all GPUs or a communication domain; the timing can be boot, library init, or runtime. The current NVL72 global fabric is one deliberate trade-off among many.

2. Hopper's soundness and limitation: The three-layer Fabric architecture (Control / Memory / Transport) is sound, but the binding of Fabric Scope and Address Scope is a legacy constraint. NVL72 is essentially a large-scale GPU NUMA machine.

3. Dynamicization + Object Directory is the direction: What is truly indispensable is the Address Resolution Service with consistency, caching, and security guarantees. The future is an AI Object Directory that maps AI objects (KV Blocks, Experts, Embeddings) to physical locations — the core infrastructure of AI Native Memory Fabric.

Hopper chooses to establish a Global Memory Fabric covering the entire NVL72 at system boot — an excellent engineering implementation for HPC workloads. But it is not the only implementation. For the future AI Native Supernode, the more reasonable direction is to dynamically establish the Address Resolution Service driven by the runtime, evolving unified addressing from "static rack-level resource" to "on-demand runtime capability."