A modern AI training chip can cost more than a car, and a single facility now packs hundreds of thousands of them to train one model. Yet on a cluster that large, those chips spend a surprising share of their time doing nothing — sitting idle, waiting on the network that connects them. "The most expensive idle asset in the world right now," as one networking firm put it, "is a GPU waiting on the network."
That inversion — the recognition that in large-scale AI training the bottleneck is the network, not the chip — is quietly redrawing the AI build-out. It has turned switches, cables and optics into one of the most contested layers of the boom, and it explains an acquisition that, years later, looks like one of the shrewdest in technology.
Why the chips wait
Training a large model means copying it across thousands of chips, each working on a slice of the data. After every step the chips must stop and exchange what they have learned — an operation engineers call an all-reduce — and none can begin the next step until all of them have finished. The whole cluster therefore moves at the speed of its slowest link: one late packet stalls tens of thousands of chips at once. On an unoptimized network, GPUs can sit idle 30% or more of the time; well-built infrastructure pulls that below 15%. The hard part of AI infrastructure is not buying a fast chip — it is wiring thousands of them together so the fast chips never have to wait.
The network becomes the computer
The stakes scale with the clusters, which have grown at a staggering pace. xAI's Colossus supercomputer in Memphis reached 100,000 GPUs in 122 days and doubled to 200,000 in 92 more. By January 2026 it had expanded to roughly 555,000 GPUs drawing about 2 gigawatts — as much power as a mid-sized city — at an estimated $18 billion, with a stated roadmap toward a million. At that size, connect it badly and an $18 billion machine performs like a far smaller one. The wiring between the chips has become the computer.
The company that bought the wire
The clearest bet on this future was placed years before the boom. In 2019, when Nvidia (NASDAQ: NVDA) was still seen mainly as a graphics-chip company, it agreed to buy Mellanox for about $6.9 billion. Mellanox made not the marquee processors but InfiniBand, the premium networking that stitches supercomputers together. The deal underwhelmed Wall Street at the time; it proved prescient. When AI demand arrived, Nvidia sold not only the chips but the network that connects them, the switches on top, and the optics in between. In its fiscal fourth quarter of 2026 the company reported record networking revenue of $11.0 billion, up 263% from a year earlier — a graphics company that had quietly become one of the largest networking players on earth, encroaching on the turf of Cisco (NASDAQ: CSCO).
Ethernet won the argument
When one company owns a critical layer, the rest of the industry writes an open standard to break the lock-in. In 2023, a group including AMD (NASDAQ: AMD), Arista (NYSE: ANET), Broadcom (NASDAQ: AVGO), Cisco, Intel (NASDAQ: INTC), Meta, Microsoft, HPE and Eviden formed the Ultra Ethernet Consortium, and in June 2025 it published a 560-page 1.0 specification to give open Ethernet the capabilities that were once InfiniBand's alone. But the argument was already settling in Ethernet's favor — and Nvidia had seen it coming. It built its own Ethernet product, Spectrum-X, and the largest AI cluster on the planet, Colossus, runs on that Ethernet rather than Nvidia's premium InfiniBand, delivering about 95% of the network's throughput against roughly 60% for traditional Ethernet. The contest is no longer open Ethernet versus Nvidia; it is whose Ethernet wins. (InfiniBand is not dead — it still leads classic supercomputing — but share is shifting.)
The copper wall
Beneath the corporate contest is a physics problem. Inside a rack, chips talk over short, thick copper links at enormous speed — Nvidia's newest rack design moves data between GPUs at 3.6 terabytes per second. Across the building, a single Broadcom switch chip now moves 102.4 terabits per second, double any before it. At those speeds copper stops cooperating: an electrical signal degrades over distance, and the industry is being pushed from electrons to photons — from copper to glass. Nvidia stuck with copper in its latest rack only because switching to optics would have added roughly 20,000 watts of power per rack; it is now moving toward optical connections as copper runs out. When a company starts talking about "co-packaged optics," that is the tell that it is hitting this wall — and building for real scale.
The least speculative layer
There are reasons for caution about the wider AI build-out: much of it is financed in a circle, with chipmakers investing in the startups that buy their chips, and AI spending far outrunning AI revenue — a pattern that has fueled bubble warnings. But the networking layer is arguably the least speculative part of the stack. Whether the winner is one giant model or a thousand small ones, every one of them has to move data between chips. The companies that make that possible — beyond Nvidia and Broadcom, names such as Arista, Marvell (NASDAQ: MRVL), Astera Labs (NASDAQ: ALAB), Credo (NASDAQ: CRDO) and optics makers like Coherent (NYSE: COHR) — sit quietly inside every gigawatt that gets built. The AI story looks like it is about chips. It is really about the wire between them.