The Inevitable Shift

We are on the horizon of the inevitable shift towards GPU-dominated infrastructure, and this movement will build several unicorn companies.

Voltage Park, a soon-to-be billion-dollar H100 provider, shows the momentum of the shift towards GPUs. Accessible GPUs are going to transform the infrastructure of the web. As Jensen says, “Moore’s law is dead,” and it is time for the next generation of computing infrastructure. CPU-based public cloud operations can’t move quickly and capture this demand for GPUs because they have built their entire business model on low-cost, low-power, commoditized CPUs. GPUs are energy-dense and require more advanced cooling than CPUs. GPU-specific suppliers, like Voltage Park, are designing their operations and infrastructure to support these requirements. These new types of suppliers are capturing this growing demand while the market waits for overpriced public cloud-supplied GPUs.

The next unicorn companies within AI are infrastructure companies. Demand is only increasing as large enterprises are now beginning to adopt AI and LLMs. NVIDIA is sold out for years to come. The GPU deficit, combined with the public cloud’s subpar design for GPU operations, has pushed customers to alternatives like Voltage Park and the secondary market.

The nascent secondary market for GPUs is a forcing function of the annual billion-dollar shortage of GPUs. The supply and pricing you get from system integrators and direct relationships with NVIDIA are consistent and stiff, just like buying a new car. You aren’t going to get a fantastic deal anymore from OEMs or integrators. NVIDIA has a tight grip on every single hardware builder. From shortages and high demand, a new bare metal marketplace is here.

When you need a car, the inevitable choice is presented: new or used. Buying a new car is a relatively standard and inflexible process with little room to make a deal. If you find a deal, it usually comes from the manufacturer as a financial incentive to the dealer (ergo, NVIDIA Inception Program). Typically, the dealership has a minimal margin on new car sales. The used car market, on the other hand, has a higher variance between cars, making it harder to price consistently. Therefore, good deals exist predominantly in the secondary market.

Hydra Host’s platform supports this new secondary market for GPUs, and we are seeing many massive and insane deals. Rising prices and endless demand have given birth to a bare metal marketplace. Like with cars, pricing is better on the secondary market. Some suppliers bought early and can run GPUs 30% cheaper than the public cloud. Data centers are refurbishing GPUs and running them at half the price. Resellers are buying cards on cheap and holding them.

If you want to see what is happening in the GPU market, sign up at Hydra Host and see what is happening.