Next Phases of the Bare Metal Market

The joke within the venture community is that they are not funding AI companies but funding infrastructure to buy NVIDIA chips. Investing in AI companies today is just a direct transfer to NVIDIA. Due to GPU demand, that is true today, but supply and pricing pressures on HPC infra will evolve the market in the next 12 months.

DCs are going direct and selling wholesale. Before the GPU revolution, there were generally only two hosting options - hyper-scalers or colocation. That is changing now due to the cost of GPUs and limited supply. Pre-GPU revolution, CPUs were highly commoditized, and you needed a massive scale to have any competitive advantage. It didn't make sense for most companies to expend capital on a commodity bare metal. The world today is changing quickly. GPUs vastly differ from CPUs: compact, immensely powerful, expensive, and not commoditized. In this new world, DCs are creating a new business line by bypassing cloud companies and renting GPUS directly to companies. Colocation space that would have previously gone to the public cloud or big corp colocation will go to the rent-a-GPU business to make cloud-like margins. Data centers are even adding financing capabilities to get more GPU business. Going direct saves AI or platform companies 15-20% on their computing and hosting costs.

Companies are starting to resell their GPUs to others. HPC hardware is being over-acquired and underutilized, but the reasons are similar to the situation with Web 1.0 companies (which spawned the public cloud). Due to infra limitations, Web 1.0 companies needed extra bare metal capacity to scale for something as simple as visitors. You had to build your sovereign infra to handle traffic spikes. Today, the market situation looks familiar. Companies need bursting capacity to massive clusters for better training runs. Rather than waiting for an unknown period of time, constantly hunting for capacity, or having your training run cut off by another customer, companies are sitting on underutilized hardware or inefficiently using GPUs ("Well, it is there, so might as well use it"). Companies are now taking advantage of the market dynamics and reselling their GPUs to the public. Companies can recoup 100% of their hardware investment at current rental prices in less than 12 months.

These two reasons are why we built Hydra Host to take advantage of the next phases of the global bare metal market. We make it super easy to manage and sell GPUs with our bare metal marketplace -> www.hydrahost.com