Aaron Ginn | Authors | Hydra Host

Aaron Ginn

There are two types of moments in history: those that feel like nothing happens and those that feel as if the world changes multiple times over in a single lifetime. The last decades have seen the rise of mega tech platforms that promised obvious efficiencies with few downsides. These platforms changed our entire society. What was promised was more choice and freedom for all with lower prices and what we got was an assault on our core identity as a society. Today, we have a choice — what do we want our future to be? In the past decade, the single greatest accelerant of societal change was mega internet platforms. With widespread adoption and near infinite scale, big tech platforms were sold as a public good and innovation free from moral responsibilities for every consumer and business to benefit from. On the surface, things appeared harmless and basically free, like selling books or providing information. Underneath their full ownership of the core infrastructure stack for the free and open web, from domains to computing power, lurked increasing lock-in with walled gardens that grew into a power that was destined to be abused. Without a check, this type of absolute power will be corrupted.

<blockquote>“Power tends to corrupt, and absolute power corrupts absolutely.” — Lord Acton</blockquote>

Then, COVID changed everything and brought light to the brewing darkness that had been brewing behind the meticulously designed and crafted walled gardens. The mask was taken off, so mega-tech platforms attempted to mask the world. They executed a near autocrat-level control over global society at once. Convenience and subsidized lock-in with “free services” turned into a society-wide hostage situation. The wolf was in the hen house. With scale, came brutal centralized control that threatens every citizen and business. With COVID, quiet conversations about censorship that only impact the fringe and politically involved now reached the mainstream. What previously impacted the 1% now suddenly impacted the other 99%. The world was in shock at the speed and intensity of this change. Business owners were thrown into jail for normal operations. Dissent led to mob-like censorship the likes infamous dictators would have been in awe of and public witch burning to unperson the heterodox made a massive comeback. People were literally arrested for surfing in the ocean and walking down the street without a cloth mask on. I was a victim of the thumb of mega tech platforms. I&nbsp;<a href="https://www.wsj.com/articles/controlling-the-virus-narrative-11584899715" rel="noopener noreferrer" target="_blank">wrote a piece in March of 2020</a>&nbsp;that was the first seminal criticism of lockdowns and COVID hysteria. It garnered millions and millions of views in less than 24 hours. My core argument warned that our politicians were taking things way too far and the evidence did not justify the rash and immoral dilution of basic freedoms. Then, I suffered a fate like many others —&nbsp;<a href="https://www.wsj.com/articles/the-lockdown-skeptic-they-couldnt-silence-11589566245" rel="noopener noreferrer" target="_blank">I was canceled</a>. My predictions and assessments were accurate, yet my access to the internet was revoked. I was deemed a thought criminal and put into internet jail for expressing precise and well-supported views. For the length of COVID hysteria, we watched the free and open internet suffocate under the weight of mega tech platforms. These platforms were founded to achieve scale but did so with a cornerstone of lock-in and centralization. These values prohibited the abilities of companies and people to freely operate and utilize the open web when it mattered the most. We asked for the promise of a free and open internet. Instead, we got an elimination of customer choice and mega tech profiting from lock-in by bundling sub-par products and abusing power without a second thought. Despite how much society is opposed, mega tech platforms have become too big to fail. It is time to change that. The world needs a new social contract with technology companies that won’t take away power but instead enables freedom, choice, sovereignty, and dignity. The world needs a new type of infrastructure platform that follows the initial vision of a free and open web with a renewed promise to ALWAYS give you choice and privacy for whatever your needs or priorities are. This is why I started&nbsp;<a href="http://www.hydrahost.com/" rel="noopener noreferrer" target="_blank">Hydra Host</a>&nbsp;with two longtime friends, Garrett and Ariel. Hydra Host is here to offer&nbsp;real&nbsp;customer choice in hosting, web infrastructure, and most importantly, compute. Infrastructure is the heartbeat of how the internet works and is fundamental to the social contract between platforms and users. Infrastructure is how every company or individual accesses the web. Without sound infrastructure, you could be cut off from your right to the web. Hydra Host is here to fix that. Let’s be real. No one really likes their hosting provider. The public cloud failed. It can’t do everything you want. Bills are confusing. In your gut, the price feels out of proportion compared to what you receive. There is constant pressure from providers to pull you further into their toolchains so you can’t leave. Their “free credits” feel like a gateway drug because you are always hearing about the better hosting provider down the road. Your infrastructure provider should not control your business or control you. <blockquote>YOU should be in charge so YOU can do choose to do what is best for YOUR company and YOU should never be concerned about your cloud infrastructure provider deciding your company can’t do business.</blockquote> From Hydra Host’s founding moment, we have followed the best security and privacy practices and adopted cutting-edge agnosticism and open-source frameworks.&nbsp;Our vision is for any user to have the ability to access any infrastructure provider in the world, whether for cost or security reasons, with a single API.&nbsp;At any time, you can decide to leave any provider or try a new provider. No refactoring. No complexity. Freedom to choose, freedom to move as easy as a click. The choice is yours. Just like with Kayak or any other marketplace, you are in the driver’s seat. Hydra Host has standardized and simplified every point of the provisioning process. Each hosting company is a common and uniform node that you can access in a simple and single programmatic framework. With our API, we prevent you from getting locked in ever again. Our commitment with Hydra Host is to constantly deliver the best hosting possible while respecting our customer's and client’s privacy. We believe the internet should be accessible to every single person and company regardless of background, creed, faith, or tradition. You can test our initial Terraform provider API for yourself —&nbsp;<a href="https://github.com/Hydra-Host/terraform-provider-hydra" rel="noopener noreferrer" target="_blank">https://github.com/Hydra-Host/terraform-provider-hydra</a> As&nbsp;<a href="https://twitter.com/get_hydrahost/status/1501280353838792706?s=20&amp;t=DLKQggKEfJyXdRyLpU8t3g" rel="noopener noreferrer" target="_blank">we allow mega tech companies to control more and abuse their power</a>, our ability to see what humanity can truly do is limited. We are choking humanity’s greatest asset for survival — creativity. The future is too important to let a handful of mega-tech companies determine what we’re allowed to do. It is harder than ever to see where the future could take us. At Hydra Host, we’re paving the way and working to ensure your right to the free and open web doesn’t ride or die on the whims of a random tech platform. With Hydra Host, we believe your future should be in your own hands. <a href="https://www.hydrahost.com/" rel="noopener noreferrer" target="_blank">Join us</a>. Choose liberty and not tech tyranny. The time is now to storm centralized infrastructure and seize the future we want, for the people and the free and open web.

<blockquote>“A revolution without dancing is a revolution not worth having.” — V for Vendetta</blockquote>

Explore how centralized cloud hosting threatens internet freedom and how Hydra Host champions user autonomy and choice.

The dangerous remedy to the desperate disease of cloud hosting

This was a wild week for the GPU market. Brand new export restrictions caused a tumultuous week. Additional limitations on future H100 orders are expected. The steady pricing from previous weeks was thrown out the window. Here is what my team at&nbsp;<a href="https://www.linkedin.com/company/hydrahost/" rel="noopener noreferrer" target="_blank">Hydra Host</a>&nbsp;is seeing. New export restrictions lay down the hammer - The US government continues to turn up the heat on advanced computing chips with export chips. The looming new rules released this week are substantial and include additional manufacturing equipment restrictions and expanded qualifications. The new proposed regulations made “Chinese” NVIDIA chips irrelevant in design and, at the same time, valuable. These chips were essentially designed to evade export restrictions, and the new rules significantly expanded their scope and authority (see images below). Almost all advanced chips from NVIDIA and some gaming chips qualify. There are some exceptions to be more targeted, but the US government’s explicit goal is enterprise data centers. The US government has capped China’s computing power, forcing it to accelerate its internal development. The black market for GPUs will aggressively develop over the coming year. I wouldn’t be surprised if GPUs become like a new drug trade. Slapping OEMs selling only cards - NVIDIA is tracking down OEMs selling GPU cards rather than building complete systems. NVIDIA aims to expand its footprint as fast as possible in data centers worldwide, and this under-the-table market is slowing it down. It is also exacerbating the shortage, so we heard NVIDIA removing allocation to OEMs, continuing to sell just chips and not building servers.

Buy. Buy. Buy whatever you can get - From these looming restrictions from the US government and NVIDIA applying more pressure to GPU-as-a-service vendors, there is a renewed aggressive buying spree. Some companies are hoarding to resell in the future, while others are concerned that nothing will be left. A handful of companies are scooping every H800 they can find, betting that the same thing that happened to A100s will happen to H800s. We saw new and used A100 supply disappear the same day. H100 servers moved faster than usual as companies expect future price increases. GPU prices will increase soon - Due to restricted supply and higher demand, prices are going up again. Orders in November and December are already higher than in October. We expect H100s to have the most considerable price increase. Resellers are also increasing their prices due to fewer buyers and more shortages. Feels like we are in a prison yard-style market again. We have supply today if you need A100, H100, or other GPUs. Ping <a href="mailto:sales@hydrahost.com" rel="noopener noreferrer" target="_blank">sales@hydrahost.com</a> for availability. We can help find any GPU for you, whether renting or buying.

Learn about the latest U.S. export restrictions on GPUs and their implications for the tech industry.

GPU Chip Restrictions

The joke within the venture community is that they are not funding AI companies but funding infrastructure to buy NVIDIA chips. <a href="https://www.theinformation.com/articles/andreessen-horowitz-discussed-backing-a-midjourney-rival-seeking-500-million-valuation" rel="noopener noreferrer" target="_blank">Investing in AI companies </a>today is just a direct transfer to NVIDIA. Due to GPU demand, that is true today, but supply and pricing pressures on HPC infra will evolve the market in the next 12 months.

DCs are going direct and selling wholesale. Before the GPU revolution, there were generally only two hosting options - hyper-scalers or colocation. That is changing now due to the cost of GPUs and limited supply. Pre-GPU revolution, CPUs were highly commoditized, and you needed a massive scale to have any competitive advantage. It didn't make sense for most companies to expend capital on a commodity bare metal. The world today is changing quickly. GPUs vastly differ from CPUs: compact, immensely powerful, expensive, and not commoditized. In this new world, DCs are creating a new business line by bypassing cloud companies and renting GPUS directly to companies. Colocation space that would have previously gone to the public cloud or big corp colocation will go to the rent-a-GPU business to make cloud-like margins. Data centers are even adding financing capabilities to get more GPU business. Going direct saves AI or platform companies 15-20% on their computing and hosting costs.

Companies are starting to resell their GPUs to others. HPC hardware is being over-acquired and underutilized, but the reasons are similar to the situation with Web 1.0 companies (which spawned the public cloud). Due to infra limitations, Web 1.0 companies needed extra bare metal capacity to scale for something as simple as visitors. You had to build your sovereign infra to handle traffic spikes. Today, the market situation looks familiar. Companies need bursting capacity to massive clusters for better training runs. Rather than waiting for an unknown period of time, constantly hunting for capacity, or having your training run cut off by another customer, companies are sitting on underutilized hardware or inefficiently using GPUs ("Well, it is there, so might as well use it"). Companies are now taking advantage of the market dynamics and reselling their GPUs to the public. Companies can recoup 100% of their hardware investment at current rental prices in less than 12 months.

These two reasons are why we built&nbsp;<a href="https://www.linkedin.com/company/hydrahost/" rel="noopener noreferrer" target="_blank">Hydra Host</a>&nbsp;to take advantage of the next phases of the global bare metal market. We make it super easy to manage and sell GPUs with our bare metal marketplace -&gt;&nbsp;<a href="http://www.hydrahost.com/" rel="noopener noreferrer" target="_blank">www.hydrahost.com</a>

 Explore how data centers are reshaping the bare metal market by renting GPUs directly, enhancing performance and reducing costs.

Next Phases of the Bare Metal Market

We are on the horizon of the inevitable shift towards GPU-dominated infrastructure, and this movement will build several unicorn companies. <a href="https://www.reuters.com/technology/crypto-billionaires-nonprofit-buys-500-million-ai-data-center-chips-2023-10-30/" rel="noopener noreferrer" target="_blank">Voltage Park</a>, a soon-to-be billion-dollar H100 provider, shows the momentum of the shift towards GPUs. Accessible GPUs are going to transform the infrastructure of the web. As Jensen says, “Moore’s law is dead,” and it is time for the next generation of computing infrastructure. CPU-based public cloud operations can’t move quickly and capture this demand for GPUs because they have built their entire business model on low-cost, low-power, commoditized CPUs. GPUs are energy-dense and require more advanced cooling than CPUs. GPU-specific suppliers, like Voltage Park, are designing their operations and infrastructure to support these requirements. These new types of suppliers are capturing this growing demand while the market waits for overpriced public cloud-supplied GPUs. The next unicorn companies within AI are infrastructure companies. Demand is only increasing as large enterprises are now beginning to adopt AI and LLMs. NVIDIA is sold out for years to come. The GPU deficit, combined with the public cloud’s subpar design for GPU operations, has pushed customers to alternatives like Voltage Park and the secondary market. The nascent secondary market for GPUs is a forcing function of the annual billion-dollar shortage of GPUs. The supply and pricing you get from system integrators and direct relationships with NVIDIA are consistent and stiff, just like buying a new car. You aren’t going to get a fantastic deal anymore from OEMs or integrators. NVIDIA has a tight grip on every single hardware builder. From shortages and high demand, a new bare metal marketplace is here. When you need a car, the inevitable choice is presented: new or used. Buying a new car is a relatively standard and inflexible process with little room to make a deal. If you find a deal, it usually comes from the manufacturer as a financial incentive to the dealer (ergo, NVIDIA Inception Program). Typically, the dealership has a minimal margin on new car sales. The used car market, on the other hand, has a higher variance between cars, making it harder to price consistently. Therefore, good deals exist predominantly in the secondary market. <a href="https://www.linkedin.com/company/hydrahost/" rel="noopener noreferrer" target="_blank">Hydra Host</a>’s platform supports this new secondary market for GPUs, and we are seeing many massive and insane deals. Rising prices and endless demand have given birth to a bare metal marketplace. Like with cars, pricing is better on the secondary market. Some suppliers bought early and can run GPUs 30% cheaper than the public cloud. Data centers are refurbishing GPUs and running them at half the price. Resellers are buying cards on cheap and holding them. If you want to see what is happening in the GPU market, sign up at Hydra Host and see what is happening.

Discover how the transition to GPU-centric infrastructure is creating new unicorn companies and transforming the tech landscape.

The Inevitable Shift

The GPU computing wars aren't just between AMD, Intel, and NVIDIA. This war isn't just between the imminent public cloud GPUs and NVIDIA. Nation-states are joining the mix because computing power and AI will define the next ten years, and pre-existing geo-political trends will push the GPU market to fragment.&nbsp;&nbsp;One of the unintended consequences of US restrictions on GPUs has been the rapid escalation of the <a href="https://www.theinformation.com/articles/chinese-regulators-tell-local-tech-firms-to-buy-fewer-nvidia-chips" rel="noopener noreferrer" target="_blank">GPU arms race</a>. These restrictions have pitted countries against each other and against international cloud companies and NVIDIA. The US government's <a href="https://www.reuters.com/technology/upcoming-us-rules-ai-chip-exports-aim-stop-workarounds-us-official-2023-10-15/" rel="noopener noreferrer" target="_blank">decision to annex NVIDIA</a> as a national security asset, a move many consider unconstitutional, has further fueled this race, undermining the world's confidence in NVIDIA's ability to transact freely.&nbsp;&nbsp;The fear of falling behind in the AI race is about more than losing technological prowess. It's a fear that power, stability, and even national defense could be at stake. The concept of "Bring your own GPU" (BYOG) is positioned to dominate the global computing race, leading to a rapid fragmentation with prevailing players in both the hardware and GPU rental space. This global GPU footprint will mirror broader geo-political relationships that defined the world pre-H100 and A100. Moreover, the potential for several countries to misuse AI against their own citizens is a reality we cannot ignore.&nbsp;

At Hydra Host, we are betting on data centers. A bet on data centers is a bet on NVIDIA. As competition for accelerators and GPUs heats up, NVIDIA will return to its reliable moat and bread and butter: data centers. As the public cloud pursues its budget GPUs and nation-state subsidies for commodity GPU alternatives, NVIDIA will remain the king and continue growing its data center footprint as the data center industry has no interest or capability of building its own GPU. You can see this clearly with NVIDIA's push towards fully incorporated builds like the GH200 and GB200.&nbsp;&nbsp;GPUs, being naturally agnostic and with a limited-to-zero need for associated cloud services, broke the back of a public cloud. There was no longer a need to stay locked in with these mega-cloud companies. GPU sets the computing world free. As a result, priorities have shifted, and non-GPU considerations are driving the global battle for data center capacity and GPU computing.&nbsp;&nbsp;Aaron Ginn, CEO

Explore the GPU computing wars shaping AI and geopolitics. Learn how NVIDIA, cloud providers, and nation-states compete for global GPU dominance.

Navigating the GPU Computing Wars

<h1>Introduction: The Anticipation of Blackwell&nbsp;</h1> In the fast-paced world of high-performance computing (HPC) and artificial intelligence (AI), NVIDIA’s upcoming <a href="https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/?ncid=pa-srch-goog-267462&amp;_bt=709953316733&amp;_bk=nvidia%20blackwell&amp;_bm=p&amp;_bn=g&amp;_bg=169122792928&amp;gad_source=1&amp;gclid=EAIaIQobChMI-vbrv9aGiAMV2zWtBh1nQipsEAAYASAAEgIT6vD_BwE" rel="noopener noreferrer" target="_blank">Blackwell architecture</a> is the next big leap everyone is eagerly awaiting. Blackwell is expected to deliver unprecedented breakthroughs in performance, energy efficiency, and scalability, set to redefine AI, HPC, and data center applications. But with Blackwell's release still on the horizon, businesses can't afford to wait. So, what are the best GPU alternatives to rent in the meantime?&nbsp; <h2>The Waiting Game: Why You Need GPU Solutions Now&nbsp;</h2> The anticipation around NVIDIA Blackwell is palpable, but waiting doesn't mean your AI and HPC projects should stall. In today’s competitive landscape, staying ahead means accessing powerful compute solutions right now. Fortunately, Hydra Host offers several GPU rental options that can seamlessly bridge the gap while you await Blackwell's official release.&nbsp; <h2>Current Alternatives to Blackwell: What to Rent Now&nbsp;</h2> <h3>1. <a href="https://hydrahost.com/nvidia/h200/" rel="noopener noreferrer" target="_blank">Commit to H200</a>&nbsp;</h3>The NVIDIA H200 GPU is the next evolution from the H100, delivering even more power and performance for demanding AI and HPC applications. While it comes at a slightly higher cost than the H100, the H200’s advanced architecture and enhanced capabilities make it ideal for businesses that need top-tier performance without compromise. This GPU ensures that your AI, machine learning (ML), and data processing tasks are completed with unmatched efficiency, keeping you ahead of the competition.&nbsp;&nbsp;<ul><li>Ideal for pushing the boundaries of AI and ML.&nbsp;</li><li>Suitable for running complex simulations.&nbsp;</li><li>Effective for training large-scale models.&nbsp;</li><li>Handles massive datasets with unmatched efficiency.&nbsp;</li><li>Ensures top-tier performance for businesses requiring no compromise.&nbsp;</li></ul> <h3>2. <a href="https://hydrahost.com/nvidia/grace-hopper/" rel="noopener noreferrer" target="_blank">On-Demand GH200s</a>&nbsp;</h3> Renting on-demand GH200 GPUs gives your business the flexibility to scale resources as needed, without the burden of long-term commitments. At just $4.99 per card on the <a href="http://brokkr.hydrahost.com/" rel="noopener noreferrer" target="_blank">Brokkr Marketplace</a>, this cost-effective option is ideal for businesses that need to stay agile and ready for Blackwell’s release.&nbsp; <ul><li>Perfect for prototyping AI/ML models.&nbsp;</li><li>Useful for testing new algorithms.&nbsp;</li><li>Prepares workloads for future transition to Blackwell.&nbsp;</li><li>Allows for nimble operations with flexible scaling&nbsp;</li><li>Cost-effective option at $4.99 per card on Brokkr Marketplace.&nbsp;</li></ul> <h3>3. <a href="https://hydrahost.com/nvidia/h100-pcie-80-gb/" rel="noopener noreferrer" target="_blank">H100 Tensor Core GPUs</a>&nbsp;</h3> The H100 Tensor Core GPUs remain a top choice for AI and deep learning tasks, renowned for their exceptional performance. With some H100 clusters coming off contract, businesses can seize the opportunity to access these powerful GPUs at competitive on-demand pricing or through attractive long-term deals, particularly suited for inference workloads. This flexibility allows you to efficiently scale your AI operations while optimizing costs.&nbsp;<ul><li>Excellent for training and inference in deep learning models.&nbsp;</li><li>Ideal for natural language processing (NLP) tasks.&nbsp;</li><li>Supports advanced scientific computing.&nbsp;</li><li>Suitable for businesses needing short-term rentals or long-term commitments.&nbsp;</li><li>Offers reliability and power to maintain momentum in AI projects.&nbsp;</li></ul> <h3>4. <a href="https://hydrahost.com/nvidia/a100-pcie-40-gb/" rel="noopener noreferrer" target="_blank">A100 GPUs</a>&nbsp;</h3> The A100 GPUs, part of NVIDIA’s Ampere architecture, continue to be a reliable workhorse for AI, data analytics, and HPC tasks. These GPUs offer an excellent balance of performance and cost, making them a savvy choice for businesses that want to maintain momentum without overspending while waiting for Blackwell.&nbsp;<ul><li>Well-suited for both training and inference workloads.&nbsp;</li><li>Ideal for AI, data analytics, and HPC tasks.&nbsp;</li><li>Provides robust power without a hefty price tag.&nbsp;</li><li>Cost-effective bridge to the future NVIDIA Blackwell architecture.&nbsp;</li><li>Maintains project momentum during the wait.&nbsp;</li></ul> <h3>GPU Comparison Table&nbsp;</h3> Here’s a detailed comparison of the H200, GH200, H100, and A100 GPUs to help you choose the best option based on your project needs:&nbsp;

<h1>Why Renting Makes Sense Now&nbsp;</h1> <ul><li>Avoiding Obsolescence: Why invest heavily in technology that may soon be outdated? Renting GPUs allows you to avoid large expenditures on hardware that could lose its edge once Blackwell is released.&nbsp;</li><li>Flexibility and Scalability: Renting GPU resources provides the agility your business needs. You can scale up or down based on project demands, without being tied down, paying only for what you use.&nbsp;</li><li>Preparing for the Future: Renting GPUs now allows your team to continue refining projects, so when Blackwell finally arrives, you’re not just catching up—you’re leading the pack in AI and HPC innovation.&nbsp;</li></ul> <h1>Conclusion: Stay Ahead of the Curve&nbsp;</h1> In the rapidly evolving AI/ML landscape, standing still is not an option. Renting GH200s, H100s, or A100s is a strategic move to keep your business moving forward during the wait for NVIDIA Blackwell. It’s about positioning yourself to leverage future advancements, not just survive them.&nbsp; Explore the comprehensive GPU rental options available in the <a href="http://brokkr.hydrahost.com/" rel="noopener noreferrer" target="_blank">Brokkr Marketplace</a> today. <a href="https://meetings.hubspot.com/ryan-horjus" rel="noopener noreferrer" target="_blank">Schedule a consultation</a> with our Lead Sales Engineer, Ryan, and find out which setup best meets your needs, ensuring you’re ready when Blackwell arrives.&nbsp;

Discover top GPU rental options like H100, GH200, and A100 while waiting for NVIDIA Blackwell. Optimize AI and HPC workloads today.

What to rent while waitign for NVIDIA Blackwell? 