This article is from the free weekly Barron’s Tech email newsletter. Sign up here to get it delivered directly to your inbox.
AI Revolution. Hi everyone. The rising demand for artificial intelligence is boosting demand for high-performance server hardware.
But there are two large limiting factors in this space: getting access to Nvidia (ticker: NVDA) graphics processors that power the most advanced AI models and the ability to quickly incorporate the latest technologies into server designs. Super Micro Computer (SMCI) has been ahead of the game on both fronts.
Founded in 1993, the firm has become the leading independent manufacturer of high-end AI servers. Its server racks fill data centers across the country, helping to power some of the most in-demand AI workloads.
In its latest quarter ending in June, revenue jumped by 70% versus the prior three months as Super Micro saw significant demand for its large language model optimized AI systems, featuring Nvidia’s best chips.?
More than half of the company’s revenue is now tied to AI, with customers ranging from start-ups and enterprises to major cloud computing vendors. According to a recent Barclays report, Tesla, Meta, and Coreweave are Super Micro customers.?
Barron’s Tech recently spoke to Super Micro CEO Charles Liang about the state of the AI market, his company’s relationship with Nvidia, and how durable the demand for AI hardware will prove to be in the coming years.
Here are edited highlights from our conversation with Liang:?
Barron’s: Describe Super Micro’s business and why customers choose your servers over HP or Dell.
Liang: Super Micro makes products based on a building block solution, which is modularized. It makes it easier for us to design products quicker and have a faster time to market. The modules help optimize the system configurations for customers’ workloads and data center environments. That’s why we can bring new technology to market earlier than others—usually by two to six months.
The other thing is we deliver a rack-scale product. We install our hardware servers, storage, [networking] switch, security, and management tools in a rack. So when a customer receives our rack, they plug in two cables—a data cable and a power cable—and are ready to run. That’s why customers like our solutions. They are turnkey.
Liquid cooling, as a replacement for more traditional fan-based cooling, is a big trend for AI servers inside the data center. Why is this happening?
Yes, liquid cooling becomes helpful in a number of areas. Number one is to save on TCO [total cost of ownership] by reducing energy power costs. That savings can be up to 30% or even 50%.
Second, it makes the system more reliable. There is less heat. And third, it allows the customer to always run their system at 100% performance [no throttling due to overheating]. Some data center environments, especially at large cloud service providers, have some thermal constraints.?
Finally, many data centers are also constrained by power budgets. As power consumption is reduced with our solution, they can install more systems given their current power capacity. This is especially important with generative AI, which needs more and more machines.
What is happening in the AI server market, and how durable is the current demand?
The demand is very strong. Back-order has been growing quickly. The only limitation is that we need chip companies—especially Nvidia—to support us with more chips.
We prepared in advance. Eighteen months ago, we tripled our high-power, high-performance system, and rack production capacity. Today, we have a production capacity of 4,000 high-capacity racks per month in Silicon Valley alone. I believe that’s the largest scale in the world today.
We are continuing to expand our capacity. By March of next year, our capacity will double again.
How close is your relationship with Nvidia? You have mentioned Super Micro and Nvidia both started in 1993.
We have been partners, almost since day one in 1993. Since we started to work together the relationship has become closer.
One good thing is the company headquarters are only about 15 minutes away from each other. That enables the teams from the two companies to work from early morning to midnight. That’s a geographical advantage.
We appreciate their very deep support.
Are you confident you’ll get good GPU supplies from Nvidia in the future??
Yes, I have a very optimistic belief. They are working hard. They are working smart. We are happy to see their progress. Their throughput will continue to grow.?
Is being based in California an advantage in working with your customers?
It’s a big help because we can work with our supplier partners, like Nvidia, Intel, and AMD. At the same time, we work with customers closely. Customers can send their experts to our site during the design, validation, and production stages.?
What type of AI workloads and AI applications are in highest demand among your customers?
Demand started to be very strong last year, with customers training systems around deep learning, generative AI, and large language models.
And now inference applications [workloads] are almost everywhere for every vertical. We have hundreds of customers. They are still waiting [for server hardware].
That’s why we continue to expand our capacity. We just need more supply from our chip companies. The customers are there.
I believe this AI revolution can be bigger than the Industrial Revolution two hundred years ago.?The impact is everywhere. It will affect education, entertainment, and how people plan, think, and design.
This wave can be very big and has many years to go.
Thanks for your time, Charles.
This Week in Barron’s Tech
- Google’s Digital Assistant Gets Smarter, Teaming Up With Bard
- Meta Unveils New AI Tools for Advertisers
- Zoom Is Taking On Microsoft and Google With Zoom Docs
- Amazon Stock Is Under Pressure. Why Two Analysts Think You Should Be a Buyer.
- Nvidia and Intel Are Bright Spots in a Soft Chip Market, Analyst Says