As I type this, NVDA has a market cap of $4.44 trillion. In terms of competitors, AMD and Intel are each only worth about 8% of that. Broadcom at about $1.4 trillion can at least sit at the same table.
But why? The market must think that there’s no competitor that is even close in the chip/GPU/AI arena. Why is there so much confidence that none of its current or future competitors, or China or other countries, will be able to compete with it technologically?
I suspect that many of Nvidia’s employees are being lured away, and bringing knowledge of its core competencies with them. The market “knows” this, but still seems to be confident that NVDA will remain dominant.
Can someone explain (in non-tech terms, if possible)?
People believe AI is going to massively change the world and Nvidia dominates the AI data center market (with an 85% or greater share). And just like Windows dominates the PC operating system market (and has continued to do so for decades), the Nvidia’s software tools provide a somewhat similar moat in the AI industry.
And anyone wanting to come into the market with a competing infrastructure, not only for the GPUs but also CUDA, has a huge hurdle to overcome. It would be billions of dollars of investment in tools, software design patterns, and chip fab just to try to compete with no guarantee of being able to unseat Nvidea without some special competitive advantage in speed or functionality.
All that being said, Nvidea is way overvalued (because the generative AI market is almost a complete bubble with no real revenue stream or business plan) and CEO Jensen Huang knows it. He’s making hay while the sun shines but the industry is either going to run into fundamental resource constraints, computational limitations (it’s already scrambling on that progressively steeper slope), or investors are just going to stop dumping tens of billions of dollars into it without seeing revenue or a real profitable product that could somehow recoup their investment.
I was on reddit yesterday, and someone referred to AI as the new gold rush, and someone else said, that made Nvidia the merchants who really got rich off the gold rush, by selling mining equipment and supplies to prospectors.
The market is correct in this assessment. AMD, and even Intel, may have GPUs that are competitive with Nvidia for gamers. However, Nvidia’s current stock price is not based on selling an $800 video card on Newegg so you stop dropping frames; it’s based on selling a datacenter worth of GPUs at $40,000 (or more) each so that AI growth can continue.
As said, AI is currently locked into Nvidia’s software stack, CUDA. Even if AMD or someone comes out with faster and cheaper GPUs for AI, converting existing software from CUDA to ROCm (AMD’s software stack) is a major undertaking.
Even if AI is a huge bubble (which it almost certainly is), billions of dollars are being sunk into it. That money isn’t just lit on fire in datacenters, what doesn’t find its way into Sam Altman’s bank account makes its way to companies like Nvidia, that are selling the shovels.
Nvidia being overvalued is really a question of how long you think the AI bubble will last. If AI gets an additional trillion dollars invested before it busts, a big piece of that will find its way to Nvidia.
There are limited places to park your money. The market highs make little sense given unprecedented volatility and likely inflation. AI is trendy and people think it will change the world. While this is true, most of its uses will be mundane, some will lower employment and some will be antisocial.
The reason, then, is a combination of hope, hype, extrapolating what AU has already done, a lack of alternatives and imagination, and FOMO on the next “Interwebz”.
Unless a major government gets involved, there isn’t US$1T of investor money out there to plow into AI. There probably isn’t even a tolerance across the investment landscape to pour in US$100B into AI development and training. People are looking pretty hard at Softbank’s investment of US$40B into OpenAI and Microsoft putting US$80B in CapEx (not all AI; most is ‘cloud’ resources and data centers as well as supporting CoreWeave). Although the “Stargate Project” is loosely budgeted out at US$500B it is difficult to see where that money is going to come from unless the Emirati investment fund MGX fronts the bulk of the money. Larry Ellison isn’t going to sink that kind of capital into OpenAI without a viable product with demonstrable ROI. Nvidea is riding the AI hype wave, and they really have nothing to lose as long as they don’t try to build out for capacity that is not going to be called upon.
While that is likely true, it won’t be in the ways that most speculators (and certainly enthusiasts) expect, and is unlikely to create durable value in the market over the long term that would justify all of the investment so far, much less the tens of billions more than people will plow into it. ‘Agentic AI’ will probably happen at some point (assuming the basic reliability problems can be addressed and the ‘attention window’ can be made sufficiently large and robust, which are big question marks) but it isn’t going to be AGI much less ‘superintelligence’ along the current feed-forward and reinforcement training approaches; it’s just going to be a kind-of smart assistant that you wouldn’t trust to manage any kind of large scale complex system without a person in the loop (a lesson we will likely learn after enough people try to implemented it in that fashion and discover the limitations, hopefully not with critical infrastructure or autonomously managing strategic weapon systems).
I used to work for Intel back in the late aughts and early 2010s. Although I wasn’t directly involved in the program, at that time Intel was trying to develop a GPU to directly rival Nvidia (code named Larrabee). That project ultimately was canceled, because to put it simply, designing a GPU is actually really hard, and knowledge from designing CPUs doesn’t directly transfer to designing a GPU. More specifically, as I understand it, designing the hardware was easy enough (for a company with the skills to design processors), designing all the stuff that goes with it like the device drivers to actually make it work as a graphics chip proved to be really difficult – at that time Intel assumed their main market would be high-end gaming graphics cards.
That said, the Larrabee architecture did get repurposed as the Xeon Phi line of processors, marketed as ultra high end CPUs for high performance supercomputers. But they never sold that well, and were discontinued in 2020, just before the whole AI revolution really took off. If does make you wonder if they had kept at it maybe they could have at least somewhat competed with Nvidia in that market. But I suspect Intel’s Xeon processors were more expensive than Nvidia’s GPUs.
Aside from CUDA, one of the key pieces of tech that NVIDIA has is the interconnect technology that they acquired when they purchased Mellanox. This was underestimated by almost everyone. But it meant that if you wish to build a $1B or $10B AI datacenter, you almost can’t go with anyone but NVIDIA. Someone else can sell you a pile of GPUs but they won’t be able to communicate effectively.
For some problems, this doesn’t matter too much–there are problems where each computational unit is basically independent, and just needs to share some results at the end over a slow link. But this is largely not true of AI workloads.
Nvidia isn’t THAT overvalued compared with other tech companies wrt to their revenue. They’re valued very high, but they also make a ton of money. It’s not all hype about the future. Doesn’t mean they’re not overvalued.
GPUs are useful for basically all scientific computation. AI and cryptomining are things that could very well decrease in the future, but the need for amazingly fast matmuls isn’t going away anytime soon.
That’s not entirely true; AMD has their Infinity Fabric which is intended to be a highly scaleable interconnect architecture. How comparable that is to Nvidea’s NVLinkn in performance I don’t know, but all of your application code (whether generative/LLM AI, or fluid/climate simulation models, or MCMC financial projections for Value-at-Risk estimates) needs to be able to operate efficiently at the base hardware level to vectorize processes with minimal overhead, and the investment in CUDA-based data traffic management and the associated APIs is a deep sunk cost that most AI developers aren’t going to back out of without some real compelling performance gain.
There are competitors, to be sure, but no one else seems to have a solution that actually works at all scales as an integrated whole. It has to be effective for two chips on the same PCB, or in the same rack unit, or in the same rack, or in the same room, or in the same building (with decreasing levels of performance, of course). And then the software support such that these things are seamless. Which integrates into the CUDA advantage as well.
It’s not that the competitors don’t realize these things of course, but the level of execution isn’t there.
Agreed; it’s a lot of pieces to put together that would require enormous investment with no assurance that it will draw significant market share away from Nvidea unless it offered some really unique advantage. Nvidea recognized this long before anyone else, albeit for high performance computing for simulation, not AI processing, but from a computing standpoint its essentially the same type of problem.
Oops, I was off by an order of magnitude. Maybe $100 billion before the bubble bursts? OpenAI will need about $40 billion per year to survive. The money that doesn’t come from investors is going to have to come from customers, and they’re only making about $4-5 billion per year in revenue.
Quiet! If they hear you, the bubble might burst.
Either way, that is still billions being fed into Nvidia, who has actual products with actual use cases.
Yeah, but there is only so much money governments will pour into ‘compute’ for more elaborate global climate circulation models, especially now that we’re going all in in denial of climate change. Maybe IB and fintech will step in with hyperfinancialization of blockchain-based securities and futures predictions with hybrid quantum computing, or whatever bullshit Satya Nadella spews out next quarter to maintain investor enthusiasm for the increasingly leaky Good Ship Microsoft. I can’t wait.
All the big quant funds are definitely big GPU users. I don’t know if they get AWS to set up a secure server for them like they do for the US government or if they airgap them in house. I’ve also heard Jane Street had built their own ASICs, but don’t know how common that is vs commodity chips.
Not fabs. Nvidea parts are being made at TSMC, and it might be hard to get fab time for a competitor but you don’t have to build a new fab. Which, I agree, would be impractical.
I agree that we’re at the stage similar to the early internet where every mom and pop flower store put an “e” in front of their name and cashed in - until they didn’t. Most of the AI businesses are going to go down hard. But who would have thought the killer app for the net was getting crap delivered to your door in a day. Maybe someone will find something that AI does well. Or can do well.
The notion of having stuff delivered to your door wasn’t new; mail order businesses had been around since the before the turn of the 20th Century but it took weeks to get your stuff delivered. Jeff Bezos’ original innovation was in the logistics of the supply chain and delivery system in being able to deliver a wide array of products to customers within 2-3 days. Later, Amazon also became a ‘tech’ company that collected and applied user data to optimize pricing and create targeted marketing, and then pivoted to the profitable AWS, but the original business was just optimizing an existing concept for maximum convenience, cannily betting that if you make shit cheap enough, people will compulsively buy it whether they need it or not.
AI (in the form of deep learning systems) does plenty of things well; I’m just dubious about the use case for language models and so-so generative tools that justifies hundreds of billions of dollars of capital investment. I know people think they’re going to be able to make their own movies with vidgen tools until they realize that the vast majority they don’t actually have the creativity or drive to make something novel and interesting, and unlike Star Trek holodecks ‘the computer’ can’t just intuit the exact thing you want from a couple of vague statements. I assume that it’ll become apparent at some point that these tools have a certain utility in the hands of a skilled user as aids to creativity or as an agentic tool with constrained parameters but we’re going to get a lot of enshittification in the meantime, or maybe everything will just be awful and derivative from now on with little incentive to take the risk of creating anything unique.
(You’ll have to pardon my cynicism but I just got done watching Alien: Romulus which was a perfectly well shot and acted movie with story, characters, plot complications, and other elements essentially copied from previously films and assembled together in a kind of random fashion with little craft, and had more ‘endings’ than The Return of the King, so apparently we don’t even need “AI” to make slop.)
I had a point when I started this but it has drifted off topic like a conversation with ChatGPT which is now advising them to shoot myself in the foot just to see if I can feel anything.
That’s the key. I got stuff from catalogs as a kid, though I lived in a place with plenty of stores. Getting something tomorrow is a game changer in that you get your stuff as quickly as you would if you had to plan a trip to a store.
I predicted immediate delivery (through a teleporter) in a story I wrote for my high school science magazine in 1968. The downside is that upgrades came and people got buried in stuff. Looking at some garages near me, I think I might have been pretty accurate.
I also want to make the case that maybe Nvidia, the company, is better run than most tech places. They’ve been around for quite a while — started in 1993, which makes them older than Google — and climbed from “minor graphics card maker” competing with the likes of Matrox and now-defunct 3dfx (which Nvidia swallowed) to becoming the world’s dominant chipmaker. They took a series of big risks (hardware transform and lighting, programmable shaders) with their early GeForce cards for games, and then took a huge leap faith with CUDA long before anyone else in the market took GPU computing seriously. You don’t really often see long-term visionary leadership like that in the tech industry, especially not from a leader who’s mostly invisible in the public eye (Jensen Huang). He doesn’t have the ego of Musk or Jobs, but over the decades, he’s made a series of big bets that paid off handsomely.
Sure, crypto, COVID, and finally AI increased the demand for their “shovels” several fold, but without their previous successes, they wouldn’t have been able to be where they are right now: right place at the right time. It took a lot of smart competition, R&D build-up, and plain good luck to get to where they are today.
There’s a very good documentary about their rise and near-future plans: Cleo Abrams: NVIDIA CEO Jensen Huang’s Vision for the Future — even to a non-tech audience. The interviewer is knowledgeable and clear-spoken and manages to ask interesting questions without jargon.
As someone who works in tech, I really respect what that Nvidia has done — much more so than anything Microsoft or Google did in the same time period. As a gamer, I’m sad that so much of their attention is taken up by AI these days, and that PC gaming has become an afterthought (and way more expensive than it used to be).
But yeah, all that said, I’m sure a lot of their current valuation is just people trying to jump on the bandwagon before the bubble bursts. If the AI hype can continue for another few years, that’s still a lot of investors who can cash out before it all implodes…