I mean, I don’t know how long the NVIDIA moats can hold. With this much money at stake, others will challenge their dominance especially in a market as diverse and fragmented as advanced semiconductors.
I think that NVIDIA’s moat is the US government. Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world?
I am a long time fan of Dave Sacks and the All In podcast ‘besties’ but now that he is ‘AI czar’ for our government it is interesting what he does not talk about. For example on a recent podcast he was pumping up AI as a long term solution to US economic woes, but a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails. Another thing that he swept under the rug was the recent Stanford study that 80% of US startups are saving money using less expensive Chinese (and Mistral, and Google Gemma??) models. When the Stanford study was released, I watched All In material for a few weeks, expecting David Sack’s take on the study. Not a word from him.
Apologies for this off-topic rant but I am really concerned how my country is spending resources on AI infrastructure. I think this is a massive bubble, but I am not sure how catastrophic the bubble will be.
I think the answer to your question is ‘not for very long.’ I frequently have breakfast with a friend who is a retired math professor and he is an avid investor in the stock market. We talk a lot about how long the US stock market will keep increasing in value. We don’t know the answer about the stock market, but it is fun to talk about. We both want to start easing out of the stock market.
They are heavy into AI investing but will tell people AI startups are just toy apps (Chamath). That podcast is full of crooks. I’d be willing to give them a pass as bunch of old white guy techies that just love to talk about tech, but they are literally at the dinner table with Trump and Musk.
This country used to have congressional hearings on all kinds of matters from baseball to the Mafia. Tech collusion and insider knowledge is not getting investigated. The All-in podcast requires serious investigation, with question #1 being “how the fuck did you guys manage to influence the White House?”.
Other notes:
- Many of them are technically illiterate
- They will speak in business talk , you won’t find a hint of intimate technical knowledge
- The more you watch it, the more you realize that money absolutely buys a seat at the table:
It's not startup success rate, it's application of the technology at companies. Meaning that 95% of the time that AI is applied to a work problem, it fails to generate material value over existing methods.
Sacks has always been absolutely disingenuous and interested in pedaling his own interests over the interests of the common good. As a total Trump shill he talks out of both sides of his mouth at the same time & accuses the left of things that he has no problem with when he or his own party does it.
Anyone who's listened to him (even those who align with him politically) for an extended period of time can't help but to notice so obviously so self interested to the point of total hypocrisy—the examples of which are too many to begin to even wanting to enumerate. Like—take the Trump/Epstein stuff, or the Elon/Trump fallout—topics he would absolutely lose his sh*t over if these were characters on the left. I find it hard to believe anyone actually ever took him seriously. Branding myself as a fan of his would just be a completely self-humiliating insult to my intelligence and my conscience IMO.
> a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails.
I mean. I think some of us knew this. There's a lot of issues with AI, some psychological, some are risk adverse individuals who would love to save hours, weeks, months, maybe years of time with AI, but if AI screws up, its bad, really bad, legal hell bad, unless you have a model with a 100% success rate for the task, it wont be used in certain fields.
I think in the more creative fields its very useful, since hallucinations are okay, its when you try to get realistic / look reasonably realistic (in the case of cartoons) that it gets iffy. Even so though, who wants to pay the true cost of AI? There's a big uphill cost involved.
It reminds me a lot of crypto mining, mostly because you need an insane amount to invest into before you become profitable.
I was also wondering if Google would try to make profit from selling TPUs, but they probably won’t because:
At least for me, Google has some real cachet and deserves kudos for not losing money selling Gemini services, at least I think it is plausible that they are already profitable, or soon will be. In the US, I get the impression that everyone else is burning money to get market share, but if I am wrong I would enjoy seeing evidence to the contrary. I suspect that Microsoft might be doing OK because of selling access to their infrastructure (just like Google).
There's no point selling TPUs when you can bundle TPU access as part of much more profitable training services. The margins are much higher providing a service as part of GCP versus selling.
I agree. Amazon and I think Microsoft are also working on their own NVIDIA replacement chips - it will be interesting to see if any companies start selling chips, or stick with services.
From what I'm hearing in my network, the name of the game is custom chips hyperoptimized for your own workloads.
A major reason Deepseek was so successful margins wise was because the team heavily understood Nvidia, CUDA, and Linux internals.
If you have an understanding of the intricacies of your custom ASIC's architecture, it's easier for you to solve perf issues, parallelize, and debug problems.
And then you can make up the cost by selling inference as a service.
> Amazon and I think Microsoft are also working on their own NVIDIA replacement chips
Not just them. I know of at least 4-5 other similar initiatives (some public like OpenAI's, another which is being contracted by a large nation, and a couple others which haven't been announced yet so I can't divulge).
Contract ASIC and GPU design is booming, and Broadcom, Marvell, HPE, Nvidia, and others are cashing in on it.
I wouldn't be surprised if a fair portion of Amazon's Bedrock traffic is being served by Inferentia silicon. Their margins on Anthropic models are razor thin and there's a lot of traffic, so there's definitely an incentive. Additionally, every model that's served by Inferentia frees up Nvidia capacity for either models that can't be so served or for selling to customers.
That's the same as saying that Google is winning the AI race because they don't like losing. They won't win anything if we are in a bubble that burst tho
A hypothetical AI bubble bursting doesn't mean that every single AI vendor fails completely. Like the Dot-Com Bubble, the market value drops precipitously and many companies fold, but because the market value does not fall to zero, the survivors (i.e. Amazon) still win.
Websites were still mostly selling goods and services in 2001. Not giving away hot takes and hallicinated summaries in exchange for eyeballs. In other words, after stuff like pets.com collapsed, people still found it useful to have pet food delivered, and the business model evolved. LLMs, on the other hand, don't seem to have a lot of public appeal. Most of the use cases are being shoved down the public's throat. Their appeal is to corporations as cost saving replacements for workers. But an AI bubble bursting would look like corporations rolling back their exuberance for the AI craze. What's already only speculatively profitable and requires enormous capex would probably become too toxic for anyone to try again for a generation.
As long as only TMSC is only top performance chip producer and it is possible to reserve all it manufacturing capacity for one two clients the NVIDIA will hold without problem...
My opinion, the problems for NVIDIA will start when China ramp up internal chip manufacturing performance enough to be in same order of magnitude as TMSC.
TSMC and NVIDIA's relationship has gone back for more than 20 years. In the NVIDIA biography they talk about how TSMC really helped NVIDIA out early on when other suppliers just couldn't meet the quality and rate demands that NVIDIA aspired to. That has led to a strong relationship where both sides have really helped each other out.
> But other people are literally getting their things fabbed by them.
Sure, but in my example Apple got access exclusively for a few months to a newer node, which would make a world of difference if you compete in the same space.
> but I wonder how important performance really is here.
Perf is important, but ime American MLEs are less likely to investigate GPU and OS internals to get maximum perf, and just throw money at the problem.
> solder on a large amount of high bandwidth memory and produce these cards relatively cheaply
HBM is somewhat limited in China as well. CXMT is around 3-4 years behind other HBM vendors.
That said, you don't need the latest and most performant GPUs if you can tune older GPUs and parallelize training at a large scale.
-----------
IMO, Model training is an embarrassingly parallel problem, and a large enough cluster leveraging 1-2 generation older architectures that is heavily tuned should be able to provide similar performance to train models.
This is why I bemoan America's failures at OS internals and systems education. You have entire generations of "ML Engineers" and researchers in the US who don't know their way around CUDA or Infiniband optimization or the ins-and-outs of the Linux kernel.
They're just boffins who like math and using wrappers.
That said, I'd be cautious to trust a press release or secondhand report from CCTV, especially after the Kirin 9000 saga and SMIC.
But arguably, it doesn't matter - even if Alibaba's system isn't comparably performant to an H20, if it can be manufactured at scale without eating Nvidia's margins, it's good enough.
> Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.
They've been trying really hard to pivot and find new growth areas. They've taken their "inflated" stock price as capital to invest in many other companies. If at least some of these bets pay off it's not so bad.
Chinese semiconductor dominance is not imminent and US containment has been somewhat effective. I don’t think that will hold on a generational timeline, but it will be hard to overcome.
You don’t think the export controls on Nvidia chips accelerated Chinese investment in ML processors and therefore their independence -> dominance in the space?
The export controls made it difficult for Chinese companies to acquire large numbers of GPUs, which prevented them from expanding business models that rely on buying more GPUs to serve more customers, which means that Chinese companies have much, much lower budgets for GPU procurement than their American counterparts. https://chinai.substack.com/p/chinai-323-the-ai-deflation-of...
So a new homegrown chip would have to capture a very large share of this relatively small market to make significant volume. That makes it rather risky for profit-driven investors.
Politically directed investment probably increased, but in the end the private sector also needs to be on board.
Most of the rest of the world can buy Nvidia if they want to. And I don't think the EV comparison works, since China already had a large domestic market and established ICE car companies who could afford to electrify some of their lineup and slowly gain market share this way.
That is not at all the case. Biden export controls for nvida chips are still in place. China also has a large domestic market for chips. The rest of the world would gladly buy competitive Chinese chips.
Building SOTA semiconductors is more art than science. All the best artists are in Taiwan.
You don't just buy (or copy) and ASML litho and turn it on. Just like you don't buy a horsehair brush and start painting Picassos. It's even more difficult than that because there are something like ~1000 sub processes and each one needs a world renowned artist in that specific art to get it done.
Its the reason why even Samsung can't match TSMC despite having the same tools and capital.
> So will the military hopes of ever protecting Taiwan.
I don't think there are too many military analysts who would claim that the USA could "protect" Taiwan if China was really determined. The USA still retains the ability to significantly increase the cost, both militarily and economically, of an invasion, and relies on this - successfully, to date - as a deterrent.
I think most people recognize however, even in Taiwan, that in terms of pure practical facts on the ground, not even the world-beating US military can overcome geography. Taiwan is 100 miles off China's cost, Guam is 2800 miles away. It is difficult for me to imagine anything, barring some incredible technological advantage that the USA shows no sign of possessing, that could outweigh such a tremendous home-turf advantage.
It is very hard to come up with a realistic, or even semi realistic, scenario in which China does not end up with Taiwan if it really wanted it.
That's a very pessimistic take, or optimistic I guess, depending on perspective.
Looking at the Chinese semiconductor development trajectory, and considering that TSMC won't be sitting on their hands, "within a decade" seems really unlikely.
I am aware, I live in Taiwan. While TSMC engineers can be poached "move between countries freely" is not true because moving from China to Taiwan is not so easy.
The key word you mention is "replicating." They'll be chasing for a while still, and it's not clear that they'll be able to leap ahead. Copying is much easier than real innovation.
I was under the impression, for years, that the US had the appropriate government, scientists and engineering in place to protect the castle. However given what I've seen in the last few years - I agree that it seems inevitable China will surpass the US in the next decade and will hold both cards and a grudge.
It's amazing how China has doubled down into STEM and green energy while the US has done exactly the opposite. The CHIPS Act propped up a company further driven into the ground by Pat Gelsinger. The last few administrations have had no focus on driving innovation and technology - only propping up the Tech Bro market making money off of attention and ads. Maybe, just maybe, the US should stop electing geriatric and short term gains ignorance?
The US needs to dig its head out of its ass if it wants to continue to be recognized as the global power it once was.
Taiwan could be protected if they were given practical control of nuclear weapons or similar, i.e. nuclear weapons sharing.
Securing Taiwanese independence is going to be necessary for the EU to ensure that there isn't a US microchip monopoly, and the only way the EU can do this is by the aforementioned means.
Zelensky at the MSC 2022: “If … results do not guarantee security for our country, Ukraine will have every right to believe that the Budapest Memorandum is not working and all the package decisions of 1994 are in doubt”[0]. 4 days later Russia invaded Ukraine. At the time it was said that Ukraine pursuing nuclear weapons was the last straw, but who know really.
If Taiwan is going to place nuclear weapons on its territory the conflict would probably escalate quickly.
Could well be a trigger. This is a reason to do it quickly by having already existing, tested nukes handed over for use by the threatened state.
There is a problem with this kind of reasoning though. It's like, somebody shows up 'try to grab my gun and I'll shoot' but the thing is, he might shoot anyway, and if you grab the gun, maybe you'll get hold of it.
When someone threatens something, that's not reason for stepping down, passivity or anything like that, it's reason to immediately risk everything on an attack to take away the thing he has that allows him to threaten you. By threatening something he only demonstrates that he must be attacked immediately.
I like an example I gave earlier with hostages. It's Monday, someone has taken a hostage and threatens to kill unless left alone. The next day he's taken another, he has the same threat. On friday, he has five. Now, you realise you should have attacked on Monday when your attack only risked one death.
So today, the question might be 'why didn't we start building nukes last year?'
It’s pretty hard to quantify. Reading the tone of NYT headlines over the last 20 years, I observe a steady increase in side-taking on both sides of the newsroom. Those headlines and stories are empirically more engaging and the market values engagement above all else.
I’m starting to wonder if post war America was this unique time in history, because I’m not saying your wrong but it also seems like historically the idea of an unbiased news media is not the norm.
In general I have trouble believing it’s the medias fault though, not that they’re without sin, I just think the forces that make something like this are bigger than the media.
> I’m starting to wonder if post war America was this unique time in history
It certainly was re: journalism.
When we only had the big three TV networks vying for our eyeballs, they each used news as a way to attract the broadest set of Americans. The nightly news was a loss leader, in the hopes that we'd watch the rest of the evening's programming/ads on that network.
The "unbiased" business didn't really exist until someone could be in all of our homes building rapport with our families every night. And cable, then more so the web, completely disrupted that.
There used to be many, many more newspapers, with room for distinction as the paper of record for any given demographic. Radio, then more so TV, put an end to most of those.
It's interesting history though - I grew up and was using computers in this period (from 300 baud acoustic couplers and ARPANet, to 14.4K modems and BBSs, then eventually the web (Sun workstation and broadband at work, dial-up at home), but was not aware of all the history myself. The invention of the web was a seminal moment, regardless of what you think of Andreessen, and like he said it could have gone differently. The private networks (AOL, Compuserve, Prodigy) could have prevailed, but luckily the internet and open standards won the day.
Yeah, I mean I’m being negative but I enjoyed the video. I just found Ben’s whole line of questioning to be pandering and not a dialog (“why not MIT?”)
I had a similar experience with BBSs and I remember the romance of using a modem. I actually grew up in Champaign and recall dialing into “prairienet.” I also remember using AOL, but mostly to find and download games.
It’s interesting to see the give and take between google sheets and excel. Google sheets came on the scene shooting for total backwards compatibility and then proceeded to develop some really interesting innovations. Now we see new features emerging on both sides that are quickly replicated by the other player. Notably off the top of my mind:
* spill formulas - google first, now supported in MSFT
* # notation - MSFT enhancement to spill formulas not yet adopted in google
* regular expressions - google first, now in Excel
* check boxes - google first, now supported in excel
There must be others. I would expect competitive dynamics where each side tries to build extensions that can’t be replicated on the other side
The secrecy is ick, but this is the future and there’s no stopping it.
There’s ample evidence that consumers won’t pay for privacy and as most consumers opt in to data sharing programs, the non-data-sharing cohort will get seriously adverse further raising the price of privacy. The equilibrium state is that only bad actors and a handful of privacy zealots will inhabit that pool and mainstream carriers won’t even bid it.
Not really. Accidents are a function of driver and environment. You can't control the other drivers. If you think of the primary purpose of insurance as to make sure that the party not at fault is made whole, then perfectly priced insurance becomes like posting a bond in order to drive. Which isn't unreasonable.
(Plus uninsured motorist coverage for the other parts you can't control, which really is an insurance function.)
There used to be opt-in insurance programs with many carriers. They used to send you a device, but I guess that was mooted by secret mass surveillance?
That’s not to say I’m brave enough to short NVDA.