It's not that Nvidia 'keeps winning', Nvidia failed mobile computing, failed embedded computing, had no market share in the server market or data centers, was left in the niche market of gaming as a companion to PC and was not that impressive all along.
then its parallel GPU got "lucky", first the bitcoin mining, then the AI. it probably did not expect and plan for this, to some extent, it got super lucky.
credit must be given to its CUDA ecosystem and the ability to better itself when chances knocked its door, it so far left all competitors in the dust, its showtime arrived, finally.
Nvidia got lucky because every quarterly all-hands meeting Jensen repeated that he kept on investing in CUDA and adding more silicon to the GPUs than strictly needed because, one day, an application would come along that would make it all worth it.
TFA says that Nvidia started aggressively seeding CUDA and GPUs for research in the early 2010s. It was much earlier than that: it started pretty much immediately after CUDA was introduced late 2006. And every new generation there were hardware features added to make GPU programming and porting of applications less painful. The first Nvision conference, precursor of GTC, was in 2008. That’s how you make your own luck.
I’ll never forget when, sometime around 2012?, he answered the question: “aren’t you afraid of Intel?”
His answer: “Not at all. Intel should be afraid of us. We will be bigger than them.” There was not a trace of doubt.
> "His answer: 'Not at all. Intel should be afraid of us. We will be bigger than them.' There was not a trace of doubt."
Given all the times that HN readers have derided grandiose executive pronouncements preceding flops, more people should recognize the above for what it is: not profundity but just puffery that happened to pan out. Not that skill and effort weren't involved in making it pan out but that any of a zillion things could have gone wrong to make that statement false and part of any manager's job is to project confidence and instill motivation despite knowing that.
I think he had a strategy - utilizing the massively parallel computation of GPUs for more general purpose compute as Moore's law tailed off - and he noticed that Intel couldn't see the lights of this in the rear view mirror.
Everybody's known that Moore's law was on its way out, for speed increases at least, since the mid 2000s - the seminal article was by Herb Sutter [1]. So hardware needed to get more parallel. But multicore is a distinctly different paradigm to CUDA, which is closer to SIMD but on a completely different order of magnitude. So Intel was never going to get to where the puck was skating.
That’s the point, though. This is no different than any other statement made by a CEO with good engineers behind them.
This time it worked out. Can’t give it a survivorship bias. I don’t personally mind CEOs being encouraging, but at least understand that they don’t really ever know.
IMO one big factor is that Nvidia is still fully engineering driven - it's engineers all the way to the top making the calls. Intel was like that as well, and then lost it (until Gelsinger). IMO you need domain experts in charge of companies, or they can't thrive in the long run, not unless there is an actual, almost unsurpassable moat.
It's called leadership. George Washington wasn't a brilliant general but he was able to convince people they were going to win against an empire. Whether he actually believed it himself we'll never know.
> Nvidia started aggressively seeding CUDA and GPUs for research in the early 2010s
I was at a niche graphics app startup circa 2000-2005 and even then NVidia invested enough to be helpful with info and new hardware, certainly better than other GPU companies. Post 2010 I was at a F500, industry leading tech company and an NVidia Biz Dev person came to meet with us every quarter usually bearing info and sometimes access to new hardware.
It's also worth noting that NVidia has consistently invested more than their peers in their graphics drivers. While the results aren't always perfect, NVidia usually has the best drivers in their class.
Oh interesting. I remember that Folding@Home way back then (ca 2009) was already testing protein folding on GPUs and it took advantage of CUDA. I never really thought much of it other than how cool it was that my mid-tier Nvidia graphics card could be used for something else other than games, but this explains how this ended up happening.
(Bit of a tangent, but that project was very influential in getting me interested in computer science because, wow, how cool is it that we can use GPUs to do insane parallel computing. So I guess, very very indirectly, Nvidia had a part in me being a software engineer today.)
By 2007/2008 there was a trend in HPC research called GPGPU. This involved havky techniques to get the shaders to do the computations you wanted.
CUDA started appearing in 2008 with a framework (compiler, debugger) to do GPGPU in a proper way. It got the monopoly. They’ve been benefiting from first movers advantage ever since.
GPGPU was a thing well before that! In 2004, it was ready covered in a few chapters of GPU Gems 1, increasing to 18 chapters of the 2005 GPU Gems 2, which including an FFT implementation.
Excellent point! Jensen is very focused and the company has worked incredibly hard on whatever they've put out there. The Shield is a testament of this focus. They find a budding niche and double down on building it from nothing. Most "self-driving" cars have Nvidia gear for a reason.
It's the best Android TV/Gaming device around, even by todays standards. It's stable and does what it was designed to do. With zero marketing from Google or Nvidia, the mass market obviously didn't care for this type of product and category but the device itself is great and works flawlessly. The Nvidia Game app also bundled and pushed the Nvidia game streaming concept around GeForce Now. Overall, Nvidia gave put their best foot forward with this device, providing standout support for both HW and SW.
The chip line they made for it powers the most popular console on the market and basically locked its manufacturer into Nvidia chips until they're willing to drop compatibility, so financially it probably worked out for them, even if the Shield line itself wasn't extremely financially successful.
I don’t know about it’s market success, but it’s a great product. We use it for as the frontend for all of the streaming platforms and PLEX as well as running some stuff directly from a NAS and IPTV.
I use them everywhere and have a big pile of chromecasts, satellite boxes, remote controls and Apple tvs now ready for eBay!
Some years before CUDA there was a lot of hype when the first GPGPU papers published in 2003 which showed significantly increasing performance using parallel computation from consumer graphics cards. At the time, it looked like competing on general purpose computation was a solid strategy: multi-core CPU from intel was still years away, showing up in 2005; starting from 2000 the rate of increase of clock speeds started slumping. We saw Intel started releasing more variants of processors, but the clock speeds weren't advancing exponentially anymore. The new battle for core supremacy was on the horizon.
It's a bit unfair to ascribe it to sheer luck. They were focused on the compute related possibilities all the way back with the GeForce 3 in 2001. Presentations from its launch were already talking about the potential of a "parallel compute monster" [1].
They saw the potential of GPU compute very early on, invested in it long term and as a result eventually ended up dominating the market. The others didn't seriously commit and so they fell behind. AMD still can't seem to commit, while Intel seems to be working hard on catching up but isn't quite there yet.
The actual quote from your link is: "Expect a massively programmable, massively
parallel and pipelined graphics monster", not a "compute monster".
While I agree that Nvidia positioned themselves well, they were not looking as far forward as you suggest. As I recall it, they seemed surprised by BrookGPU, though they moved quickly to embrace the model.
> then its parallel GPU got "lucky", first the bitcoin mining, then the AI. it probably did not expect and plan for this, to some extent, it got super lucky.
I feel like saying they got "lucky" after trying and failing in multiple other endeavors requires a special definition of luck. If someone rolls a 6 sided die six times and they roll a six once did they get lucky?
Being good also has the facade of luck, every venture is a gamble none of which are guaranteed, but putting yourself in the most optimal positions (including diversifying) will result in some successes and some failures. You don’t end up with a $1t valuation based purely on luck.
For corporations, just staying alive that long without a huge hit is very lucky. Look how many other chip companies failed or were acquired cheaply during that period.
Given how AMD/ATI has fared with their shitty software ecosystem after close to 20 years (the first paper on using GPUs for CNN was in '05/'06 -ish where they were using shaders), calling Nvidia 'lucky' is quite unfair.
Those of us who were unlucky enough to buy a AMD GPU based on 'flop-count' were hurt quite badly. Nvidia's software compute-infra is simply unmatched.
Having a competitor that seemingly refuses to compete is pretty damn lucky. There's not a single thing nVidia could have legally done to make that happen.
>then its parallel GPU got "lucky", first the bitcoin mining...credit must be given to its CUDA ecosystem
this seems like a bit of contradiction to me. they either got luck as you claim, or CUDA was good to keep things going. from my experience, CUDA was the only way to go for GPU accelerated processing. specifically, my experience was with Resolve color correction and using CUDA was the only way to go. memory is fuzzy, but the timeline on crypto was sametime-ish. because of the audience, this forum is naturally going to trend crypto vs niche video post workflows, but CUDA was definitely kicking Radeon/AMD ass in other aspects besides useless crypto.
Same. I wanted to buy a really chunky graphics card for BMD Fusion, but the price got pushed up by the cryptobro wankers mining their Dunning-Krugerrand, and now the price is getting pushed up by all the chatgpt wankers trying to run hardware-accelerated Eliza bots.
It's not pure luck tho. The GPGPU research started in early 2000s IIRC. Nvidia had invested into it earlier than anyone and more than anyone. That's how they got CUDA. Nvidia was ready for the next generation computing. It's just that no one, including Nvidia, knew when it's gonna hit the market.
The problem isn't C. The problem is that the buggy OpenCL garbage doesn't work at all. I once used hashcat with OpenCL in a security course on an AMD GPU and it made my system completely unstable. I couldn't care less what the kernel is written in. I'm never going to use hashcat on AMD GPUs ever again.
The other problem is that companies invest in some alternative to OpenCL which fragments the non-CUDA ecosystem.
I thought that I should mention a book because it got talked a lot under Nvidia's "lucky success". The book is titled: Why Greatness Cannot Be Planned: The Myth of the Objective (https://link.springer.com/book/10.1007/978-3-319-15524-1#toc).
I don't really think Nvidia's luck is really just a luck. After all, they has the vision for CUDA, bet money on it, and succeeded. They CAN stop the effort at any point during the adventure, but they choose to continue. That's not luck, that's a vision.
BTW, the same book got picked up by some Chinese talk heads as an perfect demonstration showing the good side of capitalism, which I totally agreed. After all, if Nvidia was a Chinese company, they'll probably instead put their efforts into developing some massive-spying citizen-incriminating policewrongs-neverseening camera with funding provided by the government, thus almost guaranteed "success". I'm really glad that Nvidia is operating in a country where "making hardware so everyday people can use their computers to play games" is not something to be mocked at by it's (abusive)parents-acting government. Now, look who's paying smugglers for those A100 chips at double the price while crying? A well deserved fate on my book.
I started reading that book (seems great!) and stumbled upon this: "And it’s hardly clear that computer scientists will succeed in creating a
convincingly-human artificial intelligence any time soon." Made me smile. Took but 8 years, the book was published in 2015. :)
(BTW. I'm super concerned about all the incredible amount of power nvidia has now and in the future. They get to decide the fate of the human race, I feel. And their incentive is just to make as much money as possible, while all negative externalities including the likes of extinction is left to the society to deal with. Sigh.)
There's a lot of hard work that goes into luck, but one aspect of the Nvidia story is of AMD's mismanagement. There's an alternate reality where OpenCL became the default instead of CUDA but that's not our reality.
> Nvidia failed mobile computing, failed embedded computing,
Really? What do you think high-end drones and self-driving cars are using. They invested in generic robotics software and hardware probably more than all others put together. When you go beyond Raspberry Pi it's only NVidia. Their day is coming. Another similar ecosystem will be hard to create.
It allows for machine vision and heavy maths to be executed very fast. There is basically nothing similar in an embedded format in the market. The remainder would be SoCs for Android phones (Samsung Exynos?).
Thsese SoCs have lots of hacky drivers and mostly support Android, which is not a very good fit for real time and high customization required for drones and self driving cars (AOSP build system, a google class piece of crap).
They barely exist in the network switch market. The new NVLink switches may change that for tier one inter-chassis transport to replace Infiniband RDMA, though.
Reality is Mellanox has never been a substantial networking player. Yes, in HPC, but that's a tiny market even today.
It doesn't help that Mellanox acquired Cumulus just prior to Nvidia acquiring Mellanox, and Cumulus was basically DOA. Now they are split SONiC/Cumulus with a lot of internal infighting trying to keep Cumulus relevant despite industry trends.
Don't forget the drivers. As someone who just switched from Nvidia to AMD, it is downright painful how bad AMD's implementations of Vulkan and OpenGL are. I might be getting more bang for my buck but damn do I miss not having unfixable glitches.
AMD is good at being the underdog, hopefully it will focus more on its software, the ROCm thing, which really needs some love, a lot of love indeed. The software ecosystem for AMD's RDNA(gpu) and CDNA(MI2xx MI300) is at its best a mess.
AMD shall own OpenCL and boost it heavily and make it a central piece for its ROCm framework as the preferred backend, in my opinion.
ROCm does not compile PTX IR to AMD assembly, if it did you'd be able to run nearly any compiled CUDA program on and AMD GPU without the source. ROCm is a source level compiler for CUDA (technically HIP is the actual compiler, but whatever), it allows you to a substantial fraction of the CUDA APIs. This notably also means that any Nvidia open source library that uses inline PTX assembly won't work, but fortunately AMD does have alternatives to many of the Nvidia libraries.
both OneAPI and ROCm are, or should, or could provide higher level APIs to isolate you from OpenCL's C APIs, or just leverage SPIR-V. Other than CUDA, I failed to see any other open alternatives to get heterogeneous computing working yet, OpenCL is the only one on the table as far as I can tell for now. Yes there are Vulkan compute shader etc but they're still pretty far behind, and they could be made OpenCL compatible too.
While CUDA is great, OpenCL can be made by OneAPI and ROCm more open source friendly, I hope.
By reputation, Jensen doesn't really have much in the way of SW understanding, and in that sense he is like a lot of former chip guys. They got _insanely_ lucky that CUDA took off and there's an amazing irony in SW being their primary lock on their current market.
To win, they just needed to show up. Which is more than can be said about the competition.
AMD alternatives to CUDA are/were fumbling in the dark for many years, and more open alternatives like OpenCL are too limited (by design?).
To me the situation looks quite clear: a GPU has vastly more compute than a CPU. As time goes, we will need and use more and more compute. You just need a way (general purpose language or API) to use that GPU.
For some reason, other companies in this space did not see this.
They keep failing to see this, while CUDA is a polyglot programming model, a couple of years ago at an OpenCL conference (IWOCL), someone asked the panel when Fortran support was going to happen.
Everyone on the panel reacted surprised that it would be something that anyone would want to do, and most of the answers were the kind of talk to us later.
Meanwhile PGI was shipping Fortran for CUDA, this was before them being acquired by NVidia.
I agree with this. All they had to do was the bare minimum and actually keep it alive for a few years.
This pattern is pretty common in industry. Almost all the huge companies that are winners in technology are those that got on the market and kept the thing alive - that's not sufficient, but it is necessary.
Nvidia gpus were massively inferior for Bitcoin mining, in fact, because ATI/AMD had some integer operations that allowed SHA256 to be several times more efficient.
Ironically, that's what ultimately made Nvidia the winner for gpu mining: ATI gpus had been massively deployed for Bitcoin mining prior to the dominance of mining asics. When people created new altcoins they specifically designed their work functions so that they the inventors could have an advantage vs the general public, so they designed them for nvidia gpus rather than what was already deployed. This let them buy up gpus before shortages came into effect and delayed competition from the installed base.
Sure, you can trivially port whatever to whatever, but outside of startup effect mining is naturally perfectly competitive. Being 20% less efficient vs costs means bankruptcy.
> Nvidia's moat is AI.
Nvidia had a gpu computing moat before the current AI fad, due to maturity of the CUDA ecosystem. At least AI codes are generally pretty easy to port to other architectures, similar to mining in that sense-- but the AI designers don't have a profit motive to make sure they choose algorithms that are more efficient on their hardware than yours, and your AI hardware doesn't become useless if does happen to be a few percent less efficient.
The first big mining wave was 2014-2015. This was done on 5850s, and GCN 1 and 2 GPUs. This was Bitcoin, so it was computation-focused, and I think in this era it was definitively AMD dominated due to VLIW allowing very dense execution resources plus early GCN having a large amount of raw integer processing power.
The next was 2017-2018. By this era bitcoin itself had moved off to FPGAs and ASICs, so this was around Ethereum which primarily worked based on proof of memory bandwidth. AMD GPUs were falling behind Maxwell/Pascal in terms of their memory compression (although they did use it) so they were equipped with more memory bandwidth to compensate. So for a given AMD card (Polaris, Vega, etc) you got more memory bandwidth per $, but in terms of the actual compute efficiency, NVIDIA had already pushed ahead even in Ethereum. NVIDIA was usually superior per-watt with cards like 1060, 1070, 1070 Ti, and 1080 Ti, and AMD just let you burn more watts.
However when it came to altcoins, where it was not just raw bandwidth, NVIDIA's superior compute/GPGPU efficiency took over and there were some coins that NVIDIA was 2x or more more efficient on per-watt and also the winner in absolute performance.
(the thing to remember is that compute is the part that ASICs can do efficiently, and I always questioned whether those altcoins were really ASIC-resistant. But the ProgPOW-style algorithms doing better on NVIDIA cards never bothered/confused me, the reality is that most GPGPU programs "favored NVIDIA" during this era and Ethereum's proof-of-bandwidth model was an exception. Pascal was an efficiency beast and Polaris and Vega were only ok at best, outside their enormous, dangling memory buses.)
This state of affairs persisted throughout the 5700 series until AMD launched the 6000 series, where they shifted to a design with smaller memory buses and more cache, which put them in the inverse situation of 2014-2015 where they were getting more out of a weaker memory subsystem than NVIDIA. And I think they did this on purpose because they wanted to "opt out" of the mining boom/bust cycle, and NVIDIA made a similar approach with Ada that has been extremely unpopular (despite AMD leading the way on this a few years before on their cards too).
Isn't it wonderfully coincidental that out of any amount NVIDIA could have put into the LHR to slow down when it detected mining, that they put in the exact amount that dropped their cards to the same relative mining performance as AMD, and moved to the same cache-based approaches in the next generation as well? That has always been my take around LHR - it's not that they didn't like mining revenue, it's that they didn't want NVIDIA cards to be disproportionately pulled off shelves like happened to AMD in the 2014 and 2017 mining booms. People remember the "AMD was $1600, NVIDIA was $2400" situation already, they didn't want that to persist and turn into actual marketshare.
I won't comment on the actual facts, I'll just say that I couldn't bear reading more than the first few paragraphs of the article because both guys sounded like such rabid fanbois...
NVidia has always been the first choice for graphics work, and especially now with large complex VFX pipelines there really isn't an alternative - particularly since most of that work is done on Linux which has always had excellent support from NVidia.
then its parallel GPU got "lucky", first the bitcoin mining, then the AI. it probably did not expect and plan for this, to some extent, it got super lucky.
credit must be given to its CUDA ecosystem and the ability to better itself when chances knocked its door, it so far left all competitors in the dust, its showtime arrived, finally.