The GPU is the gas station across town that’s five cents cheaper.

No, it’s the bulk order from China that’s 100x cheaper but has a minimum order quantity of 100000 units and takes 6-8 weeks to get here.

this is simply not true. the post just uses outdated hardware, as pointed out above.

a GH200 will run miles around any CPU.

The gist of the post is that optimizations and interpretations thereof must always be made with respect to the underlying hardware.

In this analogy, using better hardware affects the discount per gallon, but there are still situations where the closer gas station is the better choice.

Good analogy.

L2 cache is 1000x closer than the PCIe bus. Pretty much anything that has to respond in ~realtime to outside events will run better on the CPU. You can use the GPU to visualize the state of the system with some small delay (e.g., video games), but it is not so great at modifying state in an efficient manner - especially when serialization of events & causality are important.

> ~realtime to outside events

well, to play the devil advocate, for the outside event to affect the CPU, a signal will have to go through the PCI bus or equivalent.