> I don't understand the code that was written well enough to work on it myself any more.
That's worse than you think. Maybe not for you though.
They are trying hard to push vibe coding in my work and they had one demo that stood out.
They showed how VC had created an app to analyse multiple logs. It worked, but when they showed the code it was using csv module in python and had created all its own functions. The app was 100's of lines long. The same thing could have been achieved in a few lines of code using pandas.
The person creating the app had no experience of python, nor how do to the work. So they could never tell whats wrong.
.. And that is what is going to happen as junior people come into the workforce, as the next line being pushed is you don't need an expert to VC.
Now imagine that tool in the hands of someone who knows how to build it the right way, and can instruct the LLM to generate the code in the right way. You get what you would have created yourself, but just much faster.
> You get what you would have created yourself, but just much faster.
Faster but not same as an expert would have created.
As an expert you have assumed knowledge on your area of expertise. For example I would assume that the system would know to use pandas and not csv module, because the latter would be a stupid thing to do outside of learning python.
Having a research department is very different from whether they are actually accomplishing anything and whether or not it is a meaningful revenue stream. On all fronts this is just not true. The quantum stuff so far in the industry is vapor ware and its unclear wether IBM is really making forward progress vs yearly pronouncements of progress to keep the funding stream open
He is pointing out that the current costs to create the data centres means you will never be able to make a profit to cover those costs. $800 Billion just to cover the interest.
OpenAI is already haemorrhaging money and the space data centres has already been debunked. There is even a recent paper that points out that LLMs will never become AGI.
The article also finishes out with some other experts giving the same results.
It's already quite useful. While not all AI service providers are profitable, I've worked on projects that saved a lot of money for the company - a lot more than it cost us running the servers.
took me a while but i read it. thought it was actually a pretty good and well researched paper that does a good job rationalizing its thesis. thanks for sharing
Is this AI paper written by a reputable subject matter expert? It seems to be written by a physicist and also be the only academic work by this author in English
How many articles on this topic do we imagine there are? Thousands? Hundreds of thousands? It is hopeless to read every one by any author, no matter how unrelated to the domain, and judge them individually on their merits. Being a subject domain expert is not a perfect measure of paper quality but it's the only feasible way to make a first pass at filtering.
Even if I did read it, I have no hope of understanding if it has made a fundamental mistake because I don't have the subject matter expertise either.
(I imagine it has made a fundamental mistake anyway: for LLMs to be useful progress toward AGI they don't have to be a feasible way to create AGI by themselves. Innovation very often involves stepping through technologies that end up only being a component of the final solution, or inspiration for the final solution. This was always going to be an issue with trying to prove a negative.)
The core piece as quoted from the abstract: "AGI predictions fail not from
insufficient compute, but from fundamental misunderstanding of what intelligence demands structurally."
Then goes in detail as to what that is and why LLMs don't fit that. There are plenty other similar papers out there.
Sry to say but the fact that you argue with LLMs never become AGI, you are not up-to-date.
People don't assume LLM will be AGI, people assume that World Models will lead us to AGI.
I personally never asumed LLM will become AGI, i always assumed that LLM broke the dam for investment and research into massivce scale compute ML learning and LLMs are very very good in showing were the future goes because they are already so crazy good that people can now imagine a future were AGI exists.
And that was very clear already when / as soon as GPT-3 came out.
The next big thing will probably be either a LOT more RL or self propelling ai architecture discovery. Both need massive compute to work well but then will potentially provide even faster progress as soon as humans are out of the loop.
AlphaGo Zero doesn't need much human intervention at all.
Regarding world models: All the big ones. LeCun, Demis Hassabis, Fei-Fei Li too. And they are all working on it.
LLMs will definitly play some type of role in AGI. After all you can ask an LLM already a lot of basic things like 'what are common tasks to make a tea'. A type of guide, long term fact memory or whatever this can be called.
> AlphaGo Zero doesn't need much human intervention at all
You should research it and not just read news articles. RL did not work and required human intervention numerous times before it got close to what it is now.
Are OpenAI or Anthropic et al seriously building towards “world models”? I haven’t seen any real evidence of that. It seems more like they are all in on milking LLMs for all they are worth.
> Even if all the people are gone but the culture hasn’t changed
Can you expand on this? What was the culture then versus now?
For example back then it was the culture to have suit inspectors ensure you had the right clothes on and even measure your socks. (PBS Triumph of the Nerds)
I have. There is a whole power-mod clique over there that just turned me off using the site.
I posted about how to get Launchpad back as the spotlight feature is frustrating to use.
One of the mods posted suicide watch to my response. Asked if they were bot or what were they on about. My post got deleted and their post got edited.
Then I had another mod telling me to "suck it up" and it wasn't just a response directed at myself. When I said it wasn't constructive they responded with insults and deleted my post.
It was all surreal and totally messing with Apples brand.
> If the iPhone hadn't dropped the headphone jack nobody would've bought Airpods, thus the jack was killed.
My understanding was Apple wanted to get rid of cables entirely if they could. It's one of the major pieces of waste and the reason EU forced USB-C on everyone.
Personally I find the airpods great. I use the noise cancellation to sleep.
reply