Sorry for the caution but this is becoming very common. Lots of red flags for AI-Slop: Zenodo, single author work, lots of jargon, author has not had prior work in the field.
This is also very common, appealing to authority rather than reading the proof. If there's an issue with the proof please show me where the issue is. I am glad to learn where I made a mistake. Just run "lake build".
Out of 100 of evals, ARC is a very distinct and unique eval, most frontier models are also visual now, don't see the harm in having this instead of another text eval.
Once I read "This has been enough to get us to AGI.", credibility took a nose dive.
In general it's a nice idea, but the blogpost is very fluffy, especially once it connects it to reasoning, there is serious technical work in this area (i.g. https://arxiv.org/abs/1402.1869) that has expanded this idea and made it more concrete.
I've read this book, I don't think this was the case. I think the name was made in good-faith. It's "transformer" because it involves transformations of molecules involved in life (reactions, enzymes, metabolism). The author has used this term before 2022.
The same argument could be made for the transformer paper: hijacking a nostalgia pop-culture name to name a deep learning bi-linear operator. Many papers are guilty of this, some just become very influencial.
Could be a good idea, but without any evidence (benchmark/comparisons) it's just a flashy name and graphic.
Sounds like another "state" that gets contexualized via a gating mechanism wrt previous vectors.
Awesome, I am a fan of their work, just wish they did not use the word biology (which is rooted in living) to describe LLMs, we have enough anthropomorphizing of AI tech.
The entire paper is riddled with anthropomorphic terms - it's part of AI culture unfortunately. When they start talking about "planning", "choosing", "reasoning" it biases the perception of their analysis. One could certainly talk about a night light equipped with a photoresistor as "planning to turn on the light when it is dark", "choosing to turn on the light because it is dark, and "reasoning that since it is dark, it turned on the light"- but is that accurate?
I agree. "Planning" means we come up with alternative sets of steps or tasks which we then order into sequences or acyclic directed graphs and then pick the plan we think is the best. We can also create "Plan B" and "Plan C" for the cases that the main plan fails to execute successfully.
But as far as we know does AI internally assemeble subtasks into graphs and then evaluate them and pick the best one?
Is there any evidence in the memory traces of the executing AI that there are tasks and sub-tasks and ordering and evaluating of them, then taking a decision to choose and EXECUTE the best plan?
Where is the evidence that AI-programs do "planning"?
They're doing natural science on a thing full of complex purposive undesigned machinery. There used to be Artificial Life conferences -- the proceedings were pretty interesting. Now the objects of study are getting past a "gosh that's cute" level but I doubt anyone here's misled by the title.
Given that LLMs are literally trained on huge amounts of human-originated text and taught to model it, informing our intuitions regarding their external behaviour through a frame influenced by anthropomorphism... actually makes sense.
I really don't see the controversy here. My prompts, including ones meant for actual hard productivity (programming, image OCR and analysis, Q&A and summarisation of news articles), behave very differently when I introduce elements that work on the assumption that the model is partly anthropomorphic. We can't pretend that the behaviour replication isn't there, where is demonstrably is there.
Criticism feels harsh. Of course models don't know what they don't know. Reporters can have the same biases. They could have worded it better "lowers the probability of hallucinating", but it is correct it helps to guard against it. It's just that it's not a binary thing.
Alas, current LLM prompting has a lot of hacks. Half of them are useless, of course, while the other half are critical for success. The trick is: which one is which?
