> The "intelligence" exhibited emergent from language itself, from the massive dataset it has trawled.
I concur. Intelligence does not come from the transformer architecture, or any specific of the model. It comes from the language corpus. Human intelligence too, except for physical stuff. All our advanced skills come from language.
You take 300GB of text and put it through a randomly initialised transformer and you get chatGPT. You immerse a baby in human language, it becomes a modern adult, with all our abilities. Without language, and that includes other humans and tech, we'd be just weaker apes.
I concur. Intelligence does not come from the transformer architecture, or any specific of the model. It comes from the language corpus. Human intelligence too, except for physical stuff. All our advanced skills come from language.
You take 300GB of text and put it through a randomly initialised transformer and you get chatGPT. You immerse a baby in human language, it becomes a modern adult, with all our abilities. Without language, and that includes other humans and tech, we'd be just weaker apes.