I think that - in order to model the next word better AND properly respond to RLHF - the model simply encodes a lot of information in its billions of weights, which can be likened to billions of characters of code in some DSL.
However, due to the nature of the training, although it is very broad it is quite shallow. Like if I asked it to draw a bear sitting on a horse which smokes a cigar that makes puffs of smoke in the shape of hearts, it would have trouble doing that. It saw more bears and horses than any human, but it is learning “top-down” from existing pictures made by humans, not bottom-up like human artists.
I would wager that any “deep” result that is more than one level from the broad knowledge it has, is actually from a work uploaded by a human. Like the guy who actually knows how cigars look and can draw details. Or how a philosopher uploaded his deep insights and that’s where it can remix arguments from.
And similarly, a lot of GPT output seems to be very anodyne and generic. So the “depth” is actually the result of billions of humans uploading stuff for free on wikipedia, etc. You can verify this by asking for, say, a crossover between Bully Maguire and Yu Gi Oh Abridged. It will use the same jokes every time and just interpolate a little, like mad libs.
Now that is not to say that vapid, shallow things at scale can’t make money. Our society has a ton of it. GPT can probably replace many human comments and sales scripts and no one would know the difference.
But I think the real floodgates will unlock when we can train a model on a specific corpus of sales scripts, or dating site messages, or writings of Bill Gates etc. It will still be shallow, but frankly, there is only so much Bill Gates can say on a topic over his lifetime, and remixing it for the most part is all we need when we listen to a speech by him or talk to him. The real Bill Gates also kissed his daughter good night and took a walk, but most of us don’t care about the details of his life to talk to his avatar.
Right now it is still very expensive to train a model on someone’s entire corpus, and moreover that would mean giving it over to a corporation, to impersonate you and exploit your likeness.
Is there anything on the horizon for cheap and private trainings?
> Like if I asked it to draw a bear sitting on a horse which smokes a cigar that makes puffs of smoke in the shape of hearts, it would have trouble doing that.
um. have you seen deepfloyd? dont be so confident...
However, due to the nature of the training, although it is very broad it is quite shallow. Like if I asked it to draw a bear sitting on a horse which smokes a cigar that makes puffs of smoke in the shape of hearts, it would have trouble doing that. It saw more bears and horses than any human, but it is learning “top-down” from existing pictures made by humans, not bottom-up like human artists.
I would wager that any “deep” result that is more than one level from the broad knowledge it has, is actually from a work uploaded by a human. Like the guy who actually knows how cigars look and can draw details. Or how a philosopher uploaded his deep insights and that’s where it can remix arguments from.
And similarly, a lot of GPT output seems to be very anodyne and generic. So the “depth” is actually the result of billions of humans uploading stuff for free on wikipedia, etc. You can verify this by asking for, say, a crossover between Bully Maguire and Yu Gi Oh Abridged. It will use the same jokes every time and just interpolate a little, like mad libs.
Now that is not to say that vapid, shallow things at scale can’t make money. Our society has a ton of it. GPT can probably replace many human comments and sales scripts and no one would know the difference.
But I think the real floodgates will unlock when we can train a model on a specific corpus of sales scripts, or dating site messages, or writings of Bill Gates etc. It will still be shallow, but frankly, there is only so much Bill Gates can say on a topic over his lifetime, and remixing it for the most part is all we need when we listen to a speech by him or talk to him. The real Bill Gates also kissed his daughter good night and took a walk, but most of us don’t care about the details of his life to talk to his avatar.
Right now it is still very expensive to train a model on someone’s entire corpus, and moreover that would mean giving it over to a corporation, to impersonate you and exploit your likeness.
Is there anything on the horizon for cheap and private trainings?