There's plenty of joking and 'it'll never work' here, but doesn't this worry anyone at all?
A system that 'knows you better than you know yourself', means a system that is able to predict your responses to different stimuli more accurately than you can.
By definition such a system can manipulate you.
Let's set aside the idea of an 'AI' - which conjures up all sorts of mythical images of talking computers.
Instead consider the implications of a single corporation with a machine learning system that has access to the private communications, personal documents, and physical location and web viewing history for a billion people and can use that information to predict the behavior of each one of them better than they can predict their own behavior.
That's what we'll get if Kurzweil and Google are only partially successful in this endeavor. It sounds to me like the greatest concentration of power in human history, and an extremely dangerous thing that we should be considering the implications of today.
Now many people seem to think that Moore's law means that machine learning systems of this power are inevitable and that it's better that it's Google than a worse entity like a government who controls it.
I think that's a dangerous dismissal for two reasons - firstly it presumes that this tool wouldn't be taken over by governments once it came into existence, or that it wouldn't be used to manipulate governments to prevent its takeover.
Secondly, even if super-powerful AI is an inevitability, it's not inevitable at all that we should be routing all of our personal information to its creator so that it can be used to model and predict our behavior en masse.
> Instead consider the implications of a single corporation with a machine learning system that has access to the private communications, personal documents, and physical location and web viewing history for a billion people and can use that information to predict the behavior of each one of them better than they can predict their own behavior.
Funny the corporation that makes that more visible to me is Netflix. After rating so many movies it can pretty well predict if I would like a move or not. It does it better than me reading the description and the title.
It is kind of nice but a bit worrying. Wonder what kind of psychological profile it could build and sell to other companies if it wanted to.
A system that 'knows you better than you know yourself', means a system that is able to predict your responses to different stimuli more accurately than you can.
By definition such a system can manipulate you.
Let's set aside the idea of an 'AI' - which conjures up all sorts of mythical images of talking computers.
Instead consider the implications of a single corporation with a machine learning system that has access to the private communications, personal documents, and physical location and web viewing history for a billion people and can use that information to predict the behavior of each one of them better than they can predict their own behavior.
That's what we'll get if Kurzweil and Google are only partially successful in this endeavor. It sounds to me like the greatest concentration of power in human history, and an extremely dangerous thing that we should be considering the implications of today.
Now many people seem to think that Moore's law means that machine learning systems of this power are inevitable and that it's better that it's Google than a worse entity like a government who controls it.
I think that's a dangerous dismissal for two reasons - firstly it presumes that this tool wouldn't be taken over by governments once it came into existence, or that it wouldn't be used to manipulate governments to prevent its takeover.
Secondly, even if super-powerful AI is an inevitability, it's not inevitable at all that we should be routing all of our personal information to its creator so that it can be used to model and predict our behavior en masse.