> If you want rigorous controlled studies you'll get it in due time.
I hope so, because the alternative is grim. But to be quite honest I don't expect it'll happen, based on what I've seen so far. Obviously your experience is different, and you probably don't agree--which is fine. That's the great thing about science. When done properly it transcends personal experience, "common sense", faith, and other imprecise ways of thinking. It obviates the need to agree--you have a result and if the methodology is sound in the famous words of Dr. Malcolm "well, there it is." The reason I think we won't get results showing AI tooling meaningfully impacts worker productivity are twofold:
(1) Early results indicate it doesn't. Experiences differ of course but overall it doesn't seem like the tools are measurably moving the needle one way or the other. That could change over time.
(2) It would be extremely favorably in the interests of companies selling AI dev tools to show clearly and inarguably that the things they're selling actually do something. Quantifying this value would help them set prices. They must be analyzing this problem, but they're not publishing or otherwise communicating their findings. Why? I can only conclude it's because they're not favorable.
So given these two indications at this point in time, a placebo like effect seems most likely. That would not inspire me to sign a purchase agreement. This makes me afraid for the economy.
I hope so, because the alternative is grim. But to be quite honest I don't expect it'll happen, based on what I've seen so far. Obviously your experience is different, and you probably don't agree--which is fine. That's the great thing about science. When done properly it transcends personal experience, "common sense", faith, and other imprecise ways of thinking. It obviates the need to agree--you have a result and if the methodology is sound in the famous words of Dr. Malcolm "well, there it is." The reason I think we won't get results showing AI tooling meaningfully impacts worker productivity are twofold:
(1) Early results indicate it doesn't. Experiences differ of course but overall it doesn't seem like the tools are measurably moving the needle one way or the other. That could change over time.
(2) It would be extremely favorably in the interests of companies selling AI dev tools to show clearly and inarguably that the things they're selling actually do something. Quantifying this value would help them set prices. They must be analyzing this problem, but they're not publishing or otherwise communicating their findings. Why? I can only conclude it's because they're not favorable.
So given these two indications at this point in time, a placebo like effect seems most likely. That would not inspire me to sign a purchase agreement. This makes me afraid for the economy.