My boss has been passing off Claude generated code and documentation to me all year. It is consistently garbage. It consistently hallucinates. I consistently have to rewrite most, if not all, of what I'm handed.
I do also try and use Claude Code for certain tasks. More often than not, i regret it, but I've started to zero in on tasks it's helpful with (configuration and debugging, not so much coding).
But it's very easy then for me to hear people saying that AI gives them so much useful code, and for me to assume that they are like my boss: not examining that code carefully, or not holding their output to particularly high standards, or aren't responsible for the maintenance and thus don't need to care. That doesn't mean they're lying, but it doesn't mean they're right.
Not everyone is your boss. I have 15 years of experience coding. So when the AI hallucinates, I call that out and it improves the code it does create. If someone is passing off Ai's first pass as done, they are not using the tool correctly.
My boss has 28 years of experience coding so that clearly isn't the deciding factor here.
Yes, i suppose it is theoretically possible that you are that much better than my boss and i at coaxing good output from an LLM, but I'm going to continue to be skeptical until i see it with my own eyes.
This is basically me at my job right now. My boss used Claude Code in his spare time to write a "proof of concept" Electron app. It mostly worked but had some weird edge case behaviors. Now it's handed off to me, and fixing those edge cases is requiring me to refactor basically every single thing Claude touched. Vast majority I'm just tossing and redoing from scratch.
The original code "looks" fine, and it works pretty well even, but an LLM cannot avoid critical oversights along the way, and is fundamentally designed to its mistakes look as plausibly correct as possible. This makes correcting the problems down the line much more annoying (unless you can afford to live with the bugs and keep slapping on more band aids, i guess)
I don't know that i consider recognizing the limitations of a tool to be resistance to the idea. It makes sense that experts would recognize those limitations most acutely -- my $30 harbor freight circular saw is a lifesaver for me when I'm doing slapdash work in my shed, but it'd be a critical liability for a professional carpenter needing precision cuts. That doesn't mean the professional carpenter is resistant to the idea of using power saws, just that they necessarily must be more discerning than I do.
The thing is, Claude Code is great for unimportant casual projects, and genuinely very bad at working in big, complex, established projects. The latter of course being the ones most people actually work on.
Well either it's bad at it, or everyone on my team is bad at prompting. Given how dedicated my boss has been to using Claude for everything for the past year and the output continuing to be garbage, though, i don't think it's a lack of effort on the team's part, i have to believe Claude just isn't good at my job.
I was going to try having an AI agent analyze a well-established open source project. I was thinking of trying something like Bitcoin Core or an open-source JavaScript library, something that has had a lot of human eyes on it. To me, that seems like a good use case, as some of those projects can get pretty complex in what they're aiming to accomplish. Just the sheer amount of complexity involved in Bitcoin, for instance, would be a good candidate for having an AI agent explain the code to you as you're reviewing it. A lot of those projects are fairly well-written as they are, with the higher-level concepts being the more difficult thing to grasp.
Not attempting to claim anything against your company, but I've worked for enterprises where code bases were a complete mess and even the product itself didn't have a clear goal. That's likely not the ideal candidate for AI systems to augment.
Frankly, the code isn't messy whatsoever. There's just lots of it, and it's necessarily complex due to the domain. It's honestly the best codebase I've ever worked with - i shudder to think what nonsense Claude would spew trying to contextualize the spaghetti at my last job
As context size increases, AI becomes exponentially dumber. Most established software is far, FAR too large for AI. But small, greenfield projects are amazing for something like Claude Code.
This is why I argue that the impact of LLMs is in the tail. Its all the small to midsize shops that want something done, but don't have money to hire a programmer. Its small tasks, like pushing data around, writing a quick interface to help day to day jobs in niche jobs and technical problems. Its the ability to quickly generate prototype logos and scripts for small scale ad campaigns, for solving Nancy's Excel issue, etc. Big companies have big software and code stacks with tons of dependencies. Small shops have little project needs that solve significant issues facing their operations, but will unlikely become large enough that things like scaling issues, maintenance, integration, are ever a problem at all. Its a tail, but its long in small to midsize businesses. In research labs, which I have personal experience, AI is rapidly making feasible more ambitious projects, quicker timelines, and better code, generally.
This is the first time I've ever had a problem with the USB enclosures. And its fantastically rare, roughly one corrupt 512b block per TB of data written. With a btrfs-raid1 it's self-correcting on reads, if I didn't look at dmesg I'd never know.
I've figured out it only happens if I'm streaming data over the NIC at the same time as writing to the disks (while copying from one local volume to another), but that's all I really know right now. I seriously doubt it's a software bug.
I do also try and use Claude Code for certain tasks. More often than not, i regret it, but I've started to zero in on tasks it's helpful with (configuration and debugging, not so much coding).
But it's very easy then for me to hear people saying that AI gives them so much useful code, and for me to assume that they are like my boss: not examining that code carefully, or not holding their output to particularly high standards, or aren't responsible for the maintenance and thus don't need to care. That doesn't mean they're lying, but it doesn't mean they're right.
reply