As soon as a system gives you a receipt, a cryptographic proof, or even a reliable way to re-verify later, you've created something that can usually be repurposed as evidence for a third party
This feels like one of those cases where the technical consensus has been clear for years, but the policy and media narratives keep resetting to "maybe this time it's different."
The only real solution here is a strict egress filtering. The agent can fetch packages (npm/pip) via a proxy, but shouldn't be able to initiate connections to arbitrary IPs. If the agent needs to google, that should be done via the Supervisor, not from within the container. Network isolation is more reliable than content analysis
The logic is Defense in Depth. Even if the "cage" code is AI-written and imperfect, it still creates a barrier. The probability of AI accidentally writing malicious code is high. The probability of it accidentally writing code that bypasses the imperfect protection it wrote itself is much lower
Defense in depth doesn't mean throwing a die twice and hoping you don't get snake eyes.
The AI-generated docs claim that the AI-generated code only filters specific actions, so even if it manages to do that correctly it's not a lot of protection.
I'd add that for an ambitious financial tool (like yours), a VM might not be enough. Ideally, agents should run in ephemeral environments (firecracker microVMs) that are destroyed after each task. This solves both security and environment drift issues
Ah, let me clarify: I'm only using this to help me code faster. There are zero agents in the runtime for the financial tool.
As a matter of fact, the tool is zero-knowledge by design: state is decrypted in your browser and encrypted again before it leaves. There are no account integrations. The persistence layer sees noise. There are a couple of stateless backend tools that transiently see anonymous data to perform numerical optimizations.
Honestly stories like yours are the best part of this whole AI revolution. It’s genuinely cool that the technical barrier is no longer killing creative ideas. You’ve essentially skipped the "coder" stage and jumped straight to orchestrator (Product Owner + QA rolled into one), with AI acting as the diligent junior dev. That is a totally valid model
The only downside is that sooner or later, you hit the scaling trap. When a project grows from a silly website into a real product with users, not understanding how exactly the AI stitched those code blocks together becomes a critical risk. AI is great at putting up walls and painting facades, but the foundation (architecture and security) still needs human verification, especially once other people's data gets involved
Trying to fix broken architecture with the same AI that wrote it leads to recursive technical debt. AI can rewrite code, but it cannot make strategic decisions if the operator doesn't understand the problem. In the end, instead of a fix, you get a Big Ball of Mud, just built 10x faster
Agreed on the point that people will become busier. It’s the Jevons Paradox in its purest form: increasing the efficiency of a resource leads not to saving it, but to increasing its consumption.
We used to skip building internal tools or complex integrations because it was expensive, now it's cheap so naturally the business wants everything. Future engineers won't be writing code so much as orchestrating the chaos generated by agents. We won't stop working, but the workload is definitely going up
reply