Only tangentially related, but today I found a repo that appears to be developed using AI assistance, and the costs for running the agents are reported in the PRs. For example, 50 USD to remove some code: https://github.com/coder/mux/pull/1658
Lol, this is my PR. That cost is misleading. That workspace did far more than that change. In reality I spend ~$1000/week in tokens for all of my development work, and I'm quite happy with the exchange.
This is the cost of Anthropics pay by the token plan.
To give an analogy, Anthropics pricing is $0.10 per grain of rice (pay by the token) or;
$20 a month for quarter cup of rice each day (claude pro)
$100 a month for 10 cups of rice each day (claude max 100)
$200 a month for a sack of rice delivered to your door each day.
It's a rather insane pricing scale and here they are paying $50 there because they don't understand the pricing model (which is fair, Anthropics pricing model is crazy). Never pay by the token with Antropic! Only ever use the subscription plans.
At work we pay per token because we use a third party tool (amp) and it is very pricy. OTOH the subscription model is obviously not profitable. It's priced to capture market share, like Uber in 2010. (TBH the token pricing is probably not profitable either.)
From the point of view of a large entreprise, it makes more sense to pay a third party that can itself swap out Claude for Gemini (for example) based on pricing, then to buy subscriptions to one specific tool that is likely to suddenly cost 10x as much when they run out of VC funds. The dynamic is going to be different for individuals and small companies.
I'd like to suggest that dev lower their settings: sota model + high + thinking is definitely not needed to do this simple task. Lower settings could easily do it for less than $0.50, maybe even $0.05. I'd encourage people to operate on average to low settings and wind then up or down depending on the prompt task complexity.
That seems reasonable compared to an actual developer (depending on your region), but I had hoped for these models to make simple tasks like this fast and cheap so those developers can focus on the difficult stuff.
Yes in practice I've done fairly simple tasks using AMP at work and ended up with bills of 100-150$ for them which is somewhat less (but similar order of magnitude) as contracting out to LCoL countries. Depending on how the technology (towards cheaper cost) and financial situation (towards profitability and higher costs) evolve, it wouldn't be shocking to me that the frontier model end up more expensive than human contractors.
