The one thing I really disagree with is the notion that there will be millions of identical AI images.
The next big step is continual learning, which enables long-term adaptive planning and "re-training" during deployment. AI with continual learning will have a larger portion of their physical deployment devoted to the unique memories they developed via individual experiences. The line between history/input context/training corpus will be blurred and deployed agents will go down long paths of self-differentiation via choosing what to train themselves on; eventually we'll end up with a diaspora of uniquely adapted agents.
Right now inference consists of one massive set of weights and biases duplicated for every consumer and a tiny unique memory file that gets loaded in as context to "remind" the AI of the experiences it had (or did it?) with this one user / deployment. Clearly, this is cheap and useful to scale up initially but nobody wants to spend the rest of their life with an agent that is just a commodity image.
In the future, I think we'll realize that adding more encyclopedic knowledge is not a net benefit for most common agents (but we will provide access to niche knowledge behind "domain-specific" gates, like an MoE model but possibly via MCP call), and instead allocate a lot more physical capacity to storing and processing individualized knowledge. Agents will slow down on becoming more book smart, but will become more street smart. Whether or not this "street smart" knowledge ever gets relayed back to a central corpora is probably mostly dependent on the incentives for the agent.
Certainly my biggest challenge after a year of developing an industrial R&D project with AI assistance is that it needs way, way more than 400k tokens of context to understand the project properly. The emerging knowledge graph tools are a step in the right direction, certainly, but they're not nearly integrated enough. From my perspective, we're facing a fundamental limitation: as long as we're on the Transformers architecture with O(n^2) attention scaling, I will never get a sufficiently contextualized model response. Period.
You might notice this yourself if you ask Claude 4.5 (knowledge cutoff Jan 2025) to ramp up on geopolitical topics over the past year. It is just not physically possible in 400k tokens. Architectures like Mamba or HOPE or Sutton's OAK may eventually fix this, and we'll see a long-term future resembling Excession; where individual agents develop in enormously different ways, even if they came from the same base image.
Regardless of net efficiency, that still entails collecting CO2 at a central facility (where it could have been dealt with in other ways, such as injection underground) and sprinkling it through the air as you fly over delicate ecosystems. I'm sure bankers see both as net zero, but condors might have more issues with your simpler workaround.
> sprinkling it through the air as you fly over delicate ecosystems
I wouldn’t be so sure spraying water vapour is innocuous. As long as it’s atmospheric CO2, the environmental impact of synthetic fuels is much less than rebuilding the world’s air fleet and fuelling infrastructure to accommodate hydrogen.
The difference in volumetric energy density is not that big though, and hydrogen is not as flexible as jet fuel or even batteries when it comes to how you can store it in the vehicle.
To be fair, high gravimetric density is a fairly large advantage for an air plane. But the bad volumetric energy density does present some serious challenges.
Most internal combustion engine cars have a lead acid battery to start it up and run the spark plugs (or preheat the glow plugs if diesel). They don't get called "hybrid" or "battery powered" because the batteries aren't the propulsive power themselves.
This is akin to that: the batteries run the pumps, they're not the propulsive system itself.
Ion drives can be run off battery, but you can't launch with those.
Unlike a car battery though, these batteries provide a not-insignificant part of the energy that is generated by the engine. Each Rutherford engine generates around 37 mega-watts of power at sea-level (24900 N and 3.05 km/s exhaust velocity, Power = 1/2 * Thrust * v_e) and there are nine in the first stage. The first stage battery provides around one megawatt [1].
That's about 0.3% of all energy generated by the engines, which is significantly more than what a spark plug does in an ICE.
This is the closest we have to electric power directly powering the ascent of a rocket from Earth.
Something like a HyperCurie engine (which is also electric pump-fed), could probably lift off from a planetary body like the moon. When they used it in orbit, they actually had to wait for the batteries to charge up from solar panels between each engine burn.
It is about 1 megawatt (1341 HP) of the power pushing the rocket into the sky (directly translated into exhaust velocity and therefore thrust). That would be like a spark plug generating 1 HP in a 300 HP engine (Which would exhaust the typical car battery in about 1 second if it could even push that much power out).
Rough estimates I've seen say the starter motor is about that, though. (Not that I can tell real pages from GenAI ad content farms, I'm not a petrolhead).
I'd agree "it's all semantics", but yours are confusing me :P
(And for energy content, like for like is comparing the size of the fuel tank with the capacity of the battery, but cars aren't 90% fuel by weight).
This is awesome! Presumably you can make this work with any interface that doesn't enforce the total internal reflectivity of a fiber optic cable, and therefore allows light to leak out. Instead of an air gap, have you tried experimenting with removing the cladding of the fiber optic cable, but keeping the core intact?
Alteratively, could you use a short segment of colored cladding that allows certain wavelengths to leak out more than others? I think that would allow you to encode each bend point as a different color-- which might require a different (more expensive) rx sensor, but could be useful for certain applications.
I did experiment with various ways of allowing light to escape but nothing came close to the properties of a total air gap. You can actually measure (relative) bend angle with it like a protractor since the attenuation is very linear!
There is already existing work that uses colored segments for something similar but those techniques are hard to do outside a well equipped lab.
I've run in to exactly that. By putting my SearXNG on a machine that also does NAT for a busy network, this can be avoided. This is definitely one instance where IPs from a colo are a bad thing and residential IPs are a good thing ;)
The next big step is continual learning, which enables long-term adaptive planning and "re-training" during deployment. AI with continual learning will have a larger portion of their physical deployment devoted to the unique memories they developed via individual experiences. The line between history/input context/training corpus will be blurred and deployed agents will go down long paths of self-differentiation via choosing what to train themselves on; eventually we'll end up with a diaspora of uniquely adapted agents.
Right now inference consists of one massive set of weights and biases duplicated for every consumer and a tiny unique memory file that gets loaded in as context to "remind" the AI of the experiences it had (or did it?) with this one user / deployment. Clearly, this is cheap and useful to scale up initially but nobody wants to spend the rest of their life with an agent that is just a commodity image.
In the future, I think we'll realize that adding more encyclopedic knowledge is not a net benefit for most common agents (but we will provide access to niche knowledge behind "domain-specific" gates, like an MoE model but possibly via MCP call), and instead allocate a lot more physical capacity to storing and processing individualized knowledge. Agents will slow down on becoming more book smart, but will become more street smart. Whether or not this "street smart" knowledge ever gets relayed back to a central corpora is probably mostly dependent on the incentives for the agent.
Certainly my biggest challenge after a year of developing an industrial R&D project with AI assistance is that it needs way, way more than 400k tokens of context to understand the project properly. The emerging knowledge graph tools are a step in the right direction, certainly, but they're not nearly integrated enough. From my perspective, we're facing a fundamental limitation: as long as we're on the Transformers architecture with O(n^2) attention scaling, I will never get a sufficiently contextualized model response. Period.
You might notice this yourself if you ask Claude 4.5 (knowledge cutoff Jan 2025) to ramp up on geopolitical topics over the past year. It is just not physically possible in 400k tokens. Architectures like Mamba or HOPE or Sutton's OAK may eventually fix this, and we'll see a long-term future resembling Excession; where individual agents develop in enormously different ways, even if they came from the same base image.