If you allocate a relatively big chunk of memory for each unit of work, and at some point your allocation fails, you can just drop that unit of work. What is not practical?
I think in that case overcommit will happily say the allocation worked. Unless you also zero the entire chunk of memory and then get OOM killed on the write.
I suppose you can try to reliable target "seriously wild allocation fails" without leaving too much memory on the table.
0: Heuristic overcommit handling. Obvious overcommits of
address space are refused. Used for a typical system. It
ensures a seriously wild allocation fails while allowing
overcommit to reduce swap usage. root is allowed to
allocate slightly more memory in this mode. This is the
default.
None of that matters: what is your application going to do if it tries to allocate 3mb of data from your 2mb allocator?
This is the far more meaningful part of the original comment:
> and furthermore most code is not in a position to do anything other than crash in an OOM scenario
Given that (unlike a language such as Zig) Rust doesn’t use a variety of different allocator types within a given system, choosing to reliably panic with a reasonable message and stack/trace is a very reasonable mindset to have.
Since we're talking about SQLite, by far the most memory it allocates is for the page cache.
If some allocation fails, the error bubbles up until a safe place, where some pages can be dropped from the cache, and the operation that failed can be tried again.
All this requires is that bubbling up this specific error condition doesn't allocate. Which SQLite purportedly tests.
I'll note that this is not entirely dissimilar to a system where an allocation that can't be immediately satisfied triggers a full garbage collection cycle before an OOM is raised (and where some data might be held through soft/weak pointers and dropped under pressure), just implemented in library code.
Sure, and this is completely sensible to do in a library.
But that’s not the point: what can most applications do when SQLite tells them that it encountered a memory error and couldn’t complete the transaction?
Abort and report an error to the user. In a CLI this would be a panic/abort, and in a service that would usually be implemented as a panic handler (which also catches other errors) that attempts to return an error response.
In this context, who cares if it’s an OOM error or another fatal exception? The outcome is the same.
Of course that’s not universal, but it covers 99% of use cases.
The topic is whether Rust should be used to re-implement SQLite.
If SQLite fails to allocate memory for a string or blob, it bubbles up the error, frees some data, and maybe tries again.
Your app may be "hopeless" if the error bubbles up all the way to it, that's your choice, but SQLite may have already handled the error internally, retried, and given your answer without you noticing.
Or it may at least have rolled back your transaction cleanly, instead of immediately crashing at the point of the failed allocation. And although crashing should not corrupt your database, a clean rollback is much faster to recover from, even if your app then decides to crash.
Your app, e.g. an HTTP server, might decide to drop the request, maybe close that SQLite connection, and stay alive to handle other ongoing and new requests.
SQLite wants to be programmed in a language were a failed allocation doesn't crash, and unlike most other code, SQLite is actually tested for how it behaves when malloc fails.
In C++ it will throw an exception which you can catch, and then gracefully report that the operation exceeded limits and/or perform some fallback.
Historically, a lot of C code fails to handle memory allocation failure properly because checking malloc etc for null result is too much work — C code tends to calm that a lot.
Bjarne Stroustrup added exceptions to C++ in part so that you could write programs that easily recover when memory allocation fails - that was the original motivation for exceptions.
In this one way, rust is a step backwards towards C. I hope that rust comes up with a better story around this, because in some applications it does matter.
In the world of confusing landing pages, this project is a piece of art: what the fuck does any of this mean
> EXCITABLE. EXACTABLE. EXECUTABLE. A shared universe of destinations and memory.
> Space is a SaaS platform presented by PromptFluid. It contains a collection of tools and toys that are released on a regular cadence.
> PromptFluid Official Briefing: You are reading from the ground. Space is above. We transmit because the colony cannot afford silence.
> You can ignore the story and still use everything. But if you want the deeper current: this relay exists because the colony is fragile, and Space is the only place the tools can grow without choking the ground.
> Creating an account creates a Star. Your Star is your identity within Space and unlocks Spacewalking capabilities and certain tools that require persistent state
Interesting read, but I feel like they should have also benchmarked using COPY with Postgres. This should be far faster than a bulk insert, and it’s more in line with what they are benchmarking.
I maintain a project that publishes a SQLite file containing all package metadata, if you don’t want to use BigQuery or the API to do this kind of analysis
They wanted android developers that used google play services to basically be able to submit the same app to the amazon/fire store (without major revisions), so they reverse engineered the framework used by Google for api/hooks between the apps/apks and the "play-services"/OS levels.
Sort of spoofed the environment to prioritize compatibility in order to make it as easy as possible to grow the Amazon app store.
People don't realize that despite Android being nominally 'Open Source', the closed source Google layer on most phones makes it very difficult to exclude Google entirely from the picture and have a user friendly phone environment (both end-user and app-developer/playstore-user).
Basically only Amazon and China had resources to counter it directly with Android, or you could drop that layer and go the less user friendly route of st like AOSP 'pure' phones.
> Right now the rules is Linus can force you whatever he wants (it's his project obviously) and I think he needs to spell that out including the expectations for contributors very clearly.
>
> For myself I can and do deal with Rust itself fine, I'd love bringing the kernel into a more memory safe world, but dealing with an uncontrolled multi-language codebase is a pretty sure way to get me to spend my spare time on something else. I've heard a few other folks mumble something similar, but not everyone is quite as outspoken.
He gets villianized and I don't think all his interactions were great, but this seems pretty reasonable and more or less in line with what other people were asking for (clearer direction from Linus).
That said, I don't know, maybe Linus's position was already clear...
In many languages, like Italian which I am a native speaker of, to "spread like a cancer" doesn't have the negative subtext of the English idiom. It just means it spreads, wildly, uncontrollable. In English it gets muddled with the very negative idiom of "being a cancer", i.e. being very bad if not fatal.
I think it's because in English-speaking places (I'll say "The US and some rounding errors" to be explicit) the fact is that for a long time, cancer was a death sentence. This led to anything that is hard to kill as being called cancerous and the avoidance of such things is important (yes, this is where you chuckle and mime smoking a cigarette. There's still a population of the US that believes "smoking causes cancer" is a conspiracy by Big Pharma to push more cancer treatments or some bullshit like that.)
Calling something "cancerous" is to say it was an incurable disease that unless stamped out with some amount of precision will continue to cause rot and decay. Be it correct or not, saying "The cancer that is killing HN" is pointing a finger at a problem and scapegoating all the other problems onto it.
That's a good distinction, and it pretty much captures the exchange. Both sides felt quite strongly; Helwig used strong words. But that doesn't mean either side was unreasonable, despite some of us commenters being discomforted.
I’m pretty sure this is all completely fabricated LLM slop, created as a vehicle for the “handbook” adverts littered throughout.
1. https://www.linkedin.com/in/devrimozcay
reply