But I/O being the bottleneck never was about sequential reads, was it? I get the point of the article, though.

With modern CXL/PCIe, I guess it's not going to be that stupid to claim that RAM/memory controller is slowly becoming I/O on its own.

Old IBM's term for RAM was "storage."

I wonder whether the current huge funding in AI will ever lead to a revolution in computer architecture. Modern PCIe/CXL is already starting to blur the difference between memory and I/O. Maybe the future is going to be that CPUs, RAM, storage devices, GPUs and other devices are going to directly address one another like a mesh network. Maybe the entire virtual memory model will change to include everything to be addressed via a unified virtual memory space from a process/CPU perspective with simple read/write syscalls that translate into network packets flowing between the CPU and the destination (e.g. RAM/GPU/NVMe) and vice versa.

Don't we already almost (but not quite) have that? PCIe devices can talk directly to each other (still centralized AFAIK though) and from the programmers perspective everything you mentioned is mapped into a single unified address space prior to use (admittedly piecemeal via mmap for peripherals).

Technically there's nothing stopping me from mmaping an entire multi-terabyte nvme at the block level except for the part where I don't want to reimplement a filesystem from scratch in addition to needing to share it between lots of different programs.

When I took my first database course one topic was IO performance as measured in seek time on regular old hard drives.

You can’t really optimise your code for faster sequential reads than the IO is capable of, so the only thing really worth focusing on is how to optimise everything that isn’t already sequential.

I'm wondering what exact issue you are referring to with Databricks? I can't remember a time I had to change a line I wrote during the past 2.5 years I've been using it. Or are you talking about non-breaking changes?

They have changed a lot of their DLT (not even called that anymore lol, it's Lakeflow Pipelines now I think) syntax. I tried asking ChatGPT to convert a very simple Python one to Spark SQL, and it gave me a bunch of outdated SQL syntax.

Aside from that, if you use their Python connector package, it's a shit show to put it mildly. For example, 15.4 works with serverless but tells you (via deprecation warning) it doesn't and that you need to use 15.1 (which lacks a lot of variant stuff). So then you decide screw it I'm gonna just update to 16, except that serverless (which works on 15.4) doesn't work on 16.0 or 17.0.

At least in the Java world it is common to let the logging framework handle parameter evaluation for you.

This feels like one of those "not obvious until you've seen it in production" requirements: any production-ready logging framework should have a mechanism to delay parameter evaluation until after the threshold/destination checks are performed. Most languages have some version of deferred execution (lazy evaluation, thunks, etc.)

Not building your city in the desert is also a good idea when it comes to being a comfortable city.

It costs less to cool than to heat, by and large. And deserts have a lot of sunshine that can be converted into electricity for cooling...

But lack of water will become a huge problem when your city is growing that fast in a heating climate.

Edit: And cooling only works inside buildings or cars. Part of a comfortable city is being able to go outside and have a social life outside of a casino.

Municipal uses like drinking, showering, and watering ornamental plants is a tiny pct of desert water use. Most of it is crop irrigation, because if you can will water into existence then crops grow great in sunny deserts.

If the US' alfalfa exports to Saudi Arabia went down by 10%, we would never have a municipal water shortage in the American West in the next century.

Lack of water is a political problem. We have vast oceans which can be desalinated. Israel gets 85% of its water from desalination, they have gone from water shortages to having a water surplus.

We pump oil via pipelines vast distances, we can do the same with water.

We have virtually unlimited energy locked in Uranium which could power desalination plants, or heck you could power them with solar.

There’s plenty of water for the whole planet. There’s also plenty of clean energy (see nuclear and solar point earlier). But tapping these resources requires a functional government or at least a bureaucracy willing to allow companies to build.

There are people pushing for more shade in cities as an adaptation for a warming world. There is some crossover with the push for a reduction in cars and generally reducing the footprint of streets. Looking at old cities in hot climates I can see how this could make sense.

For Las Vegas, Cottonwoods are native and grow pretty quickly. Like many poplars they were used to grow shelterbelts.

I don't think anybody ever moved to Vegas expecting to be able to have a street life.

I used to hang out with my friends and neighbors outside all the time in Las Vegas.

...just not so much in May-August.

That's a great idea, but hard for the city of Las Vegas to implement. Clark County doesn't have any ability to build a city not in a desert either. The state of Nevada doesn't have much of anywhere to put a non-desert city either.

Very few municipalities are willing to deny new residents, either. It wouldn't be anywhere on my list of viable places to live, but population growth in the Las Vegas metro area has been consistently large since 1910 until recently (only 10% growth from 2010 to 2020). The municipalities should likely invest in livability and comfort where possible.

Extending the format just because you can – and breaking backwards compatibility along the way – is even more pointless.

If you've created an extensible file format, but you never need to extend it, you've done everything right, I'd say.

> What about an extensible format that would have as part of header an algorithm (in some recognized DSL) of how to decompress it (or any other step required for image manipulation)?

Will sooner or later be used to implement RCEs. Even if you could do a restriction as is done for eBPF, that code still has to execute.

Best would be not to extend it.

How long does this procedure take in comparison to the network transfer?

My first try would've been to copy the db file first, gzip it and then transfer it but I can't tell whether compression will be that useful in binary format.

The sqlite file format (https://www.sqlite.org/fileformat.html) does not talk about compression, so I would wager unless you are storing already compressed content (media maybe?) or random numbers (encrypted data), it should compress reasonably well.

Native compression in sqlite is offered as a closed and licensed extension.

https://sqlite.org/com/cerod.html

There's an industrial robot arm built out of LEGO Technic bricks by OrangeApps, a small company related to German robot manufacturer KUKA. [1] It's primarily used for educational purposes.

Disclaimer: I work for a subsidiary of KUKA.

[1]: https://www.orangeapps.de/?lng=en&page=apps%2Fers3

Your parent comment just seems to make their point and it's technically correct, but might have missed the nuance. All people in this conversation seemed fine.

Are you sure that the following doesn't apply to your own comment?

> It's this type of comment that makes people be needlessly careful on this site more than any other.

If you learn about the Wuppertal suspended monorail, you should also read about Tuffi, the elephant that jumped off of it in 1950.

https://en.m.wikipedia.org/wiki/Tuffi

The video contains a segment about Tuffi.

The question is whether roads are a good thing.