I've seen your work at Hard Work Party before, by the way! Really cool stuff, glad to see you've also got the startup going as well.

Maybe let each editor request one reshoot in the first week, a committee aggregates similar requests, all editors get all the reshoots once they're finalized.

Maybe include storyboards and a rubrik for what story the film is supposed to share and how we're meant to feel, but maybe not.

My GIS competition was fun too. They gave me a bunch of map data and I had to produce a report on Washington DC storm surge flood zones and potential rescue helicopter locations all within a couple hours.

I recall there being a video production category too. I didn't compete in it but you'd be given props and dialogue to turn into a video over the course of a day or two. Very few of the categories were contemporaneous competitions, most were long term project presentations.

Yes - but they've turned into something I'd really rather not watch: https://www.opus.pro/agent/human-creator-vs-ai

If you focus on fast typing/editing skills to level up, but still have bad decision-making skills, you'll just end up burying yourself (and possibly your team) faster and more decisively. (I have seen that, too.)

> how productive power users in different [fields] can be with their tools

There are a lot more tools in programming than your text editor. Linters, debuggers, AI assistants, version control, continuous integration, etc.

I personally know I'm terrible at using debuggers. Is this a shortcoming of mine? Probably. But I also feel debuggers could be a lot, lot better than they are right now.

I think for a lot of us reflecting at our workflow and seeing things we do that could be done more efficiently with better (usage of) tooling could pay off.

I recently integrated Lazy Polars and running analytics in background processes so I can reliably provide a fast table viewing experience on dataframes that would normally exhaust memory of the jupyter kernel. Analytics are run column by column and results are written to cache, if a column fits into memory individually, summary stats for the entire dataframe can be computed.

Here's a demo video of scrolling through 19M rows, and running background summary stats.

https://www.youtube.com/shorts/x1UnW4Y_tOk

Would putting an aftermarket oil pump in these modern engines protect them or is it a deeper design issue?

They spec the thinnest stuff they can get away with to add .0001mpg. Multiply that by all the Chevy 1500s GM makes or F150s Ford makes and you see the draw.

Sometimes it turns out that the thinnest stuff they can get away with just not quite thick enough at the margins or in transient conditions. And of course they stretch out the oil change interval to reduce on-paper TCO as well which doesn't help.

You can mitigate this with thicker oil (what GM did for the recall) by can go too far and create other oiling issues because thick oil drains back slower and going to some super high spec 0-W-<whatever> Euro oil may cause other problems related to soot and sludge so there's no silver bullet.

The "safe" advice most people give out is to use whatever the <nation with no emissions or fuel economy rules> version of your owners manual says to use for oil.

And if you have a high strung turbo engine you ought to take your oil change intervals seriously.

Also, what key decisions do other data catalogs make via your choices? What led to those decisions and what is the benefit to users?

I like to think of Marmot as more of "operational" catalog with more of a focus on usability for individual contributors and not just data engineers. The key focus being on simplicity, in terms of both deployments and usability.

The economically efficient way to get the fuel economy result would have been to increase gasoline taxes, but that's a non starter politically. Higher gas prices would allow people to choose to keep a cheap gas guzzling truck/car, buy a new more efficient and expensive car, or buy a new slightly more efficient slightly more expensive car. It would have been simpler though and given consumers more choice.

People’s desire to sit higher up and be in large vehicles, which have always been more expensive than smaller, lower vehicles, is what causes them to be bought. And once a significant portion have them, it becomes safer to be in one yourself, further incentivizing their purchase.

But 99% of the time, it’s just because people like the feeling of sitting higher up than others, and the ego boost from taking up more space. The simple evidence is the popularity of Suburbans/Sequoias/XC90s/etc over minivans, like Sienna/Odyssey. There is absolutely no functional benefit of the former over the latter, yet the former is more popular.

It makes no sense to buy a GM Suburban or Ford Expedition because you think a Stellantis Pacifica is low quality. The Japanese minivans have always been there for purchase, if you wanted a quality minivan.

People have been choosing to pay extra for bigger, taller cars because they want bigger, taller cars to signal ostentatious consumption, not any other reason. I’ve heard this direct from many, many people on why they chose an SUV or pickup truck over a minivan (though they will couch it in terms like “cool” or “sleek” or whatever).

There are very few countries where pedestrial fatalities have continued to rise, and the US and Canada are two of them, driven in large part by auto obesity.

You point to popularity, but I will mention that it is impossible to buy a sedan from US automakers today. The reason why is simple - profit. Larger cars are more profitable. When combined with incessant marketing that a pickup truck makes you more "manly", you can manufacture "desire" and "preference".

Toyota/Honda/Subaru/Mazda/Tesla/Volkswagen manufacture sedans made in the US, that you can buy today. Not sure why it would make a difference where it is made anyway.

If you wanted a lower priced sedan, you would choose from the 10+ great options, cheaper than a larger vehicle, and buy a sedan.

Which means if you paid more for a larger/higher vehicle, it is because you wanted the larger/higher vehicle.

This involves making it lazy for polars, allowing it to read arbitrarily large files no longer requiring loading the entire dataframe into memory. When a large dataframe initially displays, no summary stats will be available. Summary stats are computed in the background in groups of columns. Then results are cached per column. To accomplish this I wrote a polars plugin in rust that computes hashes of columns. Dealing with large data like this is tricky, operations sometimes crash, sometimes take all available memory, and sometimes they just run for a very long time. I have also been building an execution framework for Buckaroo. It uses multiprocessing based timeouts, and the caching to execute summary stats in the background.

Being able to control the execution, recover from timeouts, crashes and memory exhaustion opens up some interesting debugging tools. I have written methods that take arbitrary groups of polars expressions and produce a minimal reproduction test case through a git-bisect like process.

All of this assures that if individual columns of a dataframe fits into memory, summary stats will be computed for the entire dataframe in the background. And because it is cached, the next time you open the same dataframe, the stats will be display instantly. When exploring data I do this in an adhoc way manually (splitting up a dataframe by columns and rows), but it is error prone. This should all be automatic.

I will be presenting this at PyData Boston in December.

The Column's the limit: interactive exploration of larger than memory data sets in a notebook with Polars and Buckaroo

    use fory::{Fory, ForyObject};

    #[derive(ForyObject, Debug, PartialEq)]
    struct Struct {
        nan: f32,
        inf: f32,
    }

    fn main() {
        let mut fory = Fory::default();
        fory.register::<Struct>(1).unwrap();

        let original = Struct {
            nan: f32::NAN,
            inf: f32::INFINITY,
        };
        dbg!(&original);

        let serialized = fory.serialize(&original).unwrap();

        let back: Struct = fory.deserialize(&serialized).unwrap();
        dbg!(&back);
    }

     cargo run
       Compiling rust-seed v0.0.0-development (/home/random-code/fory-nan-inf)
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.28s
         Running `target/debug/fory-nan-inf`
    [src/main.rs:17:9] &original = Struct {
        nan: NaN,
        inf: inf,
    }
    [src/main.rs:22:9] &back = Struct {
        nan: NaN,
        inf: inf,
    }