Self plug: I made Jupyter notebooks for each chapter of this and the DFT and Physical Modeling books in this series, with Python animations/audio for some key concepts:

https://karlhiner.com/jupyter_notebooks/mathematics_of_the_d...

https://karlhiner.com/jupyter_notebooks/intro_to_digital_fil...

https://karlhiner.com/jupyter_notebooks/physical_audio_signa...

My god, animating convolution makes it so much easier to understand than having a professor draw the process on a chalkboard back in the day.

There's also a nice 3blue1brown video on the subject

Thank you: these are excellent.

Throwing in my vote - I wasn’t confused, saw your GH link and a “Zero to Hero” course name on RL, seems clear to me and “Zero to Hero” is a classic title for a first course, nice that you gave props to Andrea too! Multiple people can and should make ML guides and reference each other. Thanks for putting in the time to share your learnings and make a fantastic resource out of it!

Thanks a lot. It makes me feel better to hear that the post is not completely confusing and appropriating - I really didn't mean that, or to use it as a trick for attention.

Fixed, sorry about that

Someone else pointed this out as well - I didn't realize the original MediaFire link still worked. Updated to reflect.

Good catch! I didn’t realize the original link still worked. Also didn’t know about the “why” notes on WB index. Thanks!

Updated to reflect this.

One of the first programs I ever wrote - a program to find valid English crossword fills given a grid pattern with optional partial completions.

This project came to mind recently and I looked around on the Wayback Machine. Turns out I posted the jar on MediaFire and linked to it on an old blog on Jan 2, 2011. Luckily, there was [_one capture_ of the jar on MediaFire](https://web.archive.org/web/20240123154949/https://download1...) from oddly recently (Jan 23 2024). I downloaded and opened it on my 2023 MacBook Air, and it ran! Since it's Java, I'm guessing it runs on other computers, too :)

It was a delight to find it still working. Anyone else ever find an old program you thought you lost and get it running again?

EDIT: As a couple observant folks pointed out, the original MediaFire link actually still works, so WB machine saved the day by making it possible to find the link, but the jar was also still hosted on the today-internet :)

Nice job. Now maybe you can make it a web application using CheerpJ.

I think by “old fashioned” op means they are old technologies, not that they are obsolete.

It’s been so fun watching Bun progressing as quickly as it has! Truly incredible work, and the blog post is full of real value and time saved for future me from beginning to end - huge congrats on the 1.0.0 release, excited to see where bun goes from here!

The test use case of constructing a bio for yourself, hoping it accurately summarizes all the extremely low sample size data it happens to have of you in its web crawled training data, seems like one of the worst possible use cases for ChatGPT. It’s right there on the main page that it’s not to be trusted with factual information like this. ChatGPT will hallucinate details. It’s remarkable to me actually how often it will refuse to hallucinate, given that’s basically what its job is. I don’t find it interesting to find all these edge cases where ChatGPT produces empirically false data. It doesn’t even have the ability to look things up! If I were the OP and wanted help writing my bio, I would first write the draft myself, then use ChatGPT to help with the editing, prose, grammar, style, etc. You are the expert on the factual details of your own life, and if you’re surprised that a language model trained on web crawled data ending in 2018 is not, then all I’ve learned is that you don’t know much about what this thing is.

I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook.

ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)

I feel like the 3 youre missing there is something along the lines of "people enjoy social validation and internet points, to the extent that pretty shit content thats low effort is something we enjoy generating"

That is true, 3 would help steel my strawman. I agree that we’ll increasingly have capabilities to generate and publish garbage that’s _just_ good enough to generate clicks, and incentives to do this. In addition, I think we’ll increasingly have tools to produce content that is much more rich, imaginative, insightful, and factually correct in our future. Some more interesting questions to me are then: What will the ratio be? How will that ratio compare with what we see today? How easily will I be able to identify misinformation when I care about factual accuracy (again, compared with today)? How easily will I be able to avoid the garbage, vs find the good stuff?

How can fact checking be better facilitated as tech develops? It seems like a distinct social issue to me, one that I imagine requires verification from people with reputation (which could be aided by improved social networks probably). Id be very interested to hear if you have other ideas, as you seem optimistic on this front and would like to share in that :)

I can think of a few directions for technology aiding in fact checking:

1. Much of finding out what’s true or false is about finding consistency amongst lots of observations. So this is the science direction. If you can analyze lots of data, say from first-hand direct measurements like from a scientific instrument, or analyzing second hand observational data, say from many news sources reporting on a political event. One could also imagine multimodal analysis combining these first and second-hand kinds of data to arrive at a consensus estimate of a “true” perspective. E.g. analyzing video and audio streams recorded at said political event, combined with many text reports of the event. So this point is about data mining, and jointly estimating semantic meaning from natural language and other kinds of data, in a way that’s consistent with everything else considered factual.

2. Provenance-tracking: Think block chain - if we can provably trace a piece of data back to its primary sources, tracking all its modifications along the way, this could help with establishing provenance, and verifying legitimacy of any modifications along the way.

3. Consensus, staking/voting, etc. A lot of deciding what’s true is about seeking consensus. One thing I’m generally optimistic about here is that, for any given fact “out there,” there are many more ways to describe it incorrectly than correctly. So even though it sounds scary for consensus to be an aspect of truth finding, it always will be, and at the very bottom it’s all we can hope for. One way that’s already getting traction to make consensus mean something, is the idea of staking. So you have to put something down on the table when you claim you believe something to be true. Software can (and already is) helping to build confidence behind some claims more than others by backing claims with value (money).

4. Humans are insanely bad at reasoning rationally, because of lots of reasons. Pick your favorite fallacy. We evolved to survive long enough to reproduce and rationality is a happy accident. One could imagine software being less susceptible to simple tricks, could be less incentivized to outright lie for personal gain or power seeking, or claiming to represent their actual beliefs when they are actually pursuing other goals by conveying something they don’t actually believe.