Honestly a lot of this ML to me seems eerily similar to how in older times people would use sheep entrails or crow droppings to try and predict the future. I mean basically that is what ML is, trying to predict the future, the difference is they called it magic, we call it math, but both seem to have about the same outcome, or understandability.
> I mean basically that is what ML is, trying to predict the future
If being so reductive, that's also the scientific method. Form a model on some existing data, with the goal of it being predictive on new unseen data. Key is in favoring the more predictive models.
> they called it magic, we call it math, but both seem to have about the same outcome
One of the oft overlooked, yet critically important aspects of the scientific method is the hypothesis. You don’t design an experiment having absolutely no idea what to expect. You have an educated guess in mind (the hypothesis), and you design the experiment in such a way that says “this result will rule out my hypothesis, while this other result might confirm it.”
Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.
Note that I'm being intentionally reductive to argue that the net buscoquadnary threw around ML models and future-telling with sheep entrails also includes the scientific method.
I do also think that ML as a field progresses through the scientific method ("I theorise that this network with residual connections will converge faster, lets see if there's a significant difference") - but maybe not to the full extent it could.
> Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.
I'd say that's closer to evolutionary algorithms. GD finds (locally) the direction to tweak the weights to improve predictions on a given batch.
