Controversial opinion as a machine learning researcher:
Well roundedness (or teaching basic elements to many) will not pay off as much, it would likely yield better results for the country to invest heavily at the top.
Observation based on success of machine learning research in highly concentrated organisations (certain universities in the US, DeepMind, etc) versus countries with scattered expertise and resources (e.g. Germany).
If you can afford to, do both, but observationally, this does not seem to happen. Not for a lack of resources but rather the mindset and a society's perspective.
On the other hand, Finland punches massively above its weight when it comes to tech in general - while the US, being the world's largest economy, punches at its weight. That would indicate the generally egalitarian Finish approach is better than the darwinian US approach.
My feeling is that while the US is phenomenally good at delivering an excellent education to a small set of extremely talented people, I get the feeling that these people go on to not have enough free time to do really interesting work, and rather end up devoting their lives to industry, which is predominantly very conservative[1].
[1]: Obviously, I think this is offset by the fact that US industry is less conservative and more interested in abstract research than it is in most other countries, so you do get really cool research being done. But I think generally speaking, this model isn't as effective as just letting most people in society do more of the stuff that interests them, and less of the stuff that doesn't.
The whole idea that a country like Finland could create and maintain talent without a domestic market is completely silly, if not entirely unsupported. Which is also true for much of the US. Arguably even the Soviet Union couldn't.
And if you actually had to choose between "top talent" and "mainstream appeal" the latter has orders of magnitude better returns. A larger part of the modern economy is built on that fact, including much of Silicon Valley.
If you could just invest in talent many countries would pass the US in no time. But unsurprisingly those investments mostly yield handshakes and cocktail parties.
I think Silicon Valley is quite a good refutation of your point, actually -- it was almost entirely founded on DARPA investing in talent. After all, they funded everything from the first ICs to Chomsky.
I don't think mainstream appeal is irrelevant. I just think that if you fund scientists and technologists to do the work they find interesting, they'll produce something more fundamentally interesting and generally useful than if you fund them to make great products. Great products, after all, include things like Minecraft, pet rocks, and fidget spinners.
well, i don't know whether this is true. I am quite frustrated with germany, because our scattered expertise and resources are just not that competitive in this area. We were unable to retain our top people (Schmidhuber, Hochreiter, etc.), don't invest enough into our universities and our AI-initiative is also just not enough. A serious comparison to the top countries (United States, more interesting: Canada) does not end well.
If the TUM (Technical University of Munich) would have been sucessfull in retaining the top talents they developed, they would count Schmidhuber, Hochreiter, Schaal, etc. to their faculty and be competitive with top institutions. But they didn't and it's a symptom of a larger lack of investment. My university (KIT) also developed some pretty sucessfull researchers in "AI", but they left (for example to Carnegie Mellon, where there's a lot cooperation, especially in machine learning). Prof. Alex Waibel made a pretty important contribution to ConvNets, one could argue they developed out of his time-delay NNs. But, although he's still a Prof at KIT, he's not really that present in karlsruhe.
Just compare the total numbers of papers on NeurIPS of germany to other countries. We are getting obliterated.
I still think that a "reasonably" distributed approach is best for germany (and probably for finnland), but we have to get our shit together! I don't think this is due to our more distributed approach.
It's all pretty frustrating, i think there's a lot of wasted potential here.
Depends on the goal. If the goal is to maximize publications in good venues and have the occasional spectacular result, I think you are probably right.
However, if I grant that there is a lot of low-hanging fruit out there in the economy that can be attacked with already-existing approaches, then the Finnish approach makes sense to me.
Having a bunch of people who understand statistics, basic machine learning algorithms, and have learned the discipline/soft skills to use them successfully (cross-validate, don't overfit, etc.) might result in a lot of interesting problems being solved.
Education that provides basic outline for larger audience can be beneficial in different ways. If it can make people and decision makers more resistant to AI snake oil, it's a good thing.
Surely this depends on the results you're after? Sure, giving everyone a basic intro to AI isn't going to result in [m]any AI breakthrough discoveries or give you a workforce of AI researchers (though it might engage a few people in the subject that might otherwise have chosen to be quants). But it might help innoculate them against a lot of misconceptions about what the current state of AI can and can't do and encourage them to think critically about data sources and inferences that might be drawn from them.
If the objective is for applied research (research for the benefit of the economy), I disagree. My main argument is that we need domain experts outside of academia.
Moreover, theoretical research in AI is very open and countries with limited resources are better off stand on the shoulders of giants than try and be number 1 themselves. Finland might never beat Russia/China/US AI arms race but they just need to defend themselves should any of these 3 countries monopolize or weaponize on AI.
I have a more informed perspective. First, Finland introduced a browser (Erwise) two years before the W3c browser. Second, the U.S. is stumbling in terms of getting beyond analytic and ANN designs, and AGI is an area where few in the U.S. work. I have worked with Finland on cutting-edge tech in the past, and I wouldn't be surprised if they made some serious breakthroughs. It does not take a lot of researchers to excell in AGI, it only takes one bricoleur. (source, inventor of the Internet browser.)
Unlike the majority of our sciences, ML tech disperses extremely quickly. Software being software. Patents aside, there's nothing that machine learning research is going to give as an advantage nationally other than a couple years of warning that something is coming. On the other hand, if more software devs understood the basics of statistics and ML they'd be able to apply the tech into specialized domains.
The same thing is happening with ML that happened with web dev. Abstracting over the annoying stuff allows developers to attack the problem at the conceptual level. But you can't think conceptually without understanding thing from first principles. For example, I expect developers to understand the definition of accuracy and why maximizing it usually isn't useful.
It is 1965. Imagine if computer science was left for the top elites and never propagated to the masses. This kind of thinking is delusional, gate-keeping that only hurts.
ML/AI research can continue, but its application and implementation needs to be democratized. Everyone in CS should know ML - it is a tool just like learning algorithms.
Why would you condone not learning a tool that an engineer would use?
1. They are not teaching CS people, but a random 1% slice of the population.
2. Where did I not condone? I made an observation what I think leads to better outcomes. I said both is desirable but culturally only one thing seems to happen.
Before you angrily accuse me of "Delusional gatekeeping", maybe re-read (or maybe for the first time read) both the article and my comment.
Your confrontational tone does not help the discussion and is not in the spirit of hackernews to assume the best intentions.
I apologize for my tone but no one is accusing you. I am criticizing your thinking and your argument. Photography is democratized to the point where there is so much noise. Everyone has a camera with them at all times. If you want to be a professional photographer today, it is almost impossible barring journalism or wedding photography. I feel like ML researchers (again, not particularly you so don't take it personally) feel threatened by the mass adoption, marketing buzz and the whole ML/AI enchilada. Everyone is getting on it including the topic of this thread - i.e. the Finnish government.
This is honestly what I see when I read your parent comment. If you're offended by it, then may be you can balance your argument so it doesn't appear to be gate-keeping elitism. In fact, you've admitted it is controversial!
I think one could also imagine a world where there will be a cannibalization/consolidation on the lower end of the market because of the democratization (which I am not saying is bad or good because I am not sure about the effects ML has on people and democracies, but that is another discussion).
Rather, I observe that nobody today would hire someone to (in essence) build numpy functionality. Yet there are a lot of jobs in ML/AI currently which amount to building things that will be commoditized within years (and be used by everyone like numpy today). So in my opinion learning AI basics will not necessarily lead to more AI jobs or startups because these basic features are commoditized via big tech open source.
Second, on applications:
My research is actually in the applied domain and that's where my argument mostly stems from. Doing novel, valuable applications in AI requires concentrated resources and expertise. For example, if you think about self-driving cars as an extremely high-value application, they require highly specialized domain expertise in various domains and machine learning combined with massive compute resources.
Simple applications (e.g. applying basic machine learning tools to common workflows and processes) are prone to undergo commoditization precisely because they are obvious. So I do not think the gatekeeping argument holds because there is no way around obtaining deep and wide expertise to generate valuable application IP.
So for a country wanting to focus on applications, I still think there is a good argument that concentrating resources is a more successful approach.
>> Simple applications (e.g. applying basic machine learning tools to common workflows and processes) are prone to undergo commoditization precisely because they are obvious
Yes, Simple ML will be a commodity. Like Arduino, Ruby-on-Rails, WordPress, etc.
But in the right hands, having such tools years before the competition, could be a great thing, one that could lead to real competitive advantages.
For example, in Israel ~15% think about starting a business. That's not that far from the whole population.
So if you offered them the right education, and they played with the tools, maybe it would lead to new businesses.
And in the end, it's impossible to tell who will come with that great idea.
It makes sense... almost Power law distribution of talent. The wisest move is to hedge bets accordingly and accurately classify potential talent for said investment.
The worst move would be to do what the US is doing:
- nationalist xenophobia turning off expats from emigrating
- exacerbating massive inequality disparity/impoverishing it with people living on highway on-ramp embankment worse than S. American favelas (i.e., go to the Bay Area, drive around.. 'nuf said)
- ban the export of new technology with arms controls, killing industries and making a country radioactive to invest/do business in
- make it extraordinarily difficult for moderately wealthy entrepreneurs to work in a country for even a limited time because their passport isn't from one of a dozen or so favored countries
- ban everyone entirely, regardless of their background or achievements, if they happen to have been born in another set of countries
- double-down on fossil fuels, end clean energy tax credits
To me, the best things from Finland are Linus Torvalds, Jarkko Oikarinen and The Hydraulic Press Channel. There's probably others but the wikipedia page for famous Finns was too long.
So when it's some silicon valley tech company releasing something as part of their tech marketing strategy it's fine, but when it's from Europe it's "Propaganda"?
Well, it's not a secret that a lot of the content in HN is marketing content. Usually from companies who want to market themselves among the tech community.
I'm not complaining. I learn a lot, and most of the times everyone wins. But it's not the first time I find this US/Rest of the world double standard.
Maybe this was a knee jerk reaction
from me to that pre existing notion
Is AI settled enough that today's state-of-the-art is a stable foundation that can be reliably built upon... or are revolutionary changes just around the corner? Or, it is that the application of some generic AI, that can use any upcoming changes?
Perhaps we have a few more twenty year hype-winter cycles of NN before we know what we're doing...
> intention to "support democracy," ...
raise awareness about the opportunities and risks of AI ... where they want their government to invest.
> "That’s how society works — if enough people say they don’t like it, then we regulate it,”
Agree, democracy needs uninformed voters. But the above was just the origin.
Now the course seems more like encouraging people to see applications of AI in their regular work. Like pure vs applied maths. Agree: technology transfer/commercialization of AI will be the richest industry the world has ever seen.
[ Though it's a bit like applications of computers [funfact: many software ideas were initally called AI... like how subjects moved from philosophy to science when they got actual facts]. Although computers are an "industry", their applications are considered vertical, i.e. as part of the industry they are applied to. Same thing happened to physics. So AI applications won't be "an industry", but in every industry. ]
Another question is whether the application best starts from the AI end or the application end. You need both, but at this early stage, it's still requires an intertwined "integrated" collaboration.
I don't think we're yet at the point where you just add magic AI pixie dust to a problem.
But it can't hurt getting people introduced to it and thinking about it; and the democratic point remains true.
Can someone explain what is in this 'Elements of AI' course, since that doesn't seem to be described in the article? I'm just having a hard imagining what can be taught in a few months to e.g. the dentist in the article and still be 'practical'.
It's not a course where you learn how to implement algorithms, but a general introduction to the concepts and problems that typically are treated under the label 'AI'.
As usual, this is a course that completely ignores the first, oh, 60 or so years of artificial intelligence research which were dominated by logic-based AI, and places most of the focus on deep learning, as the only kind of machine learning that it talks about at any length.
Sometimes, I despair. It's like AI started in 2012 - and was wrapped up the same year. Deep neural nets, problem solved, AI is all done and dusted.
I've always admired Sweden for committing to robotics (it was back in the 90's I think, at a time when I was doing robotics work for Fanuc and GE). They went from nothing to becoming a major robotics player over decades.
In contrast, I think committing to AI is a mistake - the market is fundamentally different. To put it simply - there is lots and lots of work to be done in robotics, over decades, while, in contrast, there is a big burst of work to be done in AI, followed by relatively little work (compared to the scale of robotics.) I admire Finland, and I hope somebody out there figures out the business difference before they lose decades to opportunity cost.
Oh yeaaaaaah... Like modern "coding mania" at school... Guys IT world is as complex as a society: you can't train anyone to be a prime minister in such society before train in being an educated citizen.
Most Finnish child can type on a Window PC with some proprietary (cr)app. They can probably play games, watch porn sites, social networks etc and that's for some older people seems to be like magic so they think that "programming" is only a small subsequent step like you can study integrals just after derivatives. And that's the ideal way to put people far from computer disgusted by obscene software and absurd training.
Thanks for the link however the point is simple: people who write those norms have no idea what programming means and how it can be done in modern desktops.
If, and only of, we have modern Alto or LispM perhaps we can teach few "programming" concept to let most scholarized people able to script/design their own "home environment" on a desktop otherwise it's only a waste of time especially if is done as classic high schools in Italy or Sweden (starting with some Pascal dialect end try to offer C or JS after).
Finland doesn't do it like Italy or Sweden (let alone how they did it 20 years ago). Instead, kids are more likely to start with Scratch or similar: https://scratch.mit.edu
I'm curious how they can get enough teachers. I mean people who now IT enough not only to being able to actually "program" themselves for real a bit but also teach to children.
In my personal experience even at universities (Italy with very limited knowledge of Swedish unis) for IT-centric courses we have only very few really competent teachers that also are able to communicate their competence well, transmitting passion for the matter, drawing a big number of possible evolution paths, instill the desire to explore autonomously, solving personal problems etc...
Perhaps today's things have change but...
Oh, sooner or later we need to seriously teach IT like we seriously teach our motherlanguage because that's the actual nervous system of our society and we not only should but must know it enough to being able to master the part needed to our life and having an informed opinion on that topic to avoid disastrous trends like today's one, only I really can't imaging how to form enough people.
Even in casual events organized by LUG or few companies/universities I hear casual conversation from "supposed" technicians that make me shiver or from certain mailing lists...
In Finland, teaching is a respected profession with a Master's degree required and teachers are paid a proper salary and given autonomy in the classroom in return.
They don't have to be professional-grade developers - they are professionals in general pedagogy and mentoring.
I do not intend to judge the level of competence or respect of Finland's teacher's only I'm really curious how they were formed in the first place.
In general when something new appear (to stay, with success) we have a pioneering phase that start to "form and initially spread" the new "thing". This phase may be quick, however it have to be at least 5-8-10 years to be "spread and know enough". After we have a "consolidation and popularization phase" that normally last long, 10-15 years if it's quick, after it became spread and know enough to have enough "thought currents", it have developed a sort of "stable" philosophy etc. This is the stage we can start to form teachers, normally for tech schools and universities. Only after a decade or even a generation we can start to arrive at "initials schools" teachers.
Counting those years and see "IT age"... Well, we can say we are at start phase of teaching in schools, so it's really hard to have enough competent and "tested" teachers now...
Also about pedagogy and professionalism I remember a personal observation, limited to universities, before I have observed too little: teachers with professional background tend to know far better what they teach and normally are more capable to communicate it. Teachers coming directly from academia tend to know far less and are far less able not only to communicate but also to create interest and transmit passion.
Of course that's not a valid statistics nor I can generalize it for the entire world but... Well just as an example at first year of computer engineering degree course I have a course in chemistry, the first part was held by an teacher "definitively and proudly made in academia", it was a disaster not only for me. The second part by an actual chemistry doctor coming from the industry that start teaching to remain active instead of retire. For some results was still poor for other me included was really well. The same for economy course. The same for all subsequent courses and years. The third year in particular nearly no one digest mathematical physics (a sort of variant of rational mechanics adopted for a not-really-industrial engineering course), the second semester another teacher results completely changes. For essentially all.
Long story short: only few know how to transfer knowledge and I can call them "pedagogues" do not caring to much if they study that subject or they are naturally brought to teach. That's needed but not sufficient. Competence in the field is also needed and for non-academic discipline it's hard to have gained, metabolized and summarized enough to teach. Also passion for the matter is needed and that's again should be trained and formed in years of activities. Like a construction worker's that see if something is valid by instinct even before actually calculate it. Only people with such experience can properly teach. And while normally young schools are considered "easier" to teach is at that stage that we really form our knowledge and start to develop paradigms.
I should clarify that the context of my comments was grades 1-9 where the content is fairly simple and the target level is nowhere near professional skills. I'm not saying the teaching is perfect (it's always far from that) but the results in other topics are good and I don't see why they wouldn't be good in programming as well. If you're saying it's too early, I say the earlier we get the virtuous cycle going the better (the previous generation of teachers will teach the next generation to be better).
That is ideal direction - educating the non-IT people with domain knowledge what tools the IT people can provide then both can work on solving real world issues with AI. I predict lots of serious start-ups comming out of this effort.
Machine Learning is Alchemy https://youtu.be/x7psGHgatGM . We need to understand optimization to advance this field not train thousands of monkeys to press play and hope for a nice result.
Also, why did they not decide to teach 1% of the population basic coding. Now that could transform a society.
Basic programming skills have been a part of the Finnish primary school curriculum since 2016. By sixth grade students are expected to "create working programs in a graphical programming environment". Junior high includes "basic algorithmic thinking", more programming and embedded programming.
Likely in practice means trying out Scratch in primary school and creating robots in junior high.
Most machine learning models are nothing but souped up regression that swaps out standard polynomials for matrix-valued polynomials with some x'es replaced by non-linear activation functions like tanh(x)
The magic is in the toolkits that
allow one to build these complex expressions through net chain composition, and provide symbolic differentiation to easily calculate gradient on hugely complex net chain graphs.
The avg person isn't going to benefit from ML unless you also teach them the old-hat statistical fitting methods that worked well enough before ML came on the scene.
Its not worth training a bunch of folk who cant go deep into the heavy mathematics, statistics, and epistomology of what ML really is in terms of manifolds.
Just show them what ML can and cannot do, if its not possible to give them the comprehensive education. And besides, most people, once they find out ML is a shit ton of math and as profound and based as the theory of functions, shakes their head and trods off. They wanted magic soup and you gave them the biggest questions about existence..about patterns..forms..things..about consciousness, intelligence, and life, about meaning.
Don't expect the avg lowbrow person to want to explore ML once they get hit with the reality of its implications, rigor, and depth.
> Most machine learning models are nothing but souped up regression that swaps out standard polynomials for matrix-valued polynomials with some x'es replaced by non-linear activation functions like tanh(x)
Well, not quite. The non-linear activation feature is found already in "generalized linear models" (as a 'link function'). The typical "deep learning" model is a hierarchical/multi-level version of the same - and just like any other hierarchical/multi-level model, the point of that is to account in a parsimonious way for increasingly-complicated interactions among the regressors/features. But I think your broader point is right; there is a lot of hype in ML that derives purely from a lack of familiarity with basic statistical principles.
I remember signing up for this a while back, but never found the time to take it. I assume it's the same course because I remember the website looking so "trendy"
Well roundedness (or teaching basic elements to many) will not pay off as much, it would likely yield better results for the country to invest heavily at the top.
Observation based on success of machine learning research in highly concentrated organisations (certain universities in the US, DeepMind, etc) versus countries with scattered expertise and resources (e.g. Germany).
If you can afford to, do both, but observationally, this does not seem to happen. Not for a lack of resources but rather the mindset and a society's perspective.