Among arxiv publications there are 217 results that contain "large language model" in the full text and "from scratch" in the title or abstract.
There are 2873 results that contain "large language model" in the full text and use "pretrained" in the title or abstract. A 10x difference in publication count does make one feel more common than the other?
I'd need to get into more involved queries to break down the semantic categories of those papers.
To add to this, companies at Google-scale tend to have a huge variety of ML related jobs, ranging from low level things like optimising libraries for different hardware, to the more general research positions where people are working on their own pet projects. Plus everything in between - data management and curation for training models that get used in production, people who try and figure out how to productionise cutting edge research, people who build the infrastructure that other ML engineers use (and here again, everything from hardware/server people, cloud, site reliability, tooling) and the list goes on.
I know of at least one person who got an ML job at Google, but didn't apply specifically for it. They had a very strong ML background and applied for a generic software engineering and got team matched. That seems like a reasonable way to go if you don't want to go through a research interview loop.
I;'d like to echo this. I learned a long time ago that I don't want to be a "machine learning engineer"- I have no interest in designing new networks, feature selection, or training as a daily job. I know how to do all those things but it's not somethign I pursued at Google. Instead, I found jobs where I could work with those people (often the ones doing the real state of the art research at scale) using my experience, in ML, data engineering, pipelines, and HPC.
There is nothing quite like having a world-class researcher ask you to figure out why their model is exploding, and tracking down the crazy things that happen on TPUs when their math isn't absolutely perfect, then helping them fix it, and see them publish their results (or put them in prod). Or knowing enough software and hardware to debug a tensorflow TPU problem with an oscilloscope connected to the voltage regulator in a hardware lab.
Personally, i gained these skills over a long period starting in the mid-90s (working on machiine learning, and then later HPC for biology, and ultimately back to machine learning). But I am a slow learner. probably the shortest path is to get accepted to a major university and do really well in your ML and CS classes, then parlay that into a job in a FAAMG, then figure out what you want to do with all your skillz.
I got a unicorn senior RS offer without a PhD from a company that had mostly former FAANG top brass after interviewing without knowing it was for RS, I thought it was DS/ML. I declined because of "culture fit". Everyone has a PhD, they assumed I had one, I don't even have a bachelors. We still hang out and laugh about it.
I had been working in Attitude Determination and Control and Optical Systems Engineering for seven years before that interview and I just like, knew the stuff from the job. I've been back on pure-SWE roles for four years already and I don't think I could do it now. I have the intuition but I couldn't white board proofs for tree based algos and manipulate integrals like I did on that interview for sure.
It’s not impossible to happen that fast. A SAFE takes almost no lawyer time to prepare, and sometimes investors think there’s good reason to move that fast. But the fact that it does happen sometimes doesn’t mean that’s “normal” in any way. Even great people with really solid ideas usually take weeks to really put together and close a deal like that.
a “3 day raise” can mean many things, imagine something like this: 2-3 strategic angel investors already in place; prior to “starting fundraising” is a 1 month period of pitch discovery during which your angels are introducing you to investors but you are “not fundraising yet”; during this period investors start asking to invest but you are “not fundraising yet”; once you hit like $500k in interest, you email all the investors you’re already talking to and say “i’m fundraising now and already have $500k in interest for a 1.5M round” and one seed fund takes the remaining $1M and you’re done (the three days is for diligence). or 4 famous angels follow with $250k checks and you’re done. it’s an orchestrated process
You can export yes, not entirely sure if you can export to Anki.
You can export in either txt or json.
And you can export 3 ways.
1. All highlights & notes from an article (article = epub / web article)
2. All highlights & notes from a particular topic/tag
3. All highlights & notes from your entire account organized by article.
I think that's how it works. Wrote the export code a while ago so the details escape me at the moment.
You mean to say someone like NSO Group, not Cellebrite. But you should know that it's possible driving up the price of bugs helps companies like NSO, rather than hurting them. They're middlemen, taking a cut of the value of transactions between exploit developers and downstream customers. Those downstream customers, for shops like NSO, are overwhelmingly government agencies that aren't especially price-sensitive to the cost of individual bugs.
I assume NSO group operates in their own best interest. If them buying a bug and reselling it hurts them, then I think they won't do it.
Although I guess one reason they might buy a bug that would lead to financial harm is to prevent a competitor from getting it, which might be an even worse financial harm.
Shameless plug: I've made a deep learning course oriented with practical content on a wide variety of computer vision topics --> https://arthurdouillard.com/deepcourse/
I'll look at this properly when I'm not on mobile, but I noticed some minor issues. A typo that seems to be repeated a few times: "space-repetition" should be "spaced-repetition". There are also several unnecessary capitals in your opening sentence.
