> What makes LLMs difficult is getting access to all the hardware to train them, getting the data, and being able to preprocess that data.
Yes, that's my opinion too. GAOs (Grassroots AI Organisations) are constrained by access to data and the hardware needed to process the data and train the model on it. I look forward to a future where GAOs will crowdsource their computations in the same way many science labs borrow computing power from people around the world.
This is hard because you need high bandwidth between the GPUs in your cluster, bandwidth far higher than broadband could provide. I'm not even sure whether the time spend synchronizing between far-away machines would offset the increase in computational power.
Yes, that's my opinion too. GAOs (Grassroots AI Organisations) are constrained by access to data and the hardware needed to process the data and train the model on it. I look forward to a future where GAOs will crowdsource their computations in the same way many science labs borrow computing power from people around the world.