> There's actually a growing body of evidence that shows the emergent ability of LLMs to reason (the so-called "chain of thought" ability) arises only when LLMs are trained on huge amounts of code, not just natural language. Natural language training data provides the ability to sound human, but it is the programming language training data that provides LLMs with the ability to be logical.

Good on Sourcegraph to contribute back to the open source community. LLMs rely more on open source code than meets the eye.