Cavemen wander around, they fall over a pig, they point to pig and say "pig". Other cavemen observe. Later, when they want a pig, they say "pig". No one here knows anything about pigs other than that there is something in the world which causes people to say "pig" and each caveman is able to locate that thing after awhile.

The vast majority of language is nothing more than this: words point outside themselves to the world, this pointing is grown in us through acquaintance with the world.

Now, in general, the cause of my saying "pig" is not me falling over one. Suppose I say, to a friend, "I've always thought pigs were cute, until i saw a big one!"

So here, "I" points at both me as a body, but also plausibly at my model of myself (etc.), "always" modifies "thought" ... so "I've always thought" ends up being a statement about how my own models of my self over time have changed.. and so on for "pigs" and the like.

We do not know that this is what our words mean. We have no idea that what we're referring to when I say "I've always thought" -- the nature of the world that our words refers to requires, in general, science to explain. Words are, at first, just a familiar way of throwing darts at a target which we can see, but not describe nor explain.

It is this process which is entirely absent in an LLM. An LLM isnt throwing a dart at anything, it isnt even speaking. It's replaying historical darts matches between people.

And this is just to consider reference. There are other causes of our using words much more complex than our trying to refer to things, likewise, these are absent from the LLM.