I agree with the majority of your response but wanted to comment on the highlighted phrases above. While more sophisticated LLMs can produce responses that appear to be semantically informed, there is absolutely nothing in how the ‘algorithm’ of how it works that would allow it to actually interpret semantic content, produce an abstract model, and manipulate that model to ‘understand’ how the text relates to the real world. To the extent that LLMs can produce semantically-correct responses it is because there is logic that is both explicitly and implicitly built into the use of language; that is to say, although there is a nearly infinite number of ways that you can assemble words into grammatically correct sentences and link the sentences together into a flow of discussion referencing a subject, only a very small subset of these will actually make any sense, and the rest of them will read like Lewis Carroll’s Jabberwocky.
Because we normally only speak and write in ways that have actual semantic content, the data sets that LLMs are trained on reflect the statistically appropriate usage consistent with semantically-meaningful collections of words, and as a result, it has the emergent capability to mostly produce strings of tokens that read like thought-out concepts. But there is zero reason to believe that the system has any kind of internal conception of the world beyond manipulating word-tokens in a way consistent with the training data, despite enthusiasts and even some experts claiming that they can detect a ‘spark of consciousness’ from those responses. There is certainly nothing going on with these prompt-and-response systems that in any way represents cognition in a way that a neuroscientist would recognize; no ongoing mental processes, experience-driven creating and refinement of abstract concepts from physical and sensory interactions with the real world; no permanent correction of conceptual errors and misapprehensions. They are just manipulating tokens using the computational approach of heuristically adjusted weighting of connections in an artificial ‘neural network’ during its training phase and using those to to produce grammatically correct content which maps to how collections of words are statistically used in their training data sets. This gives the appearance of “semantic understanding”, at least for simple concepts, but that it is so trivially easy to ‘fool’ or confuse an LLM shows that it really doesn’t have any kind of introspection or comprehension of real world interactions.
Stranger