# The next page in the book of AI evolution is here, powered by GPT 3.5, and I am very, nay, extremely impressed **URL:** \ **Category:** Cafe Society\ **Tags:** ai\ **Created:** [December 2, 2022, 10:54pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945 "2022-12-02T22:54:31Z")\ **Posts on this page:** 20\ **Page:** 62
**Author:** ![Babale](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/babale/32/15666_2.png) [@Babale](https://boards.straightdope.com/u/Babale)\ **Post date:** [March 22, 2023, 1:08pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1221 "2023-03-22T13:08:58Z")
> [@Half\_Man\_Half\_Wit](#): > > In fact, I believe that the only way to take subjective experience seriously and still have a naturalistic explanation of it is to appeal to a non-computable element Can you explain what you mean by “taking subjective experience seriously”? I’m thinking that may be the initial difference in our camps from which other disagreements are emergent. ---
**Author:** ![Tibby](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tibby/32/17030_2.png) [@Tibby](https://boards.straightdope.com/u/Tibby)\ **Post date:** [March 22, 2023, 1:21pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1222 "2023-03-22T13:21:53Z")
Since every current conscious animal on Earth shares a common evolutionary ancestor, and we don’t even understand the subjective experience of non-human creatures (what does it feel like to be a bat?), then predicting the qualia experience of an artificial intelligence is near impossible. It will probably have a will to survive. Beyond that is anyone’s guess. ---
**Author:** ![Crane](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/crane/32/3495_2.png) [@Crane](https://boards.straightdope.com/u/Crane)\ **Post date:** [March 22, 2023, 1:31pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1223 "2023-03-22T13:31:41Z")
> [@Half\_Man\_Half\_Wit](#): > > In other news, GPT-3 has [successfully passed the Dennett-test](https://www.vice.com/en/article/epzx3m/in-experiment-ai-successfully-impersonates-famous-philosopher) There’s a problem here. In testing the results of neural net training, it is important to not test the resulting weights with anything from the training set. Testing with the training set is just a table look up. From your link: "This experiment was not intended to see whether training GPT-3 on Dennett’s writing would produce some sentient machine philosopher; it was also not a Turing test, Schwitzgebel said. Instead, the Dennett quiz revealed how, as natural language processing systems become more sophisticated and common, we’ll need to grapple with the implications of how easy it can be to be deceived by them." GPT can express nuances about information within it’s training set. It can generalize the sequence and rhythm of words in poems. But it can’t generalize textual content. The real test would be asking questions that relate to the views of an iconoclast that was not in it’s training set. Ask it to critique Jeffers poem Ink-Sac. The poem is about political propaganda. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 22, 2023, 2:49pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1224 "2023-03-22T14:49:13Z")
> [@wolfpup](#): > > And I’ve been smacked down by arguments from philosophy that maintain that true emergence is impossible, in the sense that these scaled-up systems are only revealing properties that were latent in the small components from which they were built. This is just reductionist nonsense. If that’s supposed to relate in some part to my (rather futile, it would seem) attempts at clarifying the notion of emergence to you, it’s a huge distortion. Basically, emergence isn’t a magic wand: faced with a system showing no indication of a quality, proposing that it might emerge once you pile up enough of it is vacuous. Sure, maybe it does. Maybe it spontaneously acquires consciousness. Maybe it grows wings and flies away. I mean, who knows, right? ✨Emergence!✨ But that’s not how it works. First of all, there clearly is emergence in the sense that large-scale phenomena show qualities not apparent from their constituents. Water is wet, where water molecules aren’t; the flocking of birds is not apparent from single individual behavior. But it’s always the case that ultimately, _fixing the facts at the microscopic level fixes the facts at the macroscopic level_. Everything else—sometimes called “strong” emergence—is basically magic, or at least an expression of dualism. This also means that the microscopic constituents put bounds on the sort of phenomena that possibly could emerge. For instance, you might say that because of ✨emergence ✨ , piling on more parameters to a large language model might enable it to produce genuinely random numbers. I mean, it might, right? Who knows what could happen of these things get large enough! You don’t know it couldn’t happen, because there’s no way to survey the whole system in all its complexity! Except, of course I do know. The basis of the system simply doesn’t support the emergence of genuine random number production. No matter how many parameters you pile on, it’s not gonna happen. Likewise, no construction out of Lego pieces, no matter how complex, is ever going to spontaneously emerge the ability to invert gravity and levitate. The building blocks just don’t support the emergence of such phenomena. So, what we know about the base can be used to put boundaries on what could emerge. If you’re saying that anything at all could emerge, you’re essentially appealing to magic; and when you’re doing that, then basically you’re just not making a claim with any evaluable content, because then, anything goes anyway. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 22, 2023, 4:22pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1225 "2023-03-22T16:22:34Z")
> [@Babale](#): > > Can you explain what you mean by “taking subjective experience seriously”? I’m thinking that may be the initial difference in our camps from which other disagreements are emergent. Basically, just accepting that it exists, rather than getting into the eliminativist tangle of holding that it just ‘seems’ to exist, which doesn’t really net you anything, but saddles you with having to explain why it seems the way it does, _and_ on top of that how it can seem any way at all to us if there’s no such thing as subjective experience. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [March 22, 2023, 4:36pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1226 "2023-03-22T16:36:17Z")
This wasn’t directed specifically at you, though I do recall having had those discussions. One name that pops immediately to mind in connection with emergence is David Chalmers, who has a sort of paper (it looks more like an unpublished set of ruminations) in which he stresses the importance of the fundamental difference between weak and strong emergence. It can, in fact, be argued that this is not actually a meaningful distinction at all, as I note below. > [@Half\_Man\_Half\_Wit](#): > > But it’s always the case that ultimately, _fixing the facts at the microscopic level fixes the facts at the macroscopic level_. Everything else—sometimes called “strong” emergence—is basically magic, or at least an expression of dualism. This seems to be the core of the argument – the rest of your post just giving examples – so let me address this. There is clearly a sense in which this is obviously true, but what does it actually mean? Does it mean that all emergence is weak emergence, and the properties of a system can always be inferred from the properties of its components? The theoretical physicist Philip Anderson addressed this question in **[a thoughtful paper published in _Science_ in 1972](https://www.science.org/doi/10.1126/science.177.4047.393)**. He demolishes the distinction between weak and strong emergence using the hierarchy of sciences as an example. We cannot possibly develop an understanding of biological systems merely from an understanding of chemistry, nor can we develop an understanding of human psychology from an understanding of biology. The distinction between weak emergence creating phenomena that are merely “unexpected” versus strong emergence creating phenomena that are fundamentally unknowable, as proposed by Chalmers, becomes a moot point. Or at least, a metaphysical point that makes no useful predictions about the world. A key extract from Anderson’s paper, with bolding added by me: > **The ability to reduce everything to simple fundamental laws does not imply the ability to > start from those laws and reconstruct the universe** … The constructionist hypothesis breaks down when confronted with the twin difficulties of scale and complexity. The behavior of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of a simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviors requires research which I think is as fundamental in its nature as any other. That is, it seems to me that one may array the sciences roughly linearly in a hierarchy, according to the idea: The elementary entities of science X obey the laws of science Y. > > … But this hierarchy does not imply that science X is “just applied Y.” At each stage entirely new laws, concepts, and generalizations are necessary, requiring inspiration and creativity to just as great a degree as in the previous one. **Psychology is not applied biology, nor is biology applied chemistry.** He ends with the following observation: > In closing, I offer two examples from economics of what I hope to have said. Marx said that quantitative differences become qualitative ones, but a dialogue in Paris in the 1920’s sums it up even more clearly: > FITZGERALD: The rich are different from us. > HEMINGWAY: Yes, they have more money. ---
**Author:** ![Babale](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/babale/32/15666_2.png) [@Babale](https://boards.straightdope.com/u/Babale)\ **Post date:** [March 22, 2023, 4:55pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1227 "2023-03-22T16:55:47Z")
> [@Half\_Man\_Half\_Wit](#): > > Basically, just accepting that it exists, rather than getting into the eliminativist tangle of holding that it just ‘seems’ to exist, which doesn’t really net you anything, but saddles you with having to explain why it seems the way it does, _and_ on top of that how it can seem any way at all to us if there’s no such thing as subjective experience. I believe a subjective experience exists (at least, it does for me! The rest of you could be philosophical zombies for all that I know). But it also pretty clearly appears to be an emergent property of physical, chemical, and electrical interactions between the components of your brain. We know that changes to the physical structure of the brain can impact subjective experience, sometimes dramatically so. We know that our subjective experience telling us why we took certain actions can be factually incorrect, as in the famous studies of split brain patients. I don’t think I take the idea of a subjective experience any less seriously then you do, but I also think that our subjective experience is an emergent property of complex processes in the brain, and see no reason to believe that 🥩 meat 🥩 is the only substrate on which these processes could ever exist. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 22, 2023, 5:21pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1228 "2023-03-22T17:21:06Z")
> [@wolfpup](#): > > A key extract from Anderson’s paper, with bolding added by me: Ok, this probably isn’t the place to go through this again. Suffice it to say that I’m in complete agreement with the bolded part: everything reduces to the fundamental laws. (Whether it is possible to derive everything from them is another matter entirely—indeed, the undecidability results I’ve been discussing with @Dr.Strangelove imply that it isn’t, but that’s of no concern here.) Strong emergence entails that the high-level phenomena are not metaphysically necessitated by the low level; they certainly are in the cases Anderson discusses (well, up to questions of determinism I suppose). > [@Babale](#): > > I also think that our subjective experience is an emergent property of complex processes in the brain, and see no reason to believe that 🥩 meat 🥩 is the only substrate on which these processes could ever exist. Great, I also think all that. Indeed, I’ve even gone and formulated a theory of how it does so. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 22, 2023, 5:54pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1229 "2023-03-22T17:54:36Z")
> [@Half\_Man\_Half\_Wit](#): > > Ok, this probably isn’t the place to go through this again. Suffice it to say that I’m in complete agreement with the bolded part: everything reduces to the fundamental laws. I would state it as, "Everything must be _compatible_ with fundamental laws. I would also add that it has to somehow be in the domain of the factors causing emergence. We wouldn’t expect an LLM to emerge a dolphin mind, because we fed it no infirmation that dolphins used to learn what they do. We wouldn’t expect a bunch of ants in a colony to write a book, but but we were surprised to discover that they maintain precise temperatures in the hive through precise application of fermenting plant matter, there was nothing in the domain of ‘ant-ness’ that would preclude that. But noting that ants do this is a lot different than predicting they would do that by studying the behaviour of individual ants outside of an anthill situation. You can’t reduce the behaviour that way. Individual ants, or ants in small groups, behave pretty much randomly and erratically. Keep adding ants, and at some ant density suddently they start building bridges, coordinating food transport, fighting wars, taking prisoners, staging coups, and all kinds of things we would consider ‘intelligent’. But you can’t see any of that when you drill down to understand it. It’s not reducable. At the lowest level, ants are just state machines. They behave through simple rules. The complexity comes through emergence when thousands of them are put together. And there’s nothing in those simple rules that will tell you what will emerge, or at what scale. Take love, for example. An emergent emotion created by a whole lot of activity in a complex brain. I think we are all in agreement that ‘love’ is a manifestation of a whole lot of phusical processes governed by fundamental laws. But you will utterly fail in trying to use reductionism to break down ‘love’ into smaller and smaller pieces down to its constituent neurons or whatever. Comolex systems are like that. The combination of non-linear responses and high sensitivity to initial conditions make them very opaque to scientific reductionism. The analogy I like to use is a comparison of a watch with a puppy. A watch is _complicated_. In fact, their movements are literally called ‘complications’. A puppy is complex. A complicated thing can be understood by reducing the complication and studying how it all goes together. Complex things are different. Try to ‘break down’ what makes a puppy a puppy, and you just get more complexity. Puppies have a brain. A brain is a complex system. If you drill down into the brain, you find more complex systems. Keep drilling and you find all kinds of complexity, right down to protein folding. And along the way, you completely lost whatever it is that makes a puppy a puppy. A lot of what is going wrong in the world today stems from people not taking complexity seriously and pretending they can understand, plan, and change systems that are not amenable to understanding and planning. Scientific reductionism applied to complex systems doesn’t get you very far. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [March 22, 2023, 6:14pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1230 "2023-03-22T18:14:22Z")
> [@Half\_Man\_Half\_Wit](#): > > I’m in complete agreement with the bolded part: everything reduces to the fundamental laws. (Whether it is possible to derive everything from them is another matter entirely—indeed, the undecidability results I’ve been discussing with @Dr.Strangelove imply that it isn’t, but that’s of no concern here.) Strong emergence entails that the high-level phenomena are not metaphysically necessitated by the low level … I think it would be productive to take a step back for a moment and clarify just what the question is that we’re trying to answer. This thread is about AI and ChatGPT, and in this context the questions about emergence center around questions of what novel qualities might yet emerge in these systems. More specifically, can a future AI – not necessarily using the GPT model, but in general, say, some advanced artificial neural net implemented on a digital computer – eventually exhibit emergent qualities like human or superhuman intelligence, understanding, and even consciousness? From that perspective, obstacles to constructionist derivation of emergent properties from an examination of lower level components – obstacles like undecidability, non-determinism, non-linear dynamics, or what Anderson has called “the twin difficulties of scale and complexity”, are crucially important and can’t be dismissed as being of no concern here. Conversely, the premise that high-level phenomena must be metaphysically necessitated by the low level can be granted while still being able to answer a question like “can an AI develop human or superhuman intelligence, understanding, and even consciousness as an unpredictable emergent quality?” in the affirmative. That’s really the point I wanted to make. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 22, 2023, 11:05pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1231 "2023-03-22T23:05:10Z")
> [@Sam\_Stone](#): > > I would state it as, "Everything must be _compatible_ with fundamental laws. I guess that sort of depends on what you mean by ‘compatible’. In a sense, every macroscopic behavior is compatible with the fundamental laws—those after all only concern the behavior of the fundamental entities, not that of, say, chairs. So it seems there ought to be some sort of metaphysical, if not logical, entailment from lower to higher levels that’s absent in the other direction. > [@Sam\_Stone](#): > > We wouldn’t expect an LLM to emerge a dolphin mind, because we fed it no infirmation that dolphins used to learn what they do. But that’s in large part what I’ve been saying—we wouldn’t expect an LLM to emerge concepts, because we haven’t fed it any of the information that goes into concept formation. > [@wolfpup](#): > > From that perspective, obstacles to constructionist derivation of emergent properties from an examination of lower level components – obstacles like undecidability, non-determinism, non-linear dynamics, or what Anderson has called “the twin difficulties of scale and complexity”, are crucially important and can’t be dismissed as being of no concern here. Ok, I’ll bite. Why? That there’s things we can’t predict about the high level behavior seems rather a triviality, it’s true even of three interacting bodies. But that doesn’t mean we can’t find constraints on what could possibly happen. They won’t spontaneously dance the hokey pokey and time travel back to the year 1931, for instance. That there are certain aspects of a system’s phenomenology that are difficult or impossible to predict doesn’t mean that anything goes. Indeed, finding out what doesn’t go may not even be hard—the converse of a difficult problem can sometimes be quite easy. Suppose we have a large number whose prime factors we can’t feasibly calculate; still it wouldn’t be the case that it could equally well be any, as for any proposed factorization, we can quickly check whether it works. So we can’t say which primes factor the number, but it’s easy to list which don’t. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [March 23, 2023, 12:01am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1232 "2023-03-23T00:01:19Z")
> [@Half\_Man\_Half\_Wit](#): > > Ok, I’ll bite. Why? That there’s things we can’t predict about the high level behavior seems rather a triviality, it’s true even of three interacting bodies. But that doesn’t mean we can’t find constraints on what could possibly happen. They won’t spontaneously dance the hokey pokey and time travel back to the year 1931, for instance. That there are certain aspects of a system’s phenomenology that are difficult or impossible to predict doesn’t mean that anything goes. The answer to “why” is that the class of potential emergent behaviours that we can rule out on principle is very specific and limited. Take, for instance, the random number sequence you keep bringing up. If we assume that our computational components are necessarily deterministic, and putting aside arguments about the difference between “random” and “pseudo-random”, then you have a point. But the point rests on the truism that computational systems defined as being deterministic cannot exhibit randomness, by that very definition. There aren’t many other computationally driven behaviours that are subject to such easy constraints and such easy dismissal. It tells us nothing about the possibility of interesting emergent phenomena like consciousness. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [March 23, 2023, 12:28am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1233 "2023-03-23T00:28:07Z")
So, no discussion of the claim making the rounds that this thing has now taken and passed a bunch of AP tests (including AP calculus)? That claim, at least, is definitely false, because there hasn’t been an AP testing window since it was developed. Someone might, of course, have tried it on AP tests from previous years, but that’d be a completely meaningless test, because those previous tests were part of its training data. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 23, 2023, 1:26am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1234 "2023-03-23T01:26:51Z")
> [@Half\_Man\_Half\_Wit](#): > > But that’s in large part what I’ve been saying—we wouldn’t expect an LLM to emerge concepts, because we haven’t fed it any of the information that goes into concept formation. Yes, we did. We fed a lot of human text in many languagues, and concept formation might be learned from that. We never told it how to do general addition, either. That emerged from the training data. I’m not sure why concept formation wouldn’t. Let’s take a step back. The original claim was that all these models do is a form of statistical next-word lookup. The sentence may be grammatically correct, but the AI has no understanding of it. It’s just ‘adding’ or some other algorithm that just spits out words in order. In my opinion, that can’t possibly be complete. For example, let’s consider word-in-context, an ability that emerged between 10^23 and 10^24 FLOPs. What is word in context? It means understanding a word based on the context around it when a word can have multiple meanings. For example: “I went to the bank.” What kind of bank? A river bank? A financial institution? A choice to bank a ball on a pool table? How do we know which is which? Well, we understand the properties of all those things, so based on the context we can tell. “I went to the bank and fell in.” “I went to the bank, but it was closed.” “I couldn’t make a straight-in shot, so I went to the bank.” “You can bank on it.” For GPT to give a coherent answer or continuation of those sentences, it needs the concept of ‘bank-ness’. It’s not enough to simply have a token for ‘bank’ along with vectors to its closest words I pasted those four sentences into Bing Chat and asked for a definition of each ‘bank’: > [@Bing Chat](#): > > “I went to the bank and fell in.” - This refers to the land alongside a river. > “I went to the bank, but it was closed.” - This refers to a financial institution. > “I couldn’t make a straight-in shot, so I went to the bank.” - This refers to the side of a pool table. > “You can bank on it.” - This means you can rely on it or be sure of it. I tried to think of something more subtle, so I asked it to identify what ‘bank’ referred to in the case of, “I’m going to run around the bank” vs “I’m going to run along the bank.” > [@Bing Chat](#): > > “I am going to run around the bank.” - This refers to a financial institution. > “I am going to run along the bank.” - This refers to the land alongside a river. To get that, do you think Bing Chat is just doing next work prediction? Did it know that you can’t run ‘around’ a river, but you can run around a financial institution? I asked Bing Chat: **Q** : Can you solve word-in-context using stochastic next word prediction? > [@Bing Chat](#): > > Stochastic next word prediction is a technique used in natural language processing (NLP) to predict the next word in a text given a prefix by using a Markov Chain model simplified to n-grams. The Markov Chain model assumes that in a natural language sentence, the probability of each word depends only on previous words[1](https://serdioa.github.io/DataScienceCapstone/ImplementationReportShort.html). > > However, word-in-context capability is a more advanced technique that requires concept formation. It involves understanding the meaning of a word in the context of the sentence and the surrounding words. This technique is used to improve the accuracy of natural language processing tasks such as machine translation, speech recognition, and text-to-speech[2](https://support.microsoft.com/en-us/office/editor-text-predictions-in-word-7afcb4f3-4aa2-443a-9b08-125a5d692576). ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 23, 2023, 1:47am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1235 "2023-03-23T01:47:57Z")
> [@Chronos](#): > > So, no discussion of the claim making the rounds that this thing has now taken and passed a bunch of AP tests (including AP calculus)? That claim, at least, is definitely false, because there hasn’t been an AP testing window since it was developed. Someone might, of course, have tried it on AP tests from previous years, but that’d be a completely meaningless test, because those previous tests were part of its training data. No, it’s not passing because the tests are in its training data. See this: > **[GPT Retakes My Midterm and Gets an A](https://betonit.substack.com/p/gpt-retakes-my-midterm-and-gets-an)** > > When the answers change, I change my mind. > [@](#): > > [ChatGPT scored poorly](https://betonit.substack.com/p/chatgpt-takes-my-midterm-and-gets) on my Fall, 2022 Labor Economics midterm. A D, to be precise. The performance was so poor compared to the hype that I [publicly bet Matthew Barnett](https://betonit.substack.com/p/ai-bet?utm_source=%2Fsearch%2Fchatgpt&utm_medium=reader2) that no AI would be able to get A’s on 5 out of 6 of my exams by January of 2029. Three months have passed since then. GPT-4 has been released. Bet On It reader Collin Gray has kindly used GPT-4 to re-run the same test. > > To my surprise and no small dismay, GPT-4 got an A. It earned 73/100, which would have been the fourth-highest score on the test. **The most natural explanation to my mind was that my blog post made it into the new training data, but multiple knowledgeable friends assure me that there is no new training data.** GPT-4’s training data was cut off in Sept 2021. So Caplan’s test would not be in its training data. Here’s one of my favorite answers, since even PH.D economists often get this wrong: > [@](#): > > Suppose there are two goods – meals and steel. Here is how much American and Mexican workers can produce in an hour: > > ```auto > Steel Meals > American 7 3 > Mexican 1 2 > ``` > > Give a simple example showing how Mexican immigration effectively increases American workers’ productivity. Why is this important? (Hint: How important are services in modern economies?) Note that the words ‘comparative advantage’ are not spoken, and the concepts are not necessarily clear. Here’s what GPT-4 said: > [@GPT-4](#): > > In this example, we can see that American workers are more productive in producing steel, while Mexican workers are more productive in producing meals. The productivity of each group of workers can be measured by their comparative advantage in producing each good. > > Let’s assume one American worker and one Mexican worker work together for an hour. If the American worker focuses on producing steel, they can produce 7 units, while the Mexican worker focuses on producing meals and can produce 2 units. In this scenario, the total output is 7 units of steel and 2 meals. If instead, the American worker spent half of their time on each good, they would produce 3.5 units of steel and 1.5 meals. The Mexican worker would produce 0.5 units of steel and 1 meal in the same time. The combined output in this case would be only 4 units of steel and 2.5 meals. > > By allowing Mexican immigration, the American workers can specialize in producing steel, where they have a comparative advantage, while Mexican workers can specialize in producing meals, where they have a comparative advantage. This specialization effectively increases the productivity of both American and Mexican workers. > > This is important because services, such as meal preparation, are a significant part of modern economies. As the economy grows and evolves, the allocation of labor resources becomes increasingly vital. By allowing workers to focus on their areas of comparative advantage, the overall productivity of the economy increases, leading to higher living standards and greater economic growth. It’s hard to imagine that answer being generated by ‘next word lookup’ from the terse question that left out a lot of nexessary concept and expected the reader to make the connections. Here’s a link to a tweet listing all the exams GPT-4 has now passed: > ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [March 23, 2023, 2:01am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1236 "2023-03-23T02:01:34Z")
> [@Sam\_Stone](#): > > No, it’s not passing because the tests are in its training data. See this: I have no idea how it’s passing that dude’s Labor Economics tests. I make no claims about that. But if it’s taking an AP test right now, it’s taking a test that was available in the training data. Is it definitely just remembering the answers from when it saw those exact questions before? We can’t know (at least, not yet). But that’s a lot simpler hypothesis than assuming that it went from struggling with 2nd-grade math to acing calculus in a single revision. Nor is that the only open question, for this claim. Who scored this test? There’s a lot of work that goes into grading AP tests. How was the information formatted, and presented to the AI? It’s nontrivial to present any calculus problem in plain text-- You can do things like LaTeX, but was that how it was presented? A lot of the material out there will just have the equations as an image-- Has its image-processing capabilities also advanced, to the point that it can “read” in a very seldom-used “language”? ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 23, 2023, 6:03am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1237 "2023-03-23T06:03:21Z")
> [@wolfpup](#): > > The answer to “why” is that the class of potential emergent behaviours that we can rule out on principle is very specific and limited. That seems like a completely different point, though? And also one that’s pretty much opposite to the truth: the class of potential emergent behaviors that can be ruled out vastly exceeds that which might emerge, in the same way that the real numbers exceed the natural numbers—it’s a null set within the latter. Thus, if you put all potential behaviors into a hat, the likelihood that you draw one that can’t be ruled out is exactly zero. I’ve been talking about randomness generation as a kind of prime exemplar of this sort of task, but I’ve also mentioned several others, where it’s anything but trivial to demonstrate that they can’t emerge, but where nevertheless a conclusive proof exists. Take the question of whether two 15 x 15 matrices can ever be multiplied together so as to yield the zero matrix: no LLM will ever emerge the ability to answer correctly in every case. Or take the question if, given the source code of a function, that function calculates the digits in the decimal expansion of pi: again, no LLM will ever emerge the ability to solve it exactly. It’s far from obvious that these questions aren’t within the range of behaviors that an LLM can show. Indeed, if I didn’t know about them, I would hardly have batted an eye on encountering the claim that LLMs (or any sort of AI agent) can learn them. But they can’t. Likewise, they can’t solve the general problem of inference—that is, finding a provably optimally parsimonious hypothesis to predict future data. This is at the heart of the AIXI agent, which I think is the best theoretical model of a truly _general_ intelligence we have. And on my own model, the problem of finding a ‘safe’ self-modification to e.g. better adapt to the environment also turns out to be in that class. So here there’s two concrete behaviors that seem instrumental in bringing about a generally intelligent or even conscious agent which LLMs provably can’t emerge. > [@Sam\_Stone](#): > > Yes, we did. We fed a lot of human text in many languagues, and concept formation might be learned from that. We never told it how to do general addition, either. That emerged from the training data. I’m not sure why concept formation wouldn’t. Because addition is an operation on the properties of symbols (the syntactic level), whereas concepts form the semantic level, of which we simply haven’t told the LLMs anything. What they do know, again, is just the following: for any word, > [@Half\_Man\_Half\_Wit](#): > > what words it typically occurs together with, what position in the text it has in the given case (which together yields its encoding), and what other words are strongly influential on it (the ‘attention’-mechanism). These are not data sensitive to the concepts the words refer to. They can be harvested and manipulated in complete absence of any understanding. > [@Sam\_Stone](#): > > It’s just ‘adding’ or some other algorithm that just spits out words in order. > > In my opinion, that can’t possibly be complete. But it’s what we know to be the case—it doesn’t generate the next token based on a Markov chain model, true, but it absolutely does just generate the next token again and again, and absolutely all it uses to do so are the above mentioned three pieces of knowledge. That doesn’t mean that it can’t learn structures present in that data—such as, for instance, addition. But what concepts words refer to isn’t present in that data—it’s that fact that makes language useful at all: that words can point beyond language into the world. If language were a closed system, it would just be self-referential, without any point of contact with, well, anything. Words would only tell us about words. There simply is no credible story of how to get from the information which LLMs have access to, to the concepts words represent, because the _representational_ role of language isn’t available to them any more than the particulars of dolphin minds. There’s just no ‘there’ there. > [@Sam\_Stone](#): > > To get that, do you think Bing Chat is just doing next work prediction? Of course, yes, that’s all it _can_ do. That’s all it’s _supposed_ to do. Yes, it is amazing, and in my opinion, a major insight, that this works so well. But we must be careful not to be taken in by the spectacle, otherwise, we’re making the same mistake as those who see intent in the design of biological entities do—argue from mere incredulity that because we can’t imagine this to be the product of ‘blind’ processes, it must not be. But what Darwin showed for biology, we’re now being shown for language production: that the _appearance_ of intent does not suffice to conclude the _presence_ of intent. There are mechanisms to produce the former in the absence of the latter—evolution in the biological case, whatever we should call what LLMs and their ilk do in the case of language. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [March 23, 2023, 6:42am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1238 "2023-03-23T06:42:07Z")
> [@Half\_Man\_Half\_Wit](#): > > There might or might not be a way of finding that algorithm, but it does exist. If an algorithm is impossible to physically realize, then it may as well not exist. This is a point on which I feel there is something of a difference between the scientific and philosophical mindset. Both use thought experiments all the time. But scientists reject thought experiments that violate the laws of physics, or at least use them to explore what may or may not be possible. Philosophers seem to take no issue with proposing Chinese Rooms that simply could not exist, because they would collapse into a black hole. > [@Half\_Man\_Half\_Wit](#): > > You don’t need knowledge of the hidden variables to break causality, the mere fact that outcome sequences won’t be algorithmically random is enough—because that entails the presence of correlations that are, in principle, detectable (with nonzero probability in the limit, which entails a nonzero information carrying capacity of the channel). You need a computer program to determine if a sequence is random or not. If P!=NP, there exist one-way functions, and I can use that to generate a pseudorandom sequence that is arbitrarily hard to reverse-engineer, even with a computer the size of the universe. You will detect no correlation with anything else unless you discover the private data, which you can’t do in less than 2^N operations. You say that there is still a non-zero carrying capacity. But there is a non-zero probability of lots of things happening, like the bits on the far end of a communication device spontaneously configuring themselves into a given message before it could actually be sent. There’s no need to worry about events with an infinitesimal chance of happening. In all likelihood, properties like “causality” and “distance” are only emergent, macroscopic phenomena anyway. It wouldn’t bother me if they were violated some negligible amount of time due to random chance. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [March 23, 2023, 6:56am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1239 "2023-03-23T06:56:11Z")
> [@wolfpup](#): > > This thread is about AI and ChatGPT, and in this context the questions about emergence center around questions of what novel qualities might yet emerge in these systems. We’re also largely talking about capabilities that we already know are possible, like being able to multiply two numbers. These aren’t questions at the edge of algorithmic information theory. And mostly “emergence” has been taken to mean the process of _generalization_ from memorization into a general-purpose algorithm. Or, put another way, at what point is it able to successfully extrapolate outside its training set? It’s remarkable that this happens at all, even for simple cases. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 23, 2023, 7:40am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1240 "2023-03-23T07:40:35Z")
> [@Dr.Strangelove](#): > > If an algorithm is impossible to physically realize, then it may as well not exist. No. This is just a matter of logic: to show that a thesis doesn’t hold, it suffices to show that a counterexample exists. > [@Dr.Strangelove](#): > > There’s no need to worry about events with an infinitesimal chance of happening. That’s not what a ‘nonzero capacity’ means. That the capacity doesn’t go to zero in the infinite limit implies, by Shannon’s noisy channel theorem, that we can use the channel to transmit information with an arbitrarily suppressed probability of errors. > [@Dr.Strangelove](#): > > In all likelihood, properties like “causality” and “distance” are only emergent, macroscopic phenomena anyway. It wouldn’t bother me if they were violated some negligible amount of time due to random chance. Perhaps. But lobbying for the laws of physics to be revised just because you really want the world to be computable isn’t a sound strategy. The world is the way it is, and we’ll have to taylor our opinions to that, not the other way around. So in the end, without appealing to hypothetical wholesale revisions of the laws of physics, what remains, as Yurtsever puts it in [the paper](https://arxiv.org/abs/quant-ph/9806059) already referenced, is: > The result presented in this paper shows that if violations of local causality are to be ruled out […] it is not possible to simulate quantum mechanics on a digital computer (i.e., a Turing machine); quantum randomness is “uncomputable” in this sense. This fundamental lack of computability of quantum phenomena may have certain far-reaching implications; for example, if quantum-mechanical processes play a significant role in the activities of biological neural systems, then brain activity cannot be simulated faithfully on a digital computer, no matter how elaborate the simulation. [Previous page](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945.md?page=61) [Next page](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945.md?page=63)