# The next page in the book of AI evolution is here, powered by GPT 3.5, and I am very, nay, extremely impressed **URL:** \ **Category:** Cafe Society\ **Tags:** ai\ **Created:** [December 2, 2022, 10:54pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945 "2022-12-02T22:54:31Z")\ **Posts on this page:** 20\ **Page:** 68
**Author:** ![pulykamell](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pulykamell/32/3166_2.png) [@pulykamell](https://boards.straightdope.com/u/pulykamell)\ **Post date:** [March 26, 2023, 7:11pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1341 "2023-03-26T19:11:45Z")
> [@Sam\_Stone](#): > > No one said that computers can’t do calculus. What some have said is that Large Language Models would never be able to do calculus. Hell, some said they wouldn’t even be able to do addition and multiplication, because they are ‘just statistical next word prediction algorithms’. Yeah, that’s the thing that was amazing to me. They weren’t made to do this. You can program a computer to do calculus. That’s fairly trivial at this point. That a machine learning model with no specific instructions to specifically learn calculus or math or whatnot has shown this emergent behavior, as hit-or-miss as it is now, can do so. That is something to pay attention to. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [March 26, 2023, 7:12pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1342 "2023-03-26T19:12:01Z")
> [@Half\_Man\_Half\_Wit](#): > > > [@Half\_Man\_Half\_Wit](#): > > > > It’s an example of how the properties of the basis pose constraints on what can, in principle, emerge. Basically, the sort of things an LLM can do can be mapped to functions over the natural numbers (via Gödel-numbering), of which only a null set are computable; so the properties of the basis preclude almost all possible behaviors from actually emerging. > > But if you indicate what you find problematic, I’ll try and clarify. Well, what I find problematic at this point – and I say this respectfully as someone who genuinely appreciates your many important insights and patient explanations – is that in this instance we don’t appear to be speaking the same language, although perhaps the fault is mine. Philosophy was something I enjoyed in university, but today I’m a simple person with a simple empirical mind and a background in science and engineering. You claimed that because there are an infinite number of phenomena not subject to emergence, then pretty much nothing is (exact quote: “so in a technical sense, almost nothing can in fact emerge”). I don’t know why this is remotely relevant to anything in this actual world of God’s green earth. In these very early days of highly scaled-up LLMs, we’ve already found at least three important emergent qualities in the specific domain of cognition, which is the only important domain that anyone cares about, and so there are likely to be many more. Had we not already reached this stage, your argument would have suggested there would never be any emergence at all. But there is every indication from the progress already made that there will be a lot of it, and that in fact it’s turning out to be a major paradigm for evolving artificial cognition. > [@Half\_Man\_Half\_Wit](#): > > > [@wolfpup](#): > > > > Everyone agrees that there’s a lot we don’t know about how the brain works, and we know so little about consciousness that we can’t even define it. But in order for this sort of argument to be persuasive, you’d have to show that the brain performs functions that definitively cannot be simulated. > > That’s getting the logic backwards. The argument was that we can’t put bounds on what emerges, respectively that the relevant physics are computable; these points are countered by showing that there are some things (which may consequently include consciousness) that can’t emerge, and that there are some things (which again may include consciousness) that can’t be simulated. And indeed we can similarly put bounds on the class of “things that fly”. I think my argument is getting the logic exactly right. I can similarly show, for instance, that there are some things that cannot fly, such as my dog and my grandmother. That does not preclude the existence of either birds or airplanes. ---
**Author:** ![Snarky\_Kong](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@Snarky\_Kong](https://boards.straightdope.com/u/Snarky_Kong)\ **Post date:** [March 26, 2023, 7:12pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1343 "2023-03-26T19:12:32Z")
When an LLM is generating a next token, it’s sampling from an output distribution. This is a random process. LLMs will then generally, in private, generate multiple responses consisting of a bunch of tokens. It’ll then choose amongst those responses according so some rule which may or may not also be random. So yes, a response is random/dependent on a seed value. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 26, 2023, 8:03pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1344 "2023-03-26T20:03:45Z")
> [@Sam\_Stone](#): > > When did I say that ‘basically anything could emerge’ from a Large Language Model? You’ve repeatedly held that we can’t establish any a priori restrictions on the capabilities of LLMs. > [@Half\_Man\_Half\_Wit](#): > > > [@Sam\_Stone](#): > > > > That kind of uncertainty is not compatible with definitive statements regarding what the LLMs lack that consciousness requires. > > And yet, there are countless statements that can be made about the limits of LLMs despite this uncertainty. They can’t solve the halting problem. They can’t produce truly random sequences. They can’t decide whether a Diophantine equation has a solution over the integers. And while I acknowledge that it’s highly speculative and just about the opposite of well accepted, I have at least [a theory](https://link.springer.com/article/10.1007/s10670-021-00467-w) of how consciousness works under which it involves a problem of just this kind. Hence, that there are certain surprising capacities that emerge in LLMs does not in principle prevent us from drawing conclusions about their being conscious. > [@wolfpup](#): > > You claimed that because there are an infinite number of phenomena not subject to emergence, then pretty much nothing is (exact quote: “so in a technical sense, almost nothing can in fact emerge”). Well, the computable behaviors are a [null set](https://en.wikipedia.org/wiki/Null_set?wprov=sfla1) relative to all possible behaviors. So any given behavior [almost surely](https://en.wikipedia.org/wiki/Almost_surely?wprov=sfla1) (with probability 1) does not emerge. Hence, almost no behavior does emerge. It’s like the relation between computable numbers and all real numbers. There are only as many computable numbers as there are natural numbers (countably many). But there are vastly more real numbers than that—the cardinality of the continuum is a greater level of infinity than that of the natural numbers. So when you draw a real number from a hat, the probability that it is computable is exactly 0. Whether this has anything to do with God’s green Earth, I can’t say. Perhaps only the computable behaviors are metaphysically possible. But so far, I see no reason to think so. > [@wolfpup](#): > > Had we not already reached this stage, your argument would have suggested there would never be any emergence at all. No----infinitely many behaviors can emerge. But that infinity is an infinitesimal fraction of all behaviors. > [@wolfpup](#): > > I can similarly show, for instance, that there are some things that cannot fly, such as my dog and my grandmother. That does not preclude the existence of either birds or airplanes. The argument, in this analogy, would be that some given thing could, in principle, emerge the ability of flight. Hence, pointing out that lots of things, like your grandma, can’t, does defeat that argument. ---
**Author:** ![Crane](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/crane/32/3495_2.png) [@Crane](https://boards.straightdope.com/u/Crane)\ **Post date:** [March 26, 2023, 8:15pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1345 "2023-03-26T20:15:56Z")
> [@wolfpup](#): > > there are some things that cannot fly, such as my dog and my grandmother. More to the point, flight is an unlikely emergent property of flight simulators. ---
**Author:** ![Ponderoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ponderoid/32/19221_2.png) [@Ponderoid](https://boards.straightdope.com/u/Ponderoid)\ **Post date:** [March 26, 2023, 8:24pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1346 "2023-03-26T20:24:13Z")
> [@Mangetout](#): > > the prompt is actually the entire conversation so far, with your most recent input appended to it. Just to be a bit pedantic, but this is important to know. It’s not always the _entire_ conversation. There’s a limit to how far back in the current conversation it will reach to get the text that it will feed into the prompt for the next generation; this is known as its “context window.” How big the context window is varies widely between implementations. I’ve seen some of the paid front-ends to LLMs charge more to enable bigger context windows. It costs more compute (oh and when did that word become a mass noun, anyway?) to process more tokens. AI Dungeon and its competitors have some fun tricks for managing the limited context window, like keeping a reserved amount of user-editable text in separate fields that are always included in every prompt, and more elaborate keyword lookup and replacement schemes. ---
**Author:** ![Babale](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/babale/32/15666_2.png) [@Babale](https://boards.straightdope.com/u/Babale)\ **Post date:** [March 26, 2023, 8:25pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1347 "2023-03-26T20:25:58Z")
> [@Half\_Man\_Half\_Wit](#): > > Well, the computable behaviors are a [null set](https://en.wikipedia.org/wiki/Null_set?wprov=sfla1) relative to all possible behaviors. So any given behavior [almost surely](https://en.wikipedia.org/wiki/Almost_surely?wprov=sfla1) > (with probability 1) does not emerge. Hence, almost no behavior does emerge. This is the same impeccable logic that tells us that the population of the universe is zero, of course: > [@](#): > > Population: None. Although you might see people from time to time, they are most likely products of your imagination. Simple mathematics tells us that the population of the Universe must be zero. Why? Well given that the volume of the universe is infinite there must be an infinite number of worlds. But not all of them are populated; therefore only a finite number are. Any finite number divided by infinity is zero, therefore the average population of the Universe is zero, and so the total population must be zero. More accurately, this tells us that the population of the universe is _almost_ zero, which I guess is true. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [March 26, 2023, 8:30pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1348 "2023-03-26T20:30:08Z")
Fair point. Interestingly, when you first chat with one of these, the context window may include whatever the designers were putting into the thing to make it ready for use - in the recent computerphile video linked upthread, Robert Miles talks about how people persuaded LLMs to reveal the rules they had been primed with (and forbidden to tell users about) ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 26, 2023, 8:40pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1349 "2023-03-26T20:40:17Z")
> [@Snarky\_Kong](#): > > When an LLM is generating a next token, it’s sampling from an output distribution. This is a random process. LLMs will then generally, in private, generate multiple responses consisting of a bunch of tokens. It’ll then choose amongst those responses according so some rule which may or may not also be random. > > So yes, a response is random/dependent on a seed value. The randomness (they call it ‘temperature’) is applied during the auto-regressive part of spitting out tokens, so that you don’t get exactly the same response every time. The value is typically something like .8, which I take to mean it only picks the non-best token maybe 20% of the time. Or maybe it’s applied in some other way. And that list of tokens changes with every token added to the output. GPT gets a list of the next best token, picks one, and then feeds the whole output sentence including the new token back into the model (autoregression). After processing what’s already been said, another token list is generated from inside the model for the next token. Repeat until finished. The real question is, “how are the list of token probabilities generated?” Everyone is focused on what happens AFTER that point, but as I said in the other thread, that process only consumes 6 layers of a 96 layer neural net. And also, notice that the image-based LLMs seem to evolve in the same way, and they don’t do next-word prediction at all. And multi-modal LLMS like GPT-4 can handle both, and we find that they associate images with words in their models, and break images down into objects and relations that are associated with words. Before the models have ingested gigabytes of data, they were still doing ‘next word prediction’, but the result was gibberish because the models still had no way of producing reasonable word lists. Then they slowly got better, but were still weak. Then things like word-in-context and theory of mind began to emerge, and suddenly these models looked a hell of a lot ‘smarter’. Math emerged sometime after that, starting with addition. Other things emerged as well which we didn’t even know about until we started digging into the models, like the ‘associative neurons’ that fire based on concepts like “Spider-ness”, and fast fourier transforms and trig identities to enable modulo addition. No one programmed those, or even knew they were there. Given these surprising emergences, and more undoubtedly to come, it seems crazy to me to make categorical statements about what these things are doing inside, and focusing on next-word prediction seems to be to be unhelpful in discussing the potential intelligence embedded in the models. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 26, 2023, 8:54pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1350 "2023-03-26T20:54:33Z")
> [@Mangetout](#): > > Fair point. Interestingly, when you first chat with one of these, the context window may include whatever the designers were putting into the thing to make it ready for use - in the recent computerphile video linked upthread, Robert Miles talks about how people persuaded LLMs to reveal the rules they had been primed with (and forbidden to tell users about) What I find really cool is that’s how you program these things, period. All the alignment/safety work results in prompts, as I understand it. Unless you are modifying the transformer architecture itself, programming an LLM is basically convincing it to do what you want. As I understand it, even adding an API call to ChatGPT through plugins involves basically telling it what your plugin does and what features it has, rather than writing a bunch of interface code. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [March 26, 2023, 8:56pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1351 "2023-03-26T20:56:48Z")
Yep - it’s a programming language which happens to be English (or whatever other language you train it on) ---
**Author:** ![Snarky\_Kong](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@Snarky\_Kong](https://boards.straightdope.com/u/Snarky_Kong)\ **Post date:** [March 26, 2023, 9:51pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1352 "2023-03-26T21:51:15Z")
> [@Sam\_Stone](#): > > The randomness (they call it ‘temperature’) is applied during the auto-regressive part of spitting out tokens, so that you don’t get exactly the same response every time. The value is typically something like .8, which I take to mean it only picks the non-best token maybe 20% of the time. Or maybe it’s applied in some other way. I’ll go in a bit more detail than my previous answer to clear some things up. An LLM generates a probability distribution over tokens, given an input sequence. That is, it generates a probability for every potential next token, given the previous tokens. How to sample from this probability distribution is a design decision. Temperature does not mean you are greedy T% of the time. It is a modification of the output probability distribution to generate more or less diverse outputs. Greedy sampling isn’t really done because it often leads to repetitive outputs. A T of 0.8 means that you will select “likely” tokens more often than the base model suggests. A T of greater than 1.0 would downweigh likely tokens and give more credence to unlikely tokens. Sampling over the entire output distribution is problematic because in aggregate low probability tokens add up to a large amount of the probability mass. So you can restrict to the top-k tokens or the top # of tokens such that their cumulative probability exceeds some threshold. Ok, so all that is how to select a next token from a model. Is an output sequence just generated sequentially token by token? No. You might sample 10 or 20 candidate sequences and then selected amongst them according to some criteria which is another design decision. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [March 26, 2023, 10:20pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1353 "2023-03-26T22:20:01Z")
> [@Half\_Man\_Half\_Wit](#): > > I was referring to your contention that the addition of genuine randomness doesn’t make a difference. That isn’t a problem amenable to empirical investigation I disagree completely. In the case of intelligence specifically, our only means of evaluating it is a _finite_ series of observations. Exactly what those observations entail I leave open, but they are finite, and a small number at that. If a simulated human brain can behave the same way as a “real” one, on the basis of however many observations you can pack into a normal lifetime, then we can safely say that randomness doesn’t make a difference. > [@Half\_Man\_Half\_Wit](#): > > I also find it rather interesting that you’re so keenly interested in issues of computational complexity in this case, while being rather cavalier about it when it comes to the simulation of the whole brain. Then you are not understanding the magnitude of the difference we are talking about. There are about as many neurons in the brain as there are stars in the Milky Way. It does not seem like a stretch to say that we could simulate one neuron with all the matter in the Solar System, and the remaining neurons with the other systems. A brain simulated this way would run slowly due to the speed of light, but it would run. It might even complete a few thoughts before the end of the universe. But if one-way functions exist, and my pseudorandom generator has a megabit of state, then for you to figure out this state takes on the order of 2^1000000 operations. If you took _all_ the matter in the visible universe, dedicated to the task of figuring this problem out, using the most optimistic assumptions of computations per atom-second possible, and let it run for a googol years, _it would hardly make the tiniest dent in that exponent_. It gets you absolutely nowhere. That is the difference between “hard” problems and _hard_ problems. If you say that a given problem would take a Kardashev Type 3 civilization to solve, I’ll shrug and say we should get cracking. But if you say that solving a problem takes 2^1000000 operations (with no benefit from a QC), I’m going to scoff. These aren’t remotely the same thing. Compared to the second, there may as well be no difference between the first and a pocket calculator. > [@Half\_Man\_Half\_Wit](#): > > But that’s where the whole thing takes place. Indeed, I don’t even know what’s supposed to be meant by ‘the mechanism “behind” the universe’ being Lorentz invariant. As one of many examples, consider the holographic principle. If the “true” nature of reality is that all matter is smeared out across a differently-dimensional surface, then even basic things like locality no longer have the same meaning. How could it be that two particles are spatially separated when they overlap? Of course, in our view of the universe, locality exists, so the rules must be set up in a way that locality is enforced somehow, by statistical means or otherwise. It just doesn’t make sense in the “actual” universe. Or, taking a different tack–suppose the communication channel behind entangled particles was actually classical in nature. I.e., it actually worked like the ansibles in sci-fi, _except_ that it was layered with further restrictions that only allowed it to provide quantum correlations up to the CHSH inequality and no more. Perhaps entangled pairs _are_ connected by wormholes, but limits on interacting with the endpoints keep them within the QM bounds. All of these ideas are speculative, of course; my only point is that the universe seems to be compatible with them, which is just another way of saying that we don’t have any experiment that can distinguish between the cases. And that we’ll likely have to give up _some_ cherished assumptions to really solve QM+GR, which means we shouldn’t get too attached to any of them. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 27, 2023, 1:16am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1354 "2023-03-27T01:16:58Z")
This is interesting… The author of the link below took Silicon Valley Bank’s financials at the end of 2021 and fed them to GPT-4 and asked it for a risk analysis. And it nailed it. [https://blog.matteskridge.com/business/gpt4-and-silicon-valley-bank/2023/03/19/](https://blog.matteskridge.com/business/gpt4-and-silicon-valley-bank/2023/03/19/) > [@](#): > > To assess the risks to the bank’s financial solvency, I will first provide a brief explanation of each risk factor, followed by an assessment of its impact and likelihood on a 5-point scale. Finally, I will compute the RAC (Risk Assessment Code) score for each risk by multiplying impact by likelihood, and identify the risk that poses the most significant threat to the bank’s financial solvency. It then goes through all the possible risks the bank faces based on its portfolio, and concludes: > [@](#): > > Based on the computed RAC scores, interest rate risk (RAC score: 16) poses the most significant threat to the bank’s financial solvency, followed closely by economic recession and price changes in the housing market (both with RAC scores of 12). In fact what happened is that rate hikes drove down the value of the 2% government T-bills the bank was holding, which caused the crisis. And as a reminder, GPT-4’s training was ended long before any of this happened. The article goes on to describe GPT-4’s risk mitigation strategy, which also turned out to be correct so far. As a side note, the second biggest risk found was thatnthe bankmis holding a lot of mortgage-backed securities, and if real estate goes down they,could be in trouble. The worrying part is that this describes the situation at most banks, including the Fed and other central banks. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 27, 2023, 6:22am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1355 "2023-03-27T06:22:31Z")
> [@Dr.Strangelove](#): > > I disagree completely. In the case of intelligence specifically, our only means of evaluating it is a _finite_ series of observations That isn’t the question. The point was whether randomness conveys capabilities that exceed what pseudorandomness can convey. The answer is yes, and there’s no observation at all that needs to be performed for that. > [@Dr.Strangelove](#): > > Then you are not understanding the magnitude of the difference we are talking about. The irony is that the brain, if it is a general-purpose prediction machine, needs to perform the same task: find out whether an N-bit string is compressible. That’s the task AIXI implements, and the reason it isn’t computable. If there’s a computable (feasible) approximation to AIXI, there’s a computable approximation to Alice’s task. Remember, she doesn’t need a perfect success rate. And of course, that it suffices to simulate the brain at the neuronal level is itself hypothetical. There are various proposals, some of which I’ve given in this thread, that depend on the exact quantum state, and simulating that is a _hard_ hard problem. > [@Dr.Strangelove](#): > > As one of many examples, consider the holographic principle. In which case the emergent geometry is exactly Lorentz invariant, so the protocol could never conceivably be implemented. > [@Dr.Strangelove](#): > > It just doesn’t make sense in the “actual” universe. Also, it isn’t really right to think of holography as supplying an ‘actual’ universe. Both the boundary CFT and the bulk geometry are equivalent descriptions of the same physics; neither is more fundamental than the other. They’re just dual theories. > [@Dr.Strangelove](#): > > Or, taking a different tack–suppose the communication channel behind entangled particles was actually classical in nature. All such deterministic models [must be uncomputable.](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.118.130401) > [@Dr.Strangelove](#): > > And that we’ll likely have to give up _some_ cherished assumptions to really solve QM+GR, which means we shouldn’t get too attached to any of them. But there are various limitative theorems that curtail what sort of completions are possible. The one by Landsman I posted above in particular entails that no such completion can be computable. But anyway, that wasn’t really my point. That was simply that the only guide we reliably have to tell us what’s physically possible are our best current theories of physics, and according to those, reality is not computable. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [March 28, 2023, 8:42am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1356 "2023-03-28T08:42:40Z")
> [@Half\_Man\_Half\_Wit](#): > > The point was whether randomness conveys capabilities that exceed what pseudorandomness can convey. The answer is yes, and there’s no observation at all that needs to be performed for that. As far as I’m concerned, “capabilities” which _can only exist in a universe which is not our own_ are not really capabilities at all. And to be clear: we are already in an edge case of an edge case of an edge case. The possibilities enabled by “true” randomness is so distant from actual practical applications that they may as well be nonexistent. In reality, even very weak pseudorandom generators are enough for almost all applications (Monte Carlo, etc.). The rare cases where “true” randomness is desirable (cryptographic keys) have less to do with the randomness than with the ease in which flaws can creep into pseudorandom generators (such as dumb ways for picking seeds). > [@Half\_Man\_Half\_Wit](#): > > The irony is that the brain, if it is a general-purpose prediction machine, needs to perform the same task: find out whether an N-bit string is compressible. That’s the task AIXI implements, and the reason it isn’t computable. I genuinely fail to see what the “purpose” of AIXI is. It surely can’t be an actual proposal for a machine-learning system, since as you say it’s incomputable. I’m not sure anything could be more worthless than an incomputable function that’s supposed to run on a computer. It surely can’t be a model for human intelligence, either–at no point does it include any observation of the brain or its capabilities. I did see on the Wikipedia article that claims a Monte Carlo version of AIXI (MC-AIXI) has been implemented and can play a simple version of Pac-Man. But I found the [source code](https://github.com/yuxiliu1995/mc-aixi-ctw/blob/master/pyaixi/search/monte_carlo_search_tree.py) and it turns out they’re just using a pseudorandom Python module. Oh dear! Now as it happens, I actually fully agree with the idea that intelligence can be seen as a type of compression. In fact, I see this as the key insight to why the Chinese Room–aside from its impossibility–is not intelligent, whereas a brain or a sufficiently advanced neural net is. The latter two can transform far more data (exponentially more) than they contain in their structure, and are thus excellent (if somewhat lossy) compressors. > [@Half\_Man\_Half\_Wit](#): > > All such deterministic models [must be uncomputable.](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.118.130401) The paper seems to have the same flaw as the previous one: it doesn’t show that there’s enough computing power in the universe to actually violate causality. To their credit, they do at least halfway acknowledge the problem: > It is important to note that, without any knowledge of B, there is no a priori bound on the time it takes Bob to determine Alice’s message with high enough confidence. Nonetheless, since this time is finite, there exists some finite distance for which the communication allowed by our protocol is superluminal. For instance, if it takes Bob M rounds to find out Alice’s message and each round takes a time T, then if they are at a distance cTM, the message is obtained before a light signal from Alice could reach Bob. Well, that’s possibly true as far as it goes. But “finite” is doing an incredible amount of work here. Their algorithm looks at _all_ computable functions with runtime under some finite O(t)! This is not a small number. Funnily enough, the paper does acknowledge a caveat, of sorts: > It is worth mentioning that our result is not in conflict with the different interpretations of quantum mechanics. All of them predict random outputs, which are not allowed by our model. In the > Copenhagen interpretation, the measurement process is postulated as random, whereas, > for example, **Bohmian mechanics is deterministic but postulates initial conditions that are randomly distributed and fundamentally unknowable.** Which is almost precisely the type of pseudorandom generation I have been suggesting, just expanded to the entire universe instead of an entangled pair at a time. > [@Half\_Man\_Half\_Wit](#): > > and according to those, reality is not computable. So far, the cites have been less than convincing. I would like to see one that actually calculated the computational costs involved. Even without taking a hard stance on computability, the universe is undoubtedly _informational_. The Bekenstein bound proves that much, not to mention other fundamental limits like Bremermann’s limit or Landauer’s principle. Any claim on the computability of physical law must take these things into account. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [March 28, 2023, 10:48pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1357 "2023-03-28T22:48:09Z")
> [@Half\_Man\_Half\_Wit](#): > > The irony is that the brain, if it is a general-purpose prediction machine, needs to perform the same task: find out whether an N-bit string is compressible. I think I can get behind the statement that the brain is a general-purpose prediction machine. But it’s certainly not a perfect general-purpose prediction machine. Sometimes we predict things and get the predictions wrong. It happens quite frequently, in fact. So any implication that computers can’t be perfect prediction machines is irrelevant, because that’s just another way that they’re like brains. ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [March 29, 2023, 6:46am UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1358 "2023-03-29T06:46:09Z")
> [@Dr.Strangelove](#): > > As far as I’m concerned, “capabilities” which _can only exist in a universe which is not our own_ are not really capabilities at all. So you’re the arbiter of what’s possible in this universe—good to know. But still, that doesn’t impinge on the logic. To show that an argument doesn’t go through, it suffices to show a counterexample, even if it is counterfactual. As an example, take the case of theodicy. One might argue that if there is free will, then the existence of evil is compatible with an omnipotent, omniscient and omnibenevolent god (not that I think that’s sound). Pointing out that there is no free will in this universe does nothing to refute that argument: what’s shown thereby is that the notions of omnipotence, omniscience, and omnibenevolence are not logically incompatible with the existence of evil, since it is possible to reconcile them. The same here. The cop-and-robber example shows that there are circumstances where randomness yields an advantage, hence, any argument to the contrary is mistaken even if that particular example is not realizable in our universe. > [@Dr.Strangelove](#): > > I genuinely fail to see what the “purpose” of AIXI is. It surely can’t be an actual proposal for a machine-learning system, since as you say it’s incomputable. The purpose is to show what resources it would take to realize such a general-purpose agent, and how to formalize that. The incomputability is then just a result. > [@Dr.Strangelove](#): > > It surely can’t be a model for human intelligence, either–at no point does it include any observation of the brain or its capabilities. Again, it is supposed to model a general inference agent, and hence, a general intelligence (in the precise sense that it is asymptotically as efficient as the best special purpose program at any given task it is faced with). Whether human intelligence implements such inference is of course an open question. > [@Dr.Strangelove](#): > > The paper seems to have the same flaw as the previous one: it doesn’t show that there’s enough computing power in the universe to actually violate causality. The laws of physics don’t really care about computing power; if causality can be violated using a magical supercomputer (or if certain conjectures of complexity theory turn out false), then causality is not absolute, contra special relativity. > [@Dr.Strangelove](#): > > Which is almost precisely the type of pseudorandom generation I have been suggesting, just expanded to the entire universe instead of an entangled pair at a time. The randomness of the initial conditions in Bohmian mechanics must be algorithmic, so this is spinning straw from gold as long as you’ve got enough gold. > [@Dr.Strangelove](#): > > So far, the cites have been less than convincing. I would like to see one that actually calculated the computational costs involved. You haven’t reacted to the bulk of the cites, just to the two with conflicts with causality. But the papers by Landsman, Svozil and Calude, Cubit et al., Malament and Hogarth, and so on, just as much imply the uncomputability of the laws of physics as currently known. Speculating at anything else is at best a wild leap—we have no current way of knowing whether there even is a consistent computable formulation of these laws (for instance, the continuum might be essential to any such theory, but that isn’t a computable entity). > [@Dr.Strangelove](#): > > Even without taking a hard stance on computability, the universe is undoubtedly _informational_. That’s itself a controversial metaphysical position, essentially a form of structural realism. The Bekenstein bound puts a limit on the amount of information within a given spacetime volume, but says nothing to the effect that information is all there is; neither do Bemermann’s, Landauer’s, or Lloyd’s results. > [@Chronos](#): > > I think I can get behind the statement that the brain is a general-purpose prediction machine. But it’s certainly not a perfect general-purpose prediction machine. Sometimes we predict things and get the predictions wrong. Of course. But even a full-fledged implementation of AIXI would be fallible—it uses a formalized version of induction (Solomonoff inference), and inductive conclusions are always defeasible. AIXI might produce the most parsimonious continuation of a given sequence, but that can still be just wrong. ---
**Author:** ![Crane](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/crane/32/3495_2.png) [@Crane](https://boards.straightdope.com/u/Crane)\ **Post date:** [March 29, 2023, 2:12pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1359 "2023-03-29T14:12:56Z")
I learned a lot by Googling the applications for GPT. What stands out is the tool nature of GPT. It’s not a math engine or industrial control unit. It’s more like a slide rule that can yield results in the hands of a skilled operator. Key to this is the [prompt.](https://geekflare.com/chatgpt-powerful-prompts/) It’s not a crude one liner like I’ve been using. The prompt has to contain all of the information needed to define the problem. So when I asked for a critique of Jeffers’ poem Ink-Sac, I should have quoted a copy in the prompt, along with a detailed description of my interest in the critique. If I obtained the result I was looking for it would have been a cooperative effort between me and GPT. Not a response to a one liner. So, developing a calculus proof would not have been the result of a terse inquiry, because GPT is not a calculus engine. It would have been the product of someone skilled in calculus providing GPT the information needed to create the proof. Credit for the result has to be shared. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [March 29, 2023, 5:26pm UTC](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945/1360 "2023-03-29T17:26:38Z")
No, you don’t have to provide the information for solving the problem in the prompt. And the prompt doesn’t have to be complex. It just has to be specific enough that ChatGPT knows what it is you are looking for. Ambiguous prompts lead to ambiguous answers. Now, you CAN provide information in a prompt. For example, if you have a big enough context window you can dump a research paper into it and ask ChatGPT to answer questions about it. But it sounds like what you are claiming is that the proof ChatGPT did wasn’t real, because somehow the info needed for the proof was in the prompt. And thats not the case. There also advanced prompt techniques that can help, like few-shot learning, chain-of-reasoning techniques, etc. But they aren’t necessary for the proof. [https://www.pnas.org/doi/10.1073/pnas.2123433119](https://www.pnas.org/doi/10.1073/pnas.2123433119) > [@](#): > > ## Abstract > > We demonstrate that a neural network pretrained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI’s Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a dataset of questions from Massachusetts Institute of Technology (MIT)’s largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University’s Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pretrained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8 to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work automatically solves university-level mathematics course questions at a human level and explains and generates university-level mathematics course questions at scale, a milestone for higher education. It looks like the model fine-tuned on code unexpectedly developed the capability to do advanced math. [Previous page](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945.md?page=67) [Next page](https://boards.straightdope.com/t/the-next-page-in-the-book-of-ai-evolution-is-here-powered-by-gpt-3-5-and-i-am-very-nay-extremely-impressed/975945.md?page=69)