The next page in the book of AI evolution is here, powered by GPT 3.5, and I am very, nay, extremely impressed

[wolfpup]

@Sam_Stone

Those who think LLMs are just ‘statistical word lookup engines’ are missing the important part. The language generation may be partially stochastic, but the determimation of the list of probabilities themselves is the result of that giant neural network, and it is not simple. @wolfpup 's examples are not something easily amenable to just ‘statistical word lookup’. They require some kind of logical process of analysis before the list of word probabilities can be conjured. It is hellishly complex to the point where we only have a vague understanding of how it works, other than it’s a neural network just like our own brain. Somewhere, concept formation is going on.

I just want to say that I think your post is well written, and I especially concur with this part. With regard to that last sentence about concept formation, in addition to the other examples I gave, check out this one I just ran by ChatGPT – it’s from a section of an IQ test related to logical reasoning skills. Most of those examples illustrate one or more aspects of what I’d consider impressive analytical performance. This one illustrates the decomposition of the verbal information presented into a correct set of equations, thereby transforming the question into a math problem which it then proceeds to solve. And yes, the answer is correct.

A fisherman has 5 fish (namely A, B, C,D, E) each having a different weight. A weighs twice as much as B. B weighs four and a half times as much as C. C weighs half as much as D. D weighs half as much as E. E weighs less than A but more than C. Which of them is the lightest?

Let’s use a system of equations to represent the information given:

A = 2B
B = 4.5C
C = 0.5D
D = 2E
E < A and E > C

We can use these equations to determine the relative weights of the fish.

Substituting for B in terms of C, we get:

A = 2(4.5C) = 9C

Substituting for C in terms of D, we get:

A = 9(0.5D) = 4.5D

Substituting for D in terms of E, we get:

A = 4.5(2E) = 9E

Since we know that E is lighter than A, but heavier than C, we can conclude that C is the lightest fish. Therefore, fish C is the answer to the question.