Generally, that’s overkill except during training. 16-bit float formats (either FP16 or BF16) are almost always sufficient for inference. And in many cases, 8 or even 4-bit formats. The latest NVIDIA Hopper chip does 2 petaflops in FP8. Reducing the precision both increases raw performance and the number of weights you can store.
Pytorch has automatic mixed precision (AMP) which allows training in lower precision without zeroing out gradients. But you don’t really care about that when running a model, so yeah, 32 bit is overkill and it’s very common to “quantize” a model to make it efficient for inference.
Every time I have tried to access it, the site has been “too busy”.
As for results I have seen: I don’t want to appear too sceptical, but I wonder if this is like a lot of “self driving” cars. Oh yes, the computer is doing the driving but (ahem) there’s a human operator looking over its shoulder ready to take over in, er, ‘difficult’ situations?
Okay, now I want one of these:
I started checking out ChatGPT a couple of weeks ago due to some threads elsewhere about it as I’m a writer and had concerns about it possibly taking work away. I asked it several questions and I asked it to write articles with specific parameters to see what it could come up with.
It seems to have a hard time with factual things like if I ask for certain types of restaurants in a specific state, it would come up with a list of places where many of them didn’t exist or were in another state. When I asked it to write articles on a small town in a certain state, it got many of its facts wrong and just seemed to be generating generic, easy to read filler with several “facts” that either weren’t true, or were true, but for another town, not the one it was attributing them to. In short, at first glance it looks like nice content but if you were to try to make actual use of the info provided, you’d quickly find that the “writer” had no clue what they were talking about.
I think my job is safe.
That said, I’ve heard it’s quite good at generating code as that’s what it was trained on. You’d have to ask programmers to be sure.
It can write code reasonably well, but the first time I tried it, it included some linrary functions that have been long deprected.
It makes similar errors as it does with text sometimes, but the beauty of code is that bad code generally won’t compile or it will throw errors. Then you can tell ChaTGPT what happened, and it will fix the code. When I told it the libraries it used were no longer available, ot rewrote the whole thing with a different set of libraries, and it was fine.
I wouldn’t use it for generating any code I didn’t understand, and I would walk through any code it generates looking for subtle bugs until I was more confident in it. But it is great for boilerplate work, such as writing unit tests for functions or SQL statements.
I asked ChatGPT the following question: Explain the uses of the shuffle idempotent in homological algebra. The answer I got surprised me. It started with
“In homological algebra, the shuffle idempotent is a powerful tool used to study the structure of algebraic structures such as associative algebras and their representations. The shuffle idempotent is a certain element of the tensor algebra, and its name comes from its action of shuffling tensor factors.”
This is absolutely correct. It then gave a number of examples and concluded with
“In summary, the shuffle idempotent is an important tool in homological algebra that appears in many different contexts. Its ability to express certain algebraic structures as sums over permutations or cyclic permutations makes it a powerful tool for computation and understanding.”
which is also correct. My main disappointment was that it omitted the most important example and the one for which the shuffle idempotent was created.
Try calling it on its omission, and see if it does a good job of correcting itself. Sometimes it can, sometimes it can’t, and sometimes it depends on what hints you give in your correction prompt.
I finally got it to respond to me. If you ask it factual questions, it seems to do a fairly good job of spitting out something that resembles a Wikipedia article. One suspects that Wikipedia was a large part of its training base? And it also seems to do a fair job of generating a comparison between fairly related things “eg Stratocaster vs Telecaster”.
But is a Wikipedia on steroids close to intelligent? And of course it has its ‘get out of jail free’ card: “as an AI language model I don’t /can’t…”
It’s interesting, but I don’t think it is close to strong AI.
Aha! I caught it in a simple factual error. I asked how many types of single-carbon molecules exist. And it gave a reply which mentioned acetylene and ethylene (which are not single carbon), and did not really address the original question.
I suspect it is just using massive brute-force searching for keywords in a large database of text samples with indexing based on common word patterns which occur close together. Like trigrams, which can sometimes produce text which looks superficially quite similar to natural language.
Probably not much closer to AI than Eliza, really.
It’s really not well trained on chemistry! For example:
“why is chloroform not a single carbon molecule?” which produces:
“Chloroform is not a single-carbon molecule because it is composed of more than one atom of carbon. Chloroform has one carbon atom, one hydrogen atom, and three chlorine atoms, which means it is a compound that contains multiple elements.
Single-carbon molecules, on the other hand, are composed of only one carbon atom and no other elements. Methane (CH4) and methanol (CH3OH) are examples of single-carbon molecules because they contain only one carbon atom and are not composed of any other elements besides hydrogen and oxygen.”
Hey, Eliza, get back to me on this, won’t you?
This is an example of “hallucination”: drawing incorrect inferences to collate and return incorrect or inapplicable data as an authoritative response.
In other words, a chatbot seems unable to say “I don’t know” and will make plausible-sounding shit up.
LLMs are not a replacement for Wikipedia. They aren’t designed to be a source of truth. Don’t use them like Wikipedia.
You and I know that. But they can appear to be very plausible, if you don’t already have a fairly good general education to base your judgements on?
It’s just pulling those phrases from search engine hits. It’s going to come up with some very accurate phrasing but will miss things that a human familiar with the subject would include.
For fun I asked ChatGPT to create lyrics in the style of various musicians.
The Tragically Hip:
In the streets of this small town
The sun sets and the lights go down
A lonely figure walks alone
Aching heart and a broken soul
The river flows, it never stops
A constant current, through the rocks
And we’re all just trying to keep our heads above water
Pushing forward, harder and harder
Bob Dylan:
The wind howls through the city streets,
As the darkness falls and the day retreats.
People rushing to and fro,
Trying to keep up with the ebb and flow.
Chorus:
But there’s a voice that echoes in the night,
Singing out for truth and light,
A troubadour with a guitar in hand,
Leading us to a better land.
Jay-Z:
Verse 1:
I came from the bottom, now I’m living my dream
Got a million-dollar empire, they call me the king
Never forget where I came from, always stay true
Got my mind on my money, but my heart’s with my crew
Chorus:
I’m the blueprint, the one they all follow
I make moves, they all just hollow
I’m living proof that you can rise to the top
Just stay focused and never stop
Lisa Loeb:
Sitting in my room, staring at the wall
Wondering if anyone cares at all
Trying to find the words to say
But they just won’t come out today
Chorus:
I’m just a girl trying to find my way
In this crazy, messed-up world today
Looking for a sign, looking for a clue
Hoping that someday, I’ll find the truth
Alanis Morrisette:
Verse 1:
You said that you loved me, but you lied
And now I’m picking up the pieces, trying to survive
I gave you everything, and you took it all
But when you left, you left me with nothing at all
Chorus:
You think you can break me, but I won’t fall
I’m stronger than you ever thought I could be, after all
I’ll rise up from the ashes, and I’ll stand tall
I won’t let your betrayal define me, no more
Aerosmith:
Livin’ on the edge, with the wind in my hair (Living on the Edge is an existing Aerosmith song)
Riding high, with no worries or a care
Rockin’ all night, with the music in my soul
This is where I belong, this is where I feel whole
Chorus:
I’m an Aerosmith cowboy, with a heart of steel
Living on the edge, with a burning wheel
I’m a rebel, with a rock and roll soul
A true rockstar, and I’m ready to roll
The Tragically Hip example is brutally terrible and the Aerosmith one just broke ChatGPT. The others are… well, they’re clearly done by a computer. I can see what ChatGPT is TRYING to do; the Bob Dylan one uses the word “Troubador,” the Jay-Z one is all about being from the streets, the Lisa Loeb one is sad and introspective, the Alanis one is angry. But all are OBVIOUSLY written by a computer; it’s very clear none are written by a human. I’m no poet and I could do better.
In other words, a chatbot seems unable to say “I don’t know” and will make plausible-sounding shit up.<<<
Sounds just like my brother-in-law. At the drop of a hat, he will give you a long, detailed and almost completely wrong ‘explanation’ of something he knows nothing about. One learns to nod along and say, OK, Tim. No point in arguing about actual facts.
I just dipped my toe into ChatGPT for the first time to try to unblock a brain cramp for some lyrics I’ve been working on. I didn’t feel good opting for that solution but I told myself even if it came up with something good, I wouldn’t use it. I just wanted to see what it would come up with. I felt so ashamed clicking that button.
“Write an ironic anti-gun song. Finish these lyrics: …” and I threw in my first verse.
Holy shit did that come up with some absolute sophomoric garbage. For example the chorus starts thusly:
“With irony and wit we’ll make 'em see …”
Seriously? Alanis Morissette is rolling over in her grave.
I’ve spent a bit of time playing around with it, just for amusement. I showed it to a friend a couple of months ago using the prompt “Write a wedding invitation in the style of William Faulkner,” and his (my friend’s) reaction was sublime.
Yep, definitely not good at finishing poems or just doing poems at all. But play around with Bing and Claude.ai, and see if they perform any better. I find there’s a good bit of difference between the various AIs. They all pretty much suck at poetry/lyrics, but there’s degrees of suck. I find the best thing for me to do in these sorts of situations is to query the AI and poke its “brain” about what interesting directions for me to take the lyrics might be. Something like that. I tend to use it to bounce ideas off of and to just engage in a back-and-forth about the work.
And ask for multiple suggestions. I always ask for ten options for whatever I’m trying to rewrite or add to. I usually find one of them is pretty good.
I’ve used it at work to debug some code. I found it helpful as I am a tester and
rarely do any sort of programing.
I tried asking it an obscure technical question (What is the purpose of a petticoat pipe
on a steam locomotive?) and was surprised when it quickly returned a correct and
clearly written answer. One the other hand, it didn’t seem up to date with the news.
When I ask it who is the monarch of United Kingdom it replied with Elizabeth II.
Then I had to try a few silly questions. When I used song titles that were questions
it would identify the song and not answer the question (for example: “Will You
Still Love Me Tomorrow?” and “Who Put The Bomp (In The Bomp, Bomp, Bomp)?”
And it became confused when I asked it:
Is Taylor swift?
Is Martin short?
Is Neil young?
Is Maxwell smart?
Is Marvin gay?