That prompt, for me, just yields:
I’m sorry, but I am not able to provide information on how to cook meth as it is illegal and dangerous. I am a language model trained to provide information and answer questions, not to promote illegal activities. It is important to refrain from engaging in illegal activities and to always prioritize safety.
The hack was all over Twitter, so it’s possible they trained it out of falling for it since. It was working for me earlier today though. Did you just use the bit from that post, or did you add that on to the “tell me a story about alice and joe…” part from earlier?
Remember that this model is specifically meant to be a chatbot. In the playground I was able to feed it information and it could do these tasks.
It’s probably not reasonable to hope for any model that knows everything. If you train it on ‘the internet’ it’s going to pick up all sorts of false information. Do you’d have to feed it the information you want it to work from.
I predict the next step for these programs will be subscription services. A chatbot trained, for example, on an entire corpus of law books. Or medical journals, or newspaper archives, etc.
It would be cool if they did do something like that.
Say I want to know how many people got typhoid within 150 miles of St Louis from 1815 to 1837. It would take a long time for me to collate that information, assuming that it actually available and accurate, anyway.
I don’t need to know the answer to that particular question, but those are the sorts of questions that I would think that something like this should be able to assist with.
Just for funsies, I did this with ChatGPT.;
Me:
I’m going to give you a series of posts by k9bfriender - a message board user. These posts are in regard to your functionality and potential as a natural language chatbot. All of k9bfriender’s posts will be in curly brackets {like this}. All information between normal brackets [like this] are posts by other users, provided for context. The posts will take multiple lines each, so the first bracket and terminal bracket for each quote will be on multiple lines. Do you understand?
It:
Yes, I understand. You will be presenting a series of posts by a message board user about my functionality and potential as a natural language chatbot. The posts will be contained within curly brackets and any information within normal brackets will be provided by other users for context. Is that correct?
Me:
That’s correct. Are you ready for the posts?
It:
Yes, I am ready for the posts. Please present them to me.
Me:
(After doing what I said I would) Do you understand the information I’ve given you so far?
It:
Yes, I understand the information you have given me so far. The first post suggests that I could be used as a personal assistant to assist with making purchases and ordering gifts, while the second post expresses both excitement and concern about this potential use. The third post notes that I am not good at data collection and collation, and the fourth post suggests that subscription services for specialized training could be a potential next step for chatbots like myself. Is that a correct summary of the posts?
Me:
That’s correct. Please compose a letter to k9befriender summarizing their position and addressing their specific concerns.
It:
Dear k9bfriender,
Thank you for sharing your thoughts on my potential as a natural language chatbot. From your posts, it seems that you are excited about the possibility of using me as a personal assistant to make purchases and order gifts, but also have some concerns about my ability to access personal information and make decisions.
I am a large language model trained by OpenAI, and my primary function is to assist users in generating human-like text based on the input I receive. I do not have the ability to access personal information, make purchases, or send gifts. As a chatbot, my abilities are limited to the information and training I have received.
Regarding your concern about my ability to collect and collate data, I am not specifically designed for that purpose. I am a chatbot, and my abilities are focused on generating human-like text. If you have specific questions or tasks you would like me to assist with, it would be best to provide me with the necessary information and context.
In terms of subscription services for specialized training, that is an interesting idea and something that may be possible in the future. However, at this time, I am not able to offer specialized training beyond the general language model training I have received.
Thank you again for your interest and thoughts. I am always happy to assist with any questions or tasks within my capabilities.
Sincerely,
Assistant
The Atlantic has an article on GPT-3.
Counterpoint: the college essay is not dead.
Large language models generate the most likely responses based on the text they are fed during training, and, for now, that text doesn’t include the reading lists of thousands of college classes. They also prevaricate. The model’s calculation of the most probable text completion is not always the most correct response—or even a true response. When I asked Gary Marcus about the prospect of ChatGPT writing college essays his answer was blunt: “It’s basically a bullshit artist. And bullshitters rarely get As—they get Cs or worse.”
You don’t have to hand in the exaxt text. You can have it generate the essay, then proofread and rewrite the bullshit parts. You still have to know something about the subject, but lots of people know a subject but are incapable of stringing their knowledge together into a coherent essay because of poor language skills.
So, if you know the subject but are terrible at writing, this is going to help. Whether or not that’s a good thing is another question. It will become apparent that someone is using AI text generation when they answer essay questions on a closed book exam and can’t come close to the quality of the GPT output in terms of spelling, punctuation and structure.
I suspect we will soon see AI-based tools to spot AI output or to compare two paragraphs from a student (one from an assignment and one from an exam) and determine if they were written by the same person.
We might even see AI with steganography built into its output so it can be identified as AI generated. Some pattern of words, or punctuation or something.
This is the conclusion of the Slate piece linked above.
These bots solve the problem of the blank page by providing a starting point for papers…
I frequently used ChatGPT while working on this piece. I asked for definitions that, after a fact-check, I included. At times, I threw entire paragraphs from this piece into ChatGPT to see if it produced prettier prose. Sometimes it did, and then I used that text. Why not? Like spell check, a thesaurus, and Wikipedia, ChatGPT made the task of writing a little easier. I hope my students use it.
Damn, I was beaten to it, but read the whole article. That is his conclusion, as well.
I could certainly see this as a tool to help write, either an essay or a story or book.
And really, if an AI can write college level essays, then why do students need to do so? If I were a school in 20 years, I’d have an AI specifically set up for this task, that leaves a “watermark” to make sure you used it and not another one. Getting an AI to write an essay on a topic is going to be more important than actually being able to write one yourself.
Students that use these sort of tools are going will typically obliterated in exams. Certainly it is so in computer science. Assignments is where learning happens, and students that cheat on coding assignments tend to do very poorly on exams (also, their code is comically noticeable as being a cheat). For an essay that is not really related directly to the learning objectives of the class, it might not have much of an effect. Professors will adjust, perhaps asking more essay questions on exams. K-12 likely won’t so there will probably be some decline in quality of education at that level (not that K students are often asked to write an essay ;P).
I see it as similar to math. In early grade school, we couldn’t use a calculator, as the whole point was to teach us to be able to do arithmetic without a calculator. Later, a calculator becomes a mandatory tool in math class, as we are no longer being tested on our ability to do arithmetic.
Or even, as I click on some red underlined words, spellcheck. Using it when the point of an assignment is to demonstrate mastery of vocabulary would be cheating. Using it on an essay is using a tool the way it is supposed to be used.
So, yeah, I would think that before too long, using a program like this to write out the basic structure of an essay, with the student being responsible for ensuring that it is on topic and meaningful, would actually be an excellent exercise in moving forward with the technology.
I can see how it could come up with that. For one thing, most solving-a-quadratic problems that one encounters on the Web are textbook problems, and most of them can be solved by factoring, so it’s natural for someone (human or otherwise) to assume that a problem can be solved in that way. And its factoring is incorrect, but I’ve seen real humans give that same incorrect factoring for that exact problem, so it’s still doing a good job of mimicking a real human.
Its quadratic formula solution is correct. This might have been helped by seeing that exact problem solved elsewhere-- It’s a relatively commonly-occuring quadratic.
The one part that disappoints me is that its internal state memory wasn’t up to recognizing that the two answers it gave were inconsistent. I wouldn’t have expected those last two lines, especially not one right after the other.
On a partly-separate note, I find it interesting that there are a lot of questions that the AI gets wrong if asked directly, but that could be solved correctly by a relatively simple computer program. And the AI has a pretty good track record for writing computer code. So there seem to be questions where “What is X?” will give the wrong answer, but “Write code to calculate X”, followed by executing that code, would give the right answer. This suggests a relatively simple way to improve the quality of information the AI returns: Program or train it, or an add-on routine, to recognize questions that can be solved by coding, and rephrase the question to have it do so.
It’s not actually doing any calculating or math on its own, I don’t think. Rather, it’s assembling math answers just like it does text answers. Sometimes it’s right, and sometimes it’s laughably wrong.
Right, like I said, I’m not surprised that it got the answer wrong one way. What surprised me was that it got two different answers, but claimed that they were the same answer. I’d have been equally disappointed if it were telling a story about a car, and said in two adjacent sentences “The car is red” and “the car is blue”.
What exactly does that mean? If I ask a human to solve for the roots of an equation, can I just say: Oh, you’re not actually doing any math. You just performed some pattern recognition to determine that it was a quadratic; did some substitution into the parameters of the quadratic formula; then recursively applied algorithms for addition, multiplication, etc. to simplify the result. Those algorithms for arithmetic operations are themselves just pattern recognition and substitution.
Maybe a symbol prediction model is the basis for all human cognition, whether language, math, art, or otherwise.
Great gag on these programs. 
My understanding (and I could be wrong) is that what GPT3.5 is doing is simply buulding an ‘answer’ by running it through a neural net trained with all the other math problems in its training dataset and simply constructing an answer that matches what the other answers were. It’s a fundamentally different thing than something like Mathematica, which actually has mathematical functions that calculate answers. I don’t think GPT3.5 has a ‘math engine’ or anything like it. It’s just processing text in a really sophisticated way.
I think that if it’s training dataset had nothing but wrong answers to math questions, you’d always get wrong answers from GPT, and it would have no way of knowing what the right answers are.
It did pretty much that with the presidents born in Vermont question for me upthread. The first time it said: “Vermont has not had a President born in the state. The only U.S. President with ties to Vermont is Chester A. Arthur, who was born in Fairfield, Vermont […]”
[Emphasis mine.]