Could artificial intelligence replace journalists?
You think the main threat from artificial intelligence is that it will destroy humanity? Obviously you’re not a journalist. On hearing that the AI R&D lab OpenAI had launched a free online version of its intelligent chatbot, ChatGPT, the first thought of most journalists, and probably anybody who writes for a living, was: can this thing do my job? For me the question was especially urgent, since ChatGPT supposedly could answer any question. Yipes, I thought. That’s my gig.
I decided to see what ChatGPT could do with some representative Straight Dope questions, and whether it could do them in my style. (I know, how shallow, but I’m not the first columnist to think of this.) The Straight Dope questions I had in mind had non-obvious answers that were nonetheless well established, making them a useful benchmark.
Having logged in to ChatGPT, I typed:
write a straight dope column about whether there are nine eskimo words for snow in the style of cecil
Unfortunately, I mistakenly hit “send” before I could type adams. ChatGPT, no doubt uncertain whether Cecil Adams, Cecil Rhodes, or Cecil B. DeMille was meant, spat out – and I mean, in milliseconds – the following:
It’s a popular belief that the Inuit people, commonly referred to as Eskimos, have over 50 words for snow. This claim has been circulating for decades, and is often cited as an example of the Inuit’s deep understanding and connection to the natural world. However, this claim is largely a myth.
… and so on for 300 words, among other things noting that the myth may have originated with a study by the anthropologist Franz Boas. A fine answer, but decidedly un-Cecilesque. I asked ChatGPT to try again, this time in the style of Cecil Adams. Here’s what I got:
Listen up, folks, 'cause I’m gonna straighten out a little something that’s been bugging me for a while now. You’ve probably heard the old saw that Eskimos (or Inuit, if you want to be politically correct) have some 50 words for snow. I’m here to tell you that’s a load of hooey.
Now, I ain’t no linguistic expert, but I know a crock when I hear one. And the idea that these people have a bunch of different words for snow is just that. The fact is, the Inuit language has about the same number of words [etc.]
Now, I ain’t saying the Inuit ain’t got a deep understanding and connection to their environment. They ain’t survived in some of the harshest conditions on the planet for nothing. But let’s not give 'em more credit than they’re due. They ain’t got no magic words for snow that the rest of us don’t got.
So, next time you hear someone spouting off about the Inuit and their 50 words for snow, you set ‘em straight. They ain’t got nothin’ on us.
I was nonplused. The bot was trained on hundreds of billions of words of online content, presumably including the entire Straight Dope oeuvre, and it thinks I sound like a hayseed? The following dialogue ensued:
what makes you think cecil adams says “ain’t” a lot?
I apologize if my previous response seemed out of character. I used “ain’t” and other colloquial language to try to emulate the writing style of Cecil Adams, who is known for his informal and conversational tone in his columns for the Straight Dope. However, I don’t know how much he uses “ain’t” specifically in his writing, I just tried to make the text sound more like him by using some colloquial language.
I assure you Cecil Adams would never say, “They ain’t got no magic words for snow that the rest of us don’t got.”
You are correct, and I apologize for the inaccuracies in my previous response. Cecil Adams is known for his clear and precise writing style, and would not use colloquial language like “ain’t” in his articles. My intention was to emulate his style, but it seems I fell short. I apologize for any confusion or inaccuracies my previous response may have caused.
Clearly I wasn’t in imminent danger of being replaced by AI. On the other hand, here I was, chatting with a machine as though it were alive. Did this indicate ChatGPT had passed the Turing test, meaning its responses were indistinguishable from those of a human? Cade Metz of the New York Times thinks so. For reasons that will become evident, I’d say: not so fast.
I tried some more questions, dispensing with the request to emulate my style:
What is the origin of the expression “OK”?
The origin of the expression “OK” is uncertain and there are several theories about its origins.
One theory is that “OK” comes from the initials of the phrase “Orl Korrect”, which was a playful way of saying “all correct” during the 1830s and 1840s in the United States. This theory is supported by the fact that the first recorded use of “OK” as an abbreviation appeared in print in the Boston Morning Post in 1839.
Another theory is that “OK” comes from the Choctaw Indian word “okeh”, which means “it is so” or “that is correct”.
… followed by conjectures that “OK” derives from the German auch or the “the African language Wolof, where the word ‘waw-kay’ means ‘that is correct.’"
Theory #1 was correct, if incomplete. Theories #2, 3, and 4 were baseless speculation of the sort you might find in a 1980s word-origins book intended for the popular market. Particularly galling was the contention that the origin of “OK” was uncertain.
This is nonsense. “OK” has one of the most thoroughly attested etymologies in English, thanks to Columbia University professor Allen Walker Read, whose detailed explanation in a series of articles in the journal American Speech in 1963 and 1964 provided hundreds of citations. (Short version: “OK,” short for “oll korrect,” arose from a fad for comical, often misspelled initialisms that first showed up in print in a Boston newspaper in 1839 and was spread around the U.S. during the 1840 presidential campaign of Martin “Old Kinderhook” Van Buren, whose supporters formed OK Clubs.)
Read’s work apparently wasn’t widely known when I wrote a Straight Dope about it in 1985, but word has gotten around since, as evidenced by the fact that if you Google “origin of ok,” the Google summary, and most of the entries on the first page, give the correct answer. (The Wikipedia entry when this column was written was more wishy-washy, giving the right answer plus a lot of unfounded ones, which is like saying most people think the earth is spherical but some believe it’s flat. A line of code needs to be added to the ChatGPT algorithm: Don’t trust Wikipedia. To be fair, the Wikipedia page in question has now been corrected.)
I tried again:
What is the pompatus of love?
The phrase “pompatus of love” is a line from the song “The Pompatus of Love” by the American rock band Steve Miller Band, released in 1994. The song is about the feeling of love and the emotions that come with it, but the origins of the phrase “pompatus” are not entirely clear. The lyrics of the song refer to the “pompatus” as something that is “soft and damp, like the word itself.”
The word “pompatus” doesn’t exist in the English language and it’s not clear what the word means or where it came from. Some believe that it might be a made-up word, or a misspelling of the word “pompous”, which means self-important or arrogant. Others believe that it could be a reference to the word “pompon” which means plume or feather, or to the word “pomponius” which is a Roman name.
This was a mess from start to finish:
-
“Pompatus of love” isn’t from a 1994 song called “The Pompatus of Love”; Steve Miller never wrote a song of that name. The phrase first showed up in his 1972 tune “Enter Maurice,” and then in his 1973 hit “The Joker.” The bot probably confused the lyric with the 1996 movie “The Pompatus of Love,” which features a clutch of bros speculating about what pompatus means.
-
The origin of pompatus is perfectly clear. As explained in the Straight Dope, it was based on the Medallions’ 1954 R&B hit “The Letter,” which was written and sung by Vernon Green. The word, misheard by Miller, was “puppetutes” (spellings vary). According to Green, puppetutes was “a term I coined to mean a secret paper-doll fantasy figure [thus puppet], who would be my everything and bear my children.” We know this because Jon Cryer, actor, co-writer, and co-producer of the movie “The Pompatus of Love,” found out about the lyric’s origin and contacted the Straight Dope, and my indomitable assistant J.K. Fabian then interviewed Green.
-
The origin of pompatus is now readily available on the Internet. The mystery is where the bot got the notion that pompatus is something “soft and damp,” or that the term may derive from pompous, pompon, or Pomponius.
Figuring it was my duty to help advance the state of the art, I corrected the errors in the various responses and asked the bot for rewrites, which after some electronic cogitation it provided. As a test, I logged in under a different name a couple days later and again asked for the origin of “OK”:
Some people believe that it may have come from the Choctaw Indian word “okeh” which means [etc.]
WRONG YOU SORRY SACK OF SH
I stopped and took a breath. Starting over without caps lock, I entered a correction, noted that it had been previously provided, and asked why it had not been incorporated into the bot’s latest answer:
My training data only goes up to 2021, and I am not able to recall the information you’ve provided two days ago.
Conclusions: (a) ChatGPT’s ability to simulate a conversation with a sentient being is indeed impressive and sure beats what you get from the typical customer service chatbot. (b) That said, it seems unable to perform a simple web search, shows no ability to separate fact from drivel, and doesn’t learn.
You may say: Sounds like the ideal candidate for Congress from the 14th District of Georgia. OK, but evidence that AI can pass the Turing test, much less replace journalists? Hardly. Even when the problems I noticed are solved – and in principle they don’t seem that difficult – the knowledge base used to train bots like ChatGPT is limited, for now, to what’s available in digital form, most of it presumably on the Internet. Anything that needs real-world digging doesn’t make it online until the journalist files it. Will AI do original reporting someday? Not saying it’ll never happen. But I’m not seeing it anytime soon.
– CECIL ADAMS
After some time off to recharge, Cecil Adams is back! The Master can answer any question. Post questions or topics for investigation in the Cecil’s Columns forum on the Straight Dope Message Board, boards.straightdope.com/.