Stump the AI

Miscellaneous and Personal Stuff I Must Share
1

In this thread it was noted how an AI badly got an answer wrong (and I was the one who posted its wrong answer). That wrong answer was a while ago and I tried it again and the answer was spot on. Some claim AI is unreliable enough that it is mostly worthless.

This is me…not saying anyone else said this but I think I am in the ballpark of the skeptics with this (correct me if I am wrong): AI may get 9 of 10 right but a 10% error rate is still too unreliable to be used in a serious manner.

I make no claim AI gives perfect answers. Indeed, we know it does not. But how bad is it? Let’s see!

A few rules:

  1. The question needs to be answerable. If it gave us the answers to The Goldbach conjecture or the Riemann hypothesis which of us gets the Fields Medal? (Me…definitely me)

  2. Kinda along the lines of #1…don’t ask what Busy Beaver 20 is (I think BB5 is the biggest). Maybe it is answerable in principle but not before the heat death of the universe.

  3. If it is a math question (that has an answer) I expect the poster to know the answer and understand the question asked so they can assess how the AI did. I certainly cannot do that.

  4. Philosophy questions are ok but they do not really have an answer so…not sure but go for it…see what happens.

  5. Note that my AI’s database stops in January 2025. So, anything current it will not know. Other AIs may be more recent but this is the one I am using and it told me that’s its limit to recent events.

  6. Coding questions are fine but keep it short. Mainly because I want to post answers here and long lines of code on the forum…not nice (can we do collapsible text on here…really asking, that might help).

  7. If you asked the question please make a post of your assessment of the response. Good, bad, ugly, nasty…let us know.

  8. Catch-all…rules may change. Seeing how it goes. I will certainly note any additions or modifications. Mods, of course, can intervene as they like (they don’t need my permission)

  9. Don’t use this as a way to get me to use my AI for your work. I doubt anyone would…needs to be said.

  10. If you asked the question please post a response of how you think the AI did.

One more thing…I want this to be fun, I am not an AI evangelist at all. I have found it useful but I am wary. Also, I can only do so many queries per day so if I can’t answer everything it is due to limits (and I use it for some work which takes priority).

Let’s try it for a few days and see how it goes. Have at it!

FTR: I am using Claude Opus 4

2

Yep. Under the + of the edit menu it’s called “hide details”. Confusingly, the default header for the result reads “summary”. But you can edit that.

3

Let’s try the latest SF story ID I got.

Here’s the question
Ask the guy who is pretty good at SF Story Identification - #1810 by kaylasdad99?

I’ll be curious what the AI comes up with

4

Here is the AI’s answer:

I’ll search for this intriguing alternate history time travel novel for you.Let me search more specifically for the novel you’re describing with time travel and Bonaventura.Let me search for more details about the ending of the novel where time travel stops working.Let me search for the specific details about Bonaventura in the novel.Based on my research, I can confirm that the novel you’re remembering is “Times Without Number” by John Brunner, not John Varley.

The book’s plot takes place in the years 1988–1989 in a timeline where the Spanish Armada under the command of the Duke of Parma successfully invaded England in 1588. This matches your memory perfectly.

Here are the key details that confirm this is your book:

Author: John Brunner (published in 1962, with a revised edition in 1969)

Plot Elements:

  • Don Miguel Navarro is a man with a most unusual job. As a Licentiate of the Society of Time, he is a time traveling agent for a Spanish Empire that continues to thrive four hundred years after the Armada successfully conquered England in 1588
  • The protagonist is indeed a cop/agent who specializes in time travel crimes
  • It’s not really a novel but more a collection of related stories featuring the same characters - matching your memory of it being cobbled together from short stories

Time Travel Inventor: The inventor is named Carlo Borromeo, not Bonaventura as you remembered. In 1892, an Italian named Carlo Borromeo discovered the secret of time-travel. This is likely the detail you were slightly misremembering.

Setting: This book takes place in the far-flung future year of 1988, and the protagonist does live in “Nuevo York” (spelled various ways in different sources).

The Ending: Multiple reviews confirm the brilliant ending you described. Don Miguel didn’t go back to the day of the battle, he went back further, in a way that would be nearly impossible to figure out where and when he did it, but he changed things so that instead of the Armada being led by the capable Duke of Parma, it is led by an aristocrat with no experience named Medina Sidonia. He shows up in the city of New York, in a world where time travel does not exist. This creates a paradox where the timeline with time travel essentially erases itself.

At one point Don Miguel muses that time travel is inherently unlikely, because once you have it there’s a temptation to make changes, and changes will eventually inevitably lead to a future in which time travel is not invented, like a snake swallowing its own tail.

The novel was originally published as three connected short stories in 1962 in Science Fiction Adventures magazine, then as an Ace Double that same year, and finally as a revised and expanded edition in 1969. It’s considered one of the classic time travel paradox stories, with its ending being particularly praised for facing the implications of time travel “directly and honestly.”

5

Impressive - thanks. I may have to specialize in stories that haven’t been reviewed much :slight_smile:

6

What superhero visited the Cave Of The Peaks — “Where All Things Exist!” — in issue after issue of his comic book?

7

I was surprised too. Not a question I expected at all out of the gate and had no clue how it would do. It is pretty cool…and a little scary.

8

I’m generally pretty happy with Claude.ai and find it useful, if not always accurate. Today I asked it to tell me the probability that a month will have five Fridays. In the first response it assumed this would only happen in 31 day months. I pointed out that this was incorrect. It congratulated me on finding this flaw and did the calculation with both 30 and 31 day months, it said it did not need to worry about 29 day months. I explained that it needed to handle that case. Once again it congratulated me and then proceeded to give the correct answer. Earlier in the discussion it also thought that it depended on what day of the week I chose, e.g. Monday or Friday. I explained that it did not matter and it agreed with me. So, it made multiple mistakes, but with my help it ultimately gave the correct answer.

9

You may have stumped the AI!

I apologize, but I’m having difficulty finding specific information about the “Cave of the Peaks — Where All Things Exist!” through my searches.

This sounds like it could be from a Golden Age or Silver Age comic book series, possibly from Marvel or DC Comics. The concept of a mystical cave “where all things exist” suggests it might be from a series featuring a magic-based or cosmic superhero.

Without being able to find the specific reference, I cannot definitively identify which superhero visited this location issue after issue. Could you provide any additional context, such as:

  • The approximate era (Golden Age, Silver Age, etc.)?
  • The publisher (Marvel, DC, or another)?
  • Any other details about the character or the stories?

This would help me provide a more accurate answer about which superhero frequented the Cave of the Peaks.

10

It is a weird mix of obsequious and patronizing.

11

Era would be 1960s; publisher, ‘other’; the character once went there in search of a cure for a disease.

12

Still stumped the AI! It’s answer after your additional info:

Based on all my searches, I haven’t been able to find any reference to a “Cave of the Peaks — Where All Things Exist!” in relation to Peter Cannon: Thunderbolt or any other 1960s Charlton Comics character. The search results show:

  • Captain Atom was a major 1960s Charlton character but no mention of this cave
  • Peter Cannon: Thunderbolt was another prominent Charlton character from the 1960s, with mystical/Tibetan elements, but again no reference to the Cave of the Peaks

Without being able to find this specific location despite extensive searching across multiple Charlton characters from the 1960s, I cannot definitively identify which superhero visited this cave. The mystical nature of the location (“Where All Things Exist!”) and your mention of searching for a cure for a disease suggests it could be from a more obscure character or series.

If you could provide any additional details such as:

  • The specific publisher if not Charlton
  • Any other plot elements or character names
  • The nature of the disease being cured

That might help narrow down the search to find the correct answer.

13

I tried the five Fridays question with Grok. It did the calculation correctly only for 30 day months. It made mistakes in handling 28, 29, and 31 day months. So it failed badly, worse than Claude, but did a more accurate determination of the probability of a leap year, which Claude assumed to be 25%.

14

Naw, the AI got it without realizing it; the answer is Peter Cannon: Thunderbolt, a character who (a) starred in the comic of the same name, published by Charlton, and (b) made repeated visits to the Cave Of The Peaks in the Himalayas, per the ‘Mystical Tibetan’ elements in his backstory — while explicitly spelling out that it’s “Where All Things Exist!”

15

Are you using Sonnet or Opus? I asked Opus your question…is this correct?

I’ll work through this step-by-step to find the probability that a month will have five Fridays.

Understanding the Problem

For a month to have five Fridays, it must:

  1. Have at least 29 days (since 4 weeks = 28 days would give exactly 4 Fridays)
  2. Start on a day that allows the 29th, 30th, or 31st to fall on a Friday

Step 1: Identify which months can have 5 Fridays

31-day months: January, March, May, July, August, October, December (7 months) 30-day months: April, June, September, November (4 months)
28/29-day month: February (1 month)

Step 2: Determine starting days that yield 5 Fridays

For each month length, let’s see which starting days give us 5 Fridays:

For 31-day months:

  • If the 1st is a Friday: Fridays on 1, 8, 15, 22, 29 ✓ (5 Fridays)
  • If the 1st is a Saturday: Fridays on 7, 14, 21, 28 ✗ (4 Fridays)
  • If the 1st is a Sunday: Fridays on 6, 13, 20, 27 ✗ (4 Fridays)
  • If the 1st is a Monday: Fridays on 5, 12, 19, 26 ✗ (4 Fridays)
  • If the 1st is a Tuesday: Fridays on 4, 11, 18, 25 ✗ (4 Fridays)
  • If the 1st is a Wednesday: Fridays on 3, 10, 17, 24, 31 ✓ (5 Fridays)
  • If the 1st is a Thursday: Fridays on 2, 9, 16, 23, 30 ✓ (5 Fridays)

31-day months have 5 Fridays when starting on: Wednesday, Thursday, or Friday (3 out of 7 days)

For 30-day months:

  • If the 1st is a Friday: Fridays on 1, 8, 15, 22, 29 ✓ (5 Fridays)
  • If the 1st is a Saturday: Fridays on 7, 14, 21, 28 ✗ (4 Fridays)
  • If the 1st is a Sunday: Fridays on 6, 13, 20, 27 ✗ (4 Fridays)
  • If the 1st is a Monday: Fridays on 5, 12, 19, 26 ✗ (4 Fridays)
  • If the 1st is a Tuesday: Fridays on 4, 11, 18, 25 ✗ (4 Fridays)
  • If the 1st is a Wednesday: Fridays on 3, 10, 17, 24 ✗ (4 Fridays)
  • If the 1st is a Thursday: Fridays on 2, 9, 16, 23, 30 ✓ (5 Fridays)

30-day months have 5 Fridays when starting on: Thursday or Friday (2 out of 7 days)

For 29-day February (leap year):

  • If the 1st is a Friday: Fridays on 1, 8, 15, 22, 29 ✓ (5 Fridays)
  • All other starting days give only 4 Fridays

29-day February has 5 Fridays when starting on: Friday only (1 out of 7 days)

For 28-day February (non-leap year):

  • Can never have 5 Fridays (maximum is 4)

Step 3: Calculate probabilities over a 400-year cycle

The 400-year Gregorian calendar cycle is perfect for this calculation because:

  • It contains exactly 146,097 days
  • The cycle repeats exactly (including leap years)
  • It accounts for the leap year rules (every 4 years, except centuries not divisible by 400)

In 400 years:

  • 303 years are non-leap years (with 28-day February)
  • 97 years are leap years (with 29-day February)

Months with 5 Fridays per 400-year cycle:

  1. 31-day months: 7 months × 400 years × (3/7) = 1,200 occurrences
  2. 30-day months: 4 months × 400 years × (2/7) = 457.14… ≈ 457 occurrences
  3. 29-day February: 97 leap years × (1/7) = 13.86… ≈ 14 occurrences
  4. 28-day February: 0 occurrences

Total months with 5 Fridays: 1,200 + 457 + 14 = 1,671

Total months in 400 years: 400 × 12 = 4,800

Final Probability Calculation

Probability = 1,671 / 4,800 = 0.348125

Converting to a fraction: 1,671/4,800 = 557/1,600

Answer

The probability that a randomly selected month will have five Fridays is:

  • 557/1,600 (exact fraction)
  • 0.348125 (exact decimal)
  • ≈ 34.81% (percentage)

This means roughly 1 out of every 3 months has five Fridays, which makes intuitive sense given that most months need to start on one of 2-3 specific days to achieve this.

16

That’s interesting. Right but not knowing it…hmmm

Still a cool result (I think)

17

I just did it and it answered perfectly:
https://x.com/i/grok/share/nsi33fdKag7GMfVYetdgBBrX8

Handled the 400-year Gregorian calendar cycle and gave the answer in the form of a reduced fraction (557/1600).

I used “think” mode, which you really need for any kind of multi-step problem like this.

18

I think Claude did a bit better by going a step further and giving us a percentage. (see my post just above) and then a little bit more explanation of what is happening.

19

Well, it’s the right answer either way. I did the same computation myself to verify, but used a decimal calculator because fractions are annoying to work with.

I asked for a bit more elaboration and it gave several different methods:
https://x.com/i/grok/share/tQG15Yr8UKiBjBol5kEGkwaYX

With a final summary of:

Final Answer (Restated): The probability is 557/1600, or approximately 0.348125, 34.8125%, odds of 557:1043, about 4.18 months per year, or 1671 months in a 400-year cycle.

It’s important to always use the thinking modes for stuff like this. Not sure if that’s the main difference between Opus and Sonnet, but for Grok it’s a checkbox.

20

Opus is the deeper thinker but they never really spell out the differences between it and Sonnet. I am using Opus for all answers here but usually I use Sonnet and use Opus for math and code. Why? No reason…seems the right thing to do to me. A lot of what they are doing is opaque…I need to read more.

Claude also has a research mode which goes super deep and takes 10 minutes or more (and on my one test all my tokens for the day were gone…but it finished my query). Research is not meant for doing math though. Not sure of its value yet since it take a lot longer and I can barely do one a day. But it’s there so…cool.