Future of General Questions in the AI age

I tend to spend my Factual Questions (née General Questions) bandwidth on attempting to provide accurate yet hopefully accessible technical information. As an aside, I do wish there was more back-and-forth there since it is impossible to pitch the level of a post to everyone all at once, and iteration would be both helpful and enjoyable. But either way, I’m not worried about being “out of a job” anytime soon, as shown below.

We can see how ChatGPT fares on the most recent thread I’ve engaged in. Putting in the original post essentially verbatim:

Imaginary Time? Help me process this

First graf okay. Second graf incorrect twice. “Negative or complex” is just wrong definitionally, and – very critically – the result in this paper and the phenomenon in general is not “non-classical”. The experiments being reported are classical systems.

Incorrect twice here. This experiment did not use a metamaterial. ChatGPT hallucinated that based on the fact that many complex time delay papers do relate to systems that involve metamaterials. Some years back there was a big pop sci craze over light being “brought to a standstill” using metamaterials, for instance. But that is not the situation here. Additionally, the phrase “they determined that the pulse emerged earlier than expected—or with a ‘delay’ that is effectively imaginary” is just broken. The measurement wasn’t about early or late, and even if it was, that wouldn’t mean “imaginary”. It fundamentally was about frequency shifts.

This is largely word salad and doesn’t infer what the OP is asking about. It sounds nice, but it is vacuous. The sentence, “[A]nalogous concepts in cosmology help model the very early universe or quantum gravity, though that is far more theoretical,” is worse than empty because it implies some deep connection when the underlying interest in the article is only that (1) this approach to wave math is helpful and (2) that it is realizable in a microwave waveguide setting. Quantum gravity!? And then: “[T]hey deepen our understanding of wave physics, quantum systems, and possibly advanced technologies.” Same issue: this sort of grandstanding sounds like what an under-researched eighth-grade school report would say.

Again, not a quantum measurement here. The response also has yet to note anything about the connection between the imaginary part of the time delay relating to frequency shifts, which is a key take-away.

More vacuous filler. Especially egregious is “refine our understanding of time, causality, and the early universe.” This work has nothing to do with that, and that reply actually reinforces the incorrect interpretation of the work (i.e., that it has anything to do with time as a spacetime coordinate measure.)

This matches very much my experience with ChatGPT. It all looks pefectly reasonable, but it’s wrong in many places and is just sort of “off” in others, but these issues would be 100% unnoticed by a lay reader. The ChatGPT answer would provide plenty of good feels without any backing.