Can LLMs (large language models) be used to decipher ancient languages?

That, in fact, is exactly why I was using Austen as my trial text, and why I suggested it above. If you’re trying to turn, say, William Gibson into Classical Latin, you’ll crash into a terminological wall, as you note. So I consciously and deliberately tried to avoid that by choosing source texts that (a) minimize superficially modern vocabulary as much as possible while (b) maximizing literary complexity in order to challenge the bots’ ability to “understand” the language and map a Latin equivalent. The point is to expose the computational model, not to struggle with an archaic dictionary.

That being said, there are, of course, subtle conceptual discontinuities between modern English and ancient Latin that also add to the difficulty of translation. For example, there is quite famously no evidence in surviving Roman writing that they had a fully developed concept of internal guilt, i.e. the voice in your head that nags you when you’ve done something wrong. Rather, they had an honor/shame culture, and all their writing expresses forms of guilt in the sense of worry about being caught and exposed and humiliated. Given the subtle psychology involved, I’m unsure how a machine translator would even begin to approach handling that challenge.

But those are beside the point. The key takeaway from my tinkering is to understand that the 'bots do not “translate” in the sense of knowing and applying equivalent semantic meaning, they “translate” by constructing statistical models and choosing the words and phrases that are most likely to represent correspondent expression.

I think with Linear B, there was at least the context that it was believed to encode some form of Greek language - so whilst there wasn’t a convenient object like the Rosetta Stone, there was at least a notion of what direction the translation needed to head toward; this makes it possible to embed a sort of empiricism into the process of decipherment where conjecture can be tested.

Linear B wasn’t in any meaningful sense a “successor” to Linear A, and it also wasn’t a language. The deciphering was that it turned out it was just Greek, with letters that look different.

I think that’s pretty much what ChatGPT does in English right now. It is manipulating symbols. It does not understand what it’s saying in any meaningful sense. It is not a synthetic human.

Can you explain what you mean by this? I’m “natively bilingual” (I think, at what age do you need to be bilingual for this to be the case?) and I’m not sure what you describe matches my experience.

But isn’t that because Linear B is the script for m Linear A used for a language we already know (Mycenaean maybe?), while Linear A is an unknown language?

Eta: what Chronos said.

I hadn’t heard this claim, but looking it up, I do see it mentioned in a few places. The idea seems to be that internal guilt comes from the idea that God is watching so you should behave even if you won’t be found out, via Judaism and then Christianity?

I get the idea, but wouldn’t the Romans believe their own gods would know if they were being dishonorable? Even if the gods aren’t omniscient, you’d think dishonor would be something they’re specifically watching for?

The Egyptians had the idea of weighing your heart against a feather, which seems consistent with internal guilt? So it can’t be a purely monotheistic idea?

If Roman literature just doesn’t ever bring up the concept, though, that does seem like hood evidence for the claim.

That said, the idea that guilt was invented by Judaism would track with every Jewish mom I know :rofl:

Sure. First, the reason I asked the question was in order to give a better answer to Chronos’s question, because I think the answer would be better explained differently depending on their experience.

One of the bilingual people I work with, the most fluent in both languages, has expressed this difficulty translating on multiple occasions. I can see that he has no difficulty in code-switching, but when he translates, I’ll often correct something (nuance or word choice), and he’ll agree that my translation is better. More or less, he seems to find that slowing down to translate throws him. His understanding is quick and fluent, but reading language X makes him think in language X, and explaining it in language Y is harder. He can do it just fine, of course, but it is not something he has ever practised, and it isn’t an automatic feature of having both languages.

Another person appears to read the Welsh phrase by phrase, form a picture in his mind, and describe that picture in English. I’m only guessing as to the process, but his translations are always much looser, though, again, he demonstrates clear understanding. He’s less fluent in Welsh, having not lived in a Welsh-speaking community for many decades, but his understanding of both langauges is excellent. His ability to code-switch is much better than his ability to translate. When learners translate, their mistakes are almost entirely due to missing vocabulary or misunderstanding grammar, and quite different from what the native speakers do (even when missing vocabulary).

I, on the other hand, who was monolingual until adulthood, find it much easier to translate than to code-switch. I can sit with a book in Welsh and read it in either language (that is, I can understand without translating, or I can translate at the speed I sight-read—both a bit slower than I’d read English, though). My speed at that is 100% down to practice at translating, rather than an inherent feature of having another language.

The point of all of this for LLMs is that whatever the three of us are doing, it’s not what machines are doing. We seem to have quite individual processes, and while there are elements that are the same (pattern recognition enables creation; there is a databank of vocabulary), the human brain seems to work quite differently.

If three different people do it differently, that certainly implies that the statement “computers do it the same way as all humans” is false. But it makes it seem more plausible, not less, to say that “computers do it in the same way as at least some humans”.

I am not confident in my understanding how how computers do translation. As I understand it, they have an enormous bank of sets of bilingual texts in various languages, and simply pull chunks from that, sometimes triangulating where necessary.

I’ve seen humans who attempt to work with languages like that, but they’re never very successful. Humans have understanding, which machines don’t, and I think that makes a difference to how they process the information.

I think that’s probably a bit like saying that all the human brain is doing is manipulating nerve impulses. ChatGPT is not a synthetic human, but there appears to be a sense in which sufficiently-adept manipulation of language is understanding.

I suppose that a debate hinges on this. I am not a linguist or psychologist, but ChatGPT demonstrates that is possible to know that blue is a color without understanding what color is or what blue looks like. It only knows how people talk about color and blue, not what they are. To have an understanding of meaning requires being able to relate language tokens to objects and actions in the real world, or at least to abstract concepts. A LLM does none of that.

I suppose there is a hypothetical form of AI that is basically an electronic human brain, which can receive the same sensory input and have the same IRL experiences as humans, like Data, and therefore could grasp meaning of language. But we’re nowhere close to that.

As I understand it, that used to be the way that computer translators worked (or at least, a big chunk of it). But now, translators based on LLMs significantly outperform those, and LLMs were never even intended to be translation programs in the first place, and nobody’s entirely sure how an LLM does anything it does, because they’re not so much designed as grown.

A lot of AIs now combine text and image capabilities. You can show an AI a picture of the sky and ask what color it is, and it’ll tell you blue, and you can ask it to make you a picture of a person wearing a blue shirt, and it’ll show you the picture. Now, granted, that’s still only “seeing” image files, not the “real world” directly, but it’s been really easy to make image files from the real world for a long time now.

Right. ChatGPT especially is extremely good at describing photos. I picked a photo from Google images and asked ChatGPT to describe it in detail:

Summary

This photo captures a humorous and whimsical moment in nature. Here’s a detailed description:

Subject: A red squirrel is the central focus of the image. Its body is suspended mid-air in a full split, with each of its hind legs gripping onto a tall flower stalk on either side. The pose is unexpectedly acrobatic and amusing, resembling a gymnast or martial artist mid-split.

Squirrel's Expression: The squirrel has a calm, slightly amused expression, as if it's unfazed by its comical pose. It is holding a small object (perhaps a nut or seed) in its front paws, nibbling on it with a casual demeanor.

Flowers: The two tall stalks the squirrel is hanging from are covered in clusters of pink and purple blossoms. These flowers appear to be lupines, known for their vertical, spire-like arrangement of small blooms. The delicate flowers contrast sharply with the boldness of the squirrel’s pose.

Background: The backdrop is a softly blurred, lush green, indicating a natural outdoor setting, likely a forest or garden. The use of a shallow depth of field draws attention directly to the squirrel and flowers, giving the photo a professional, polished look.

Overall Vibe: The photo is playful and surreal — as if it were staged for a squirrel-themed gymnastics competition. The moment feels perfectly timed and is likely a product of great patience and timing by the photographer.

This is an interesting point but I think it can be argued that your own experience if the colour blue is indirect. Your brain got the information about it third-hand from your eyes, which got it second hand from photons that bounced off the blue object.
I’ve never seen firsthand the process of transcription of DNA inside a cell nucleus, but I’ve had it described to me in various ways and to some extent, it can be said that I understand it. If we have to witness something in person to understand it, 90% of science is dead.

You have an understanding of a model of DNA abstracted for its relevant properties. Yes, you can understand it. But I’m still not convinced that an AI understands it the same way a person does. And I don’t understand it the same way a biophysicist does. We don’t have to witness something in person to understand a model of it for scientific purposes, one that can be used to explain and predict behavior. But knowing that the acceleration due to gravity is 9.8 m/s² is a lot different than having a coconut fall on your head.

Back to color, the color purple is not a spectral color. It occurs in the brain when the eyes detect a combination of red and blue light. People can see it; some animals cannot. The AI can analyze pixels and say, “Well humans see this number and call it “purple” so I guess, yeah, that must be what purple is.” Humans have a subjective experience of color. The AI doesn’t; it just knows that it can map some numbers to what humans understand to be purple. That’s where I think the difference is.

Can a person who was blind from birth understand color? You can explain it, you can tell them that red is a color in a particular frequency range, and if you give them the numbers of a pixel they can tell you whether that color is red. But do they understand the meaning of red? They will never know what a sighted person experiences when seeing red. Neither does an AI.

This gets into philosophical questions like, “What is meaning?”

I don’t imagine it does, but I don’t think human understanding is necessarily the only valid model of a phenomenon that could be described as ‘understanding’.

In another thread, we’ve discussed whether every human has a comparable experience of looking at the same colour; there’s no way to know if we do, but it’s entirely possible we don’t, because there is also mapping in our own brains and it’s not hardwired.

A blind person can at least have an abstract understanding of “red,” in the same way that I, a male person, can have an abstract understanding of “pregnancy,” and get quite a good sense of its full range of literal and metaphorical meanings. We just can’t have an experiential understanding of red / pregnancy, but we can map that abstract understanding onto other things: a blind person is aware of how sound works, if not sight, and a male person is aware of various physical sensations that help contextualize pregnancy. AI, however, doesn’t have an experiential understanding of anything, only an abstract functional “understanding.” I think that’s a key difference.

What about the multimodal AI that can interpret text and images?

I don’t see how that changes the point. AI has never experienced a squirrel or a garden, but it has been taught to recognize images of squirrels and gardens. It probably begins from something like red squirrel standing between 2 lupine flowers with spread… | Flickr, which describes the photo as “red squirrel is standing between 2 lupine flowers with spread legs,” and then extrapolates from there. Nothing experiential about it.