Mind-Reading Machine: Image Authenticity?

Factual Questions
1

So, a new neuroscience breakthrough is currently making the rounds on the web – apparently, a Japanese research team has found a way to read images directly from a test subject’s brain activity. True, so far those are only 10x10 pixel black and white pictures, but it’s quite impressive nonetheless. However, the picture accompanying the articles on this discovery – for instance, here – seems, and I’m sorry for parroting that old internet battle cry, fake: both "n"s in the word ‘neuron’ are exactly the same, and, since the technology is obviously not exact as is evidenced by the level of noise in the reproduction, this strikes me as rather suspect – if the variations are random, the two letters should be distinct in their representation; if the variations weren’t random, it would be easy to subtract them, giving a much improved (exact, even) representation.

So, my question is, is this claim legit? If so, why the embellishment? (Although I seem to be unable to find the source of the image, so it might just be some visualisation done for an article that is now being passed around as genuine – can somebody perhaps translate the Japanese?)
Also, what are the ramifications of such a technology and possible future improvements on it (some people already call it a ‘dream reader’, however that seems a bit early to tell)?

2

If I read the article right, so far what they claim to have done is to “read” images in the visual cortex produced by actually seeing something, not by just imagining it. I don’t know if one will necessarily lead to the other. (When we make an image in our “mind’s eye”, is it generated in the same area of the brain as are images that come from our eyes?)

ETA: There must be a qualitative difference in the brain between images from our eyes, and images created by thought, otherwise we wouldn’t ever know if we were seeing something real.

If the article is accurate, all they’re doing so far is using a human eye and brain as a really crappy camera.

3

Half Man Half Wit, that’s a good point. I can’t answer your question but I can speculate, here goes:

They trained their software prior to the “neuron” test with 400 random 10x10 images which implies they used a neural network to map human brain fmri readings in the areas they were taking them, to output image values. It’s possible this mapping (which is then used to interpret the new images) reduced the detail enough that the 2 N’s came out the same. There are 2 places where information is being lost, the areas in the brain are being grouped into 3d voxels, and the neural network mapping is providing a “closest match” between current inputs and previously trained inputs.

4

Regarding reading dreams or mind reading in general: They were able to interpret the processing of visual stimulus because they ran 400 tests on 10x10 images prior to the actual interpretation so they knew how to interpret brain activity. To do the same thing for dreams or thoughts they would need to run tests to determine how your particular brain stores any of an almost infinite number of thoughts. Not practical.

5

I haven’t read on it in any detail, but is it possible that they didn’t actually show the test subject the word “neuron” as such, but, rather, just showed the subject each letter of the alphabet, and then later, for a cute graphic, simply displayed the particular letters which make up the word “neuron” (in so doing, necessarily using the same ‘n’ twice)?

6

I’ve just read the actual article, and here is in more detail what they actually did:

They did indeed use machine learning to reconstruct visual images from brain imaging data.

Subjects were shown a training set of 440 images, each shown for 6 seconds, with a 6 second break between them (that comes up to 88 minutes to train the system.) In the reconstruction step, they were shown random images chosen from a set of five letters and five geometric patterns. Each image was shown for 12 seconds. Using an fMRI, they imaged brain activity in the V1 and V2 areas of the brain. They also show that it’s possible to perform reconstruction by showing the images for only 2 seconds.

The machine learning algorithm is not a neural net but rather a technique called “sparce logistic regression”.

About the ‘n’ being the same in both images: each image in the “reconstruction” stage was shown six or eight times, in random order. The images that you see in the press are an average of the eight reconstructed images. In the paper you can see all the reconstructed images and individually, they’re somewhat less neat.

The higher up you go in the brain’s visual chain of processing, the less reliable the reconstruction becomes. They found that using imaging of V1 to V4 did not yield better results than when they used only V1 and V2, with V1 contributing the most to the machine learning model.

From my, admitedly limited, knowledge, it would be difficult to use their approach as-is to try to reconstruct dreams. Accurate reconstruction depends on V1 and V2. This is where the signals coming from your eyes are first processed. At this stage, only very low-level processing is done by the brain. Importantly, at this stage, the processing is still very much spatial. However, when you go to later processing areas such as V3 and V4, much of that spatial information is discarded in favour of more abstract concepts. This is why they had good results with imaging of V1 and V2, but not with V3 and V4. As it happens, V1 and V2 probably do not play a big role in dreaming. From what I’ve read, people with lesions to these areas can be blind in “real life” but will nevertheless “see” in their dreams. However, people with lesions in V4 will be unable to see, for instance, faces in real life and in dreams.

7

Thanks for your answers, everybody!

Ah, that clears that up, then. Thanks!

That’s very enlightening; however, it seems like the method as such would still be applicable to ‘higher-level’ processing stages, provided you could find a way to map the abstractions an individual brain uses to the visual inputs (be they from the eyes or generated by dreams/memories), right? I think that question is equivalent to asking whether or not a given (data-)structure in the mind has a corresponding pattern of brain activity; from there, it would only be a matter of figuring out which activity pattern corresponds to what mental image, even though that may be impractical, though if there’s some sort of regularity in the way the patterns match to the images that workload would be greatly reduced (i.e. you’d only have to learn the basis vectors of the space used to represent (certain kinds of) images and the way the brain does linear combinations with them, and you could predict the brain activity pattern a certain image would produce). But that’s just lard-assed speculation (because I’m right now too lazy to read up on how the brain actually does these things).

However, just to make that clear, I’m not worried about the device becoming a mind-reading machine, it seems that the level of cooperation necessary by the subject would render such a use highly impractical. But I do believe that it might, for instance, help people with locked-in syndrome to communicate with their environment, or even allow for critical examination of so-called ‘facilitated communication’.

8

It depends on how narrowly you want to define “their method”. As they make clear in the paper, there have been several previous successful attempts at retrieving visual information from fMRI imagery. For example, in one experiment the system was able to tell if the subject was looking at a particular image from a given set. What is special about the current research is that they are able to reconstruct any arbitrary 10x10 pixel image. The algorithm they employ, however, depends on there being some sort of spatial correlation in the data they’re looking at. The precise mathematics they used won’t work if the spatial information is lost. Now that doesn’t mean that one day you won’t be able to do something similar with data from high-level areas, it’s just not a trivial step to make.

The problem is that they were looking at neurons that react to specific areas of the visual field. However, when you move way up on the chain of processing, you’re not looking at groups of neurons that fire when there’s a bright spot in a specific point in your visual field – you’re looking at neurons that fire when you see Aunt Elma’s face. While it may be possible to tell if someone is dreaming of a particular person, you would somehow need to make a huge dictionary of every abstract idea that could possibly find its way in a dream in order to “record” them.

Just so that people don’t get their tin-foil hats out, here’s a picture of the machine they used to get their imaging: