The inverted spectrum is a common argument. Philosophers of Mind often propose dilemmas of this nature, expending every effort to tease a notional “what it feels like” element away from the physical.
I’m afraid I find it improbable that they ever will particularly convincingly. Yellow is E-M radiation of wavelength between ~565 and 590 nm. Alhue must also have a characteristic wavelength, even if it is off the visible scale: 300 nm (“ultra violet”), for example. There are various means by which a person can ‘rank’ their perception of a given colour, such as colour space or by simply composing a matching colour on a computer application.
So, what happens when John is shown an object reflecting 570 nm light, ie. something “yellow” and asked to place it on a colour space or match it in Microsoft Paint? If all the aliens did was effectively write a bit of “brain code” such that
IF 565nm<incoming_wavelength<590nm THEN incoming_wavelength=300 nm;
(ie. all yellows become, say, ultra-violet) he would simply point to the yellow region of the colour space and compose a yellow Paint colour, both of which his brain-code was continually “mixing down” to UV rather like a Beat Frequency Oscillator in an electric radio. However (assuming John can now see actual UV just as easily as “mixed-down yellow” UV) the researchers would presumably find out pretty quickly that John could see invisible things such as the security markings on their equipment or the patterns on certain flowers only usually visible to bees (who can see in UV) - there is clearly something other than simply “what it feels like” that has changed here, and measurably.
So, the experiential part of the experiment appears no more paradoxical than building a radio. But, as others have asked, what of John’s memory here? His memory comprises stored images formed when 570 nm was not “mixed down”. Quite apart from the apparent impossibility of altering a memory formed with a given wavelength so that it is remembered with a replacement wavelength, is there any noticeable difference between what John remembers and what he sees today? Again it would seem that, whatever the characteristic wavelength of Alhue was, he would likely remark on the fact that he can “see in” that wavelength now but cannot remember being able to do so before.
All of these paradoxes, from “Mary in the black & white room” to “what it’s like to be a bat” to the “Chinese room” appear to take some essential feature of explaining qualia and either discard it completely or alter it in some impossible manner before presenting us with a “Ha! Try explaining qualia now, smarty pants.” I believe (but cannot falsifiably demonstrate yet - that it the challenge of 21st Century neurophysics) that all qualia can be explained with reference to physical sensory input, received by some physical apparatus, “encrypting” different levels of physical memory which are subsequently “reactivated” (ie. recalled). The OP’s paradox appears to founder on the simple impossibility of ‘swapping’ a specific element in an entire memory while leaving the rest the same.
So, more generally, I do not find the inverted spectrum plausible without some drastic physical change accompanying it. In this respect I’m with Daniel Dennet.