OK, trying to avoid to much descent into golden eared lunacy.
First up, you really owe it to yourself to seek out and try a high end headphone setup. You will be gobsmacked. Not necessarily this setup, but any reasonably high end headphones, amp and source. What is referenced isn’t even all that expensive, but is probably well into the land of diminishing returns. Compared to a Sennheiser Orpheus, there is a way to go. And Sennheiser are a serious professional level engineering company.
If you use a USB digital input from your iPhone (which is now possible without much effort - in the past it was not), and you use lossless compression of the audio, you get bit perfect output of the audio, so as good as the original source material. The iPhone ceases to be anything that matters, it could just as well be any other digital source.
If you use the DAC in the iPhone you have a pretty low end DAC, it is built to a price, to a low operating voltage, and to a low power draw. All of these things result in compromises in quality. So to the same issues are found in the little amplifier that is sued to drive the headphones. And then you get to the environment that these chips operate in. It is electrically noisy, really noisy. There are RF signal and computer clocks and signals singing away, and this makes it hard for the audio components to work as well as they otherwise might.
So inside the DAC/Amp box you have a lot of freedom to get things much more right.
First up, you need to clean up the clock of the digital signal. This is one of the most critical things in digital audio, and for many years took on a level of deep mystique. Noise in the clock is known as jitter or phase noise. If you allow the clock that controls the output timing of the digitised signal into the analog domain vary in time, it changes the signal, and not in a happy manner. If the variations were just random noise (AIWN - additive independent white noise) or other purely stochastic noise it wouldn’t matter so much, but the machinations of the digital systems conspire to add all sorts of correlated noise into the clock, and these can and will cause weird distortion artefacts. Not the usual harmonic distortion one is used to in analog systems, but things like aliases of the output frequencies at sum and differences relative to the original. These are not harmonically related to the original, and thus the ear’s masking functions do not remove them from perception nearly as well, and they are peculiarly objectionable.
So the internals of the exteral box can contain a much quieter electrical environment, and will use specialised additional circuitry to attenuate the phase noise in the input clocks. Next there will be a much higher specification DAC chip. It can be fed with much higher voltages and be allosed to draw much higher power levels, and fed very clear stable power, power that is usually very carefully managed with extremely careful routing and decoupling so as to avoid coupling of switching noise into parts of the system where it isn’t wanted. The analog parts of the DAC system can be much higher specification, and use a range of techniques. Critically there is usually a current to voltage converter - DACs usually output current, and this need to be converted to a voltage, at the same time applying filtering to remove alias frequencies of the audio. This isn’t easy to get really right, and usually benefits from high specification op-amps (or when you want to buy an argument, custom discrete components.) Again, careful power supply design and circuit board layout is critical, as you are still working in the RF domain. Indeed IMHO and experience, circuit layout is more critical than the use of supposedly high specification components. Even a seemingly trivial mistake can couple significant amounts of logic switching energy into the wrong path.
Then you get to the amplifier. IMHO there is a lot of snake oil here, as building an essentially perfect amplifier for purpose isn’t difficult anymore. But there are lots of amplifier designs that are tweaked to a particular sound, and some can sound very good with particular headphones. More a musical tuning than HiFi, but can sound lovely. But good headphones do need a reasonable amount of power to drive properly, and your iPhone isn’t going to manage it. Not on the very low power rials and power budget available. A good amplifier design does warrant a carefully designed power supply, with very low noise, and given the circumstances of its use, getting it down to distortion levels that are essentially negligible at any frequency and power level not unreasonable. That requires power levels and voltages not available in any phone.
$1000 for a DAC/amplifier isn’t all that big a number. One of the well known such devices is the Benchmark DAC2. It costs double. There is no snake oil. Just very careful engineering. However, don’t worry, if you want snakeoil, there is plenty out there to empty your wallet.
Personally, I have a pair of Sennheiser HD650, an Apogee Duet I use as a DAC, and a PPA amplifier I built.
