It’s also a social problem. People who write benevolent code tend to do it for a living or for fun and are mainly incentivized by their paycheck or the joy of creating new features for the world. You build something, test it reasonably for bugs, and move on to the next interesting project. Most coders of this sort don’t enjoy pouring over the same few lines of code for a thousand hours in a row, and then doing it again for each and every library and compiler used to build their code, just to hunt for that once-in-a-lifetime security bug. And getting exhaustive third-party audits is both expensive and still no guarantee, in the mathematical proof sense, that the code will be vulnerability-free.
On the other hand, malicious hackers and government entities ARE incentivized to find these rare bugs, because they get a payoff only if they DO find one. So they can dedicate all their time on the one target (or a small selection of targets) looking for even one error that the original programmers made.
It’s the difference between authoring a 1000 page novel and being that OCD guy who found the one punctuation error the editor missed.
Even JPEG is not some trivial thing. It’s not just “hey, display a block of red pixels and then a block of blue”… it has to deal with progressive decoding, interlacing, error correction, color spaces, EXIF extensions, and god knows what else. Each one of those features is probably written by a different team and integrated over decades, and a bug in any one of those components is a possible vector for steganographic malware. Hell, it might even actually display as a valid image with error-corrected noise – may or may not look like much to the human eye, but the JPEG decoder doesn’t know any better. It’s technically a “valid” image, except that one bug in that one library from 15 years ago means the next 80 bytes is read too and accidentally executed via a buffer overflow…
TLDR: Shit is hard. Security is boring (to most). It’s easier to fix problems as they’re found rather than to exhaustively audit every single line of every single component of every single program.