In a past life, I spent three years answering variations on this question for the Air Force. Modern boosters are between 95% and 99% reliable, though, which is damned good for something so complicated. I don’t reject your premise, but I think the streak of Titan IV failures will eventually be tracked back to a root cause, and the other failures are unhappy coincidences exacerbated by the US Space Industry’s increasing reliance on PowerPoint engineering. If you don’t want to stick around for my professional (but niche) opinions, about why getting a booster up is so hard, you can get a broader view of the reliability issue from here showing statistics about what systems have failed and when. But one quotation from your OP really rang true for me: “Launching rockets is inherently risky and unforgiving, they say.” Below find my own explanations of exactly how unforgiving a discipline rocket science is, and marvel at how we were ever able to get to the moon.
Solid Boosters
Modern solid motors are fabricated by pouring a thick rubbery mixture into a steel or composite case, curing and drying the mixture, and then attaching a nozzle to the motor case. The mixture’s burn rate has an exponential relationship with (among other things) temperature and pressure, so much effort is spent keeping the surface area inside the burning booster constant and high. Burning from the back end to the front (like a cigarette or candle) is easy but inefficient, so these systems have a star-shaped hole through their centerline that burns and keeps the surface area nearly constant as it erodes away; corners in this star shape cause stress and strain that can aggravate failure modes. The “modern” propellant mixture has proven extremely reliable (better than 99%), but there are still flaws. Poor handling practices can cause cracks in the rubbery walls; the accompanying increase in surface area creates a pressure spike and can burst the motor case. Inspecting 1m-diameter (or larger!) cylinders for these tiny cracks is time-consuming, expensive, and imperfect. Nozzles are made of carbon composites, steel, and other materials blended or glued together, and imperfections in the manufacturing or bonding process can create small hard-to-detect flaws in the nozzle surface. If burning propellant begins to erode the nozzle near these flaws, you get a hole in the nozzle wall (throwing your booster off-course) or the propellant accretes at the flaw site and plugs up the throat of the nozzle (creating a pressure spike, followed by spontaneous self-disassembly of the motor case). The list goes on from there, but the story is almost always the same: a very very small flaw can cause an otherwise-sound design to fail when confronted with the extreme temperatures and pressures associated with rocket propulsion. It’s a controlled explosion, and any loss of control generally results in an uncontrolled explosion.
Liquid Boosters
Liquid boosters blow me away - I can’t figure out why more of them don’t fail! The turbopumps in a liquid engine feed system spin at tens of thousands of RPMs moving horribly corrosive oxidizers and highly explosive fuels past each other at extreme pressures within tiny pipes; giant thin-walled tanks contain the propellants and become structural elements when pressurized; the fuel is run through a thin-walled jacket around the outisde of the rocket nozzle before it’s injected into the thrust chamber – this preheats the fuel, cools down the nozzle, and creates just a little more excitement.
There are thousands of moving parts and dozens of actuators linked together by a control computer that adjusts flow rates up and down each millisecond or so to ensure the reaction doesn’t get out of control. The G-forces caused by liftoff can make the fuel slosh around inside the tanks, so that the control computer is also steering the rocket to keep it on course. Throughout all of this the system is doing complicated differential equations to determine how it’s going to hit a known point in space at a given velocity, and achieve a positional accuracy given in centimeters and a velocity accuracy given in centimeters per second. (The solids have to do this too, but their whole goal is to simply stop the burning when they have enough velocity). Basically, you’ve got a machine that’s at least an order of magnitude larger than your car, moving at speeds five orders of magnitude higher than your car’s top speed, with timing and precision that your car can’t hope to match. Everything is happening so damn fast during a launch – the boost phase on a modern ICBM lasts less than three minutes! Telemetry can help analyze most failures after the fact, but even then it’s rare to be able to say what precisely went wrong, and almost impossible for a rocket’s computerized system to perform the complex system analysis in-flight to correct for trouble. Again, it comes down to a controlled explosion – the rocket wants to fail. It wants to find the quickest way to the lowest energy state possible. The guidance computer is responsible for keeping the hot end down, the pointy end up, and herding dozens of variables into their nominal performance envelopes while dealing with the complex ripple effects of changing those variables.
Basically, everyone in the space industry wants boosters to be more reliable, but there’s no single answer to make that happen. The problems are complex and contradictory and we make progress very slowly because new booster or engine designs are very risky endeavors even for huge companies like Lockheed or Boeing.