Aside from the fact that you are comparing tactical rockets, which are smaller, cheaper, easier to manufacture, generally lower pressure, see smaller loads (and still see failures in the 0.1-0.2% range even in mature operational systems), there are a vast number of failure modes and contributing factors that make rockets–both solid propellant motors and liquid propellant engines–prone to failure.
Large suborbital and space launch rocket vehicles operate at extremes of pressure and temperature, pushing structural materials to stress and thermal limits. They are very, very difficult to test in a “flight-like” manner without actually flying them; even though ground “static fire” tests are performed on motors and engines, these generally cannot replicate all the inertial, thermal, and atmospheric conditions seen in flight, and so analysis and “best guestimates” have to be done to assess survivability in flight. Most testing is done at a component and subsystem or subscale testing, which may not replicate failure modes seen in an actual flight; for instance, POGO oscillation (a type of destructive resonance seen in liquid propellant plumbing), and flight enhanced “head end” erosion (accelerated burning of the forward end of the exposed motor grain due to inertial loads in flight) cannot be seen in a ground static fire test and are only experienced during actual flights after much of the design effort is complete. And the problem with flight tests–aside from expense and likelihood of negative program impact in the case of failure–is that there are so many things going on at once, and the limit to instrumentation and telemetry bandwidth often makes it difficult to monitor the comprehensive state of the vehicle.
Other conditions, like cavity resonance combustion (sinusoidal) vibration, turbulent combustion instability, nozzle throat erosion, chamber wall burn-through, bondline separation, o-ring erosion, liquid tank slosh, detached flow in the diverging nozzle, base end heating from plume recirculation, and many other contributory factors to failure modes are highly variable and difficult to determine by analysis or ground testing alone. Despite an extensive testing program and a knowledge base from the Titan development (which had the same problem) the Saturn V S-IC and S-II first and second stages both experienced very serious POGO phenomena, requiring a couple of design cycles. Since such cycles are expensive–often requiring rework or disposal of existing hardware, redesign of tooling, and qualification testing of new hardware–unless they are shown to be catastrophic or detrimental to fundamental capability, they are usually accepted by placarding (reducing the capability from Design Reference Mission specification), ad hoc modification, or my favorite, PowerPoint engineering to waive away known risks.
A prime example of the last is the Titan IVB SRMUs, in which many indications of partial burn-through were noted but dismissed by The Aerospace Corporation (an independent mission assurance contractor who had no vested interest in the profitability or schedule pressure of the vehicle) as not reaching a critical level until the catastrophic failure of Titan 34D-9, which ultimately resulted in the Titan IV being axed. The failure of STS-51-L (Challenger) due to a long-standing problem with o-ring erosion, which was known from STS-2 (but never seen to a critical extent in sea level horizontal static firing) is another well-known example of how testing and analysis doesn’t show you everything, and pressure to meet cost and schedule often results in (intentionally or otherwise) overlooking signs of incipient failure despite the best engineering effort. Even highly mature systems–which for most large rocket vehicles means a few dozen, or at most a couple hundred, launches–still experience anomalies that indicate that all failure modes are not fully anticipated and may result in catastrophe at some unfortunate alignment of the heavens.
Modern orbital and suborbital rocket launch systems are incredibly complex systems that integrate components developed independently by multiple and often competing contractors, pushed to the limits of material capability, often being used in ways that were not intended by design requirements at the outset, and experiencing conditions that cannot be replicated in ground system and subsystem testing. Every launch is something of a “cross fingers and pray to the gods of flight” that not one of hundreds of potential failure modes–including failures due to fully qualified and acceptance tested components that suddenly decide to go on the blink for no good-god-damned reason whatsoever, or a failure because some hung-over technician installed a widget backwards which the quality assurance person didn’t check before he signed off and the idiot design engineer didn’t design to be installed in one-and-only-one orientation–don’t suddenly appear in mid-flight and, in the parlance of O-5 in an failure review board briefing, “fuck us hard in the ass.”
Looking at the flight failures of new vehicles using largely known technology, like the Boeing Delta III or the SpaceX Falcon 1 gives some sense for the complexity of maturing such vehicles into a reliable system. Even when you do experience, diagnose, and fix one critical failure mode, it may mask a dozen or more other possible failures that you will only discover through testing and failure. With tactical rockets that cost only a few tens of thousands of dollars to build and test, you can afford to test the system dozens or hundreds of times before fixing the design. For a system that costs tens or hundreds of millions to build (and more in ground support equipment and range support costs) the amount of testing you can perform before fixing the design is highly limited, and the criticism (deserved or not) for a single failure prevents a thorough flight test regime that stresses the system to determine all potential failure modes.
Stranger