Probably the best starting point for these questions is the wiki on Pressurized Water Reactors.
There are a few key points mentioned there. Look at the Advantages and Disadvantages sections. One of the advantages is the small size. Another is the inherent stability (from the negative temperature coefficient of reactivity), which is a form of passive nuclear safety.
Yet another neat feature of PWR design, caused by that same negative temperature coefficient, is that their power follows steam demand.. In other words, when the throttles are opened, the reactor power automatically increases, when the throttles are closed, the reactor power automatically drops. This is a result of a chain of events: throttle opens -> pressure drops in steam generator -> temperature of boiler water drops -> reactor coolant is cooler coming out of steam generator -> cooler coolant is more dense -> more dense coolant is better moderator -> better moderator makes reaction increase. And the whole thing works in reverse when power generated is greater than power consumed.
You can imagine that these advantages are well suited to military use. Big civilian power plants don’t have great swings in power demand, but a naval vessel in combat must be able to start and stop on a dime and go to all power levels in between.
The disadvantages listed in the wiki mostly point to cost. The military can afford to spend more money to have higher quality steel alloys and better fuel. Civilian plants must turn a buck, so they opt for other designs.
The fact that they run on purified water as coolant is an added bonus. The Navy made one single vessel with a liquid-metal cooled reactor. Here is how it worked out:
[QUOTE= Frank von Hippel et al (February 2010). Fast Breeder Reactor Programs: History and Status. International Panel on Fissile Materials. pp. 90–91.]
Although makeshift repairs permitted the Seawolf to complete her initial sea trials on reduced power in February 1957, Rickover had already decided to abandon the sodium-cooled reactor. Early in November 1956, he informed the Commission that he would take steps toward replacing the reactor in the Seawolf with a water-cooled plant similar to that in the Nautilus. The leaks in the Seawolf steam plant were an important factor in the decision but even more persuasive were the inherent limitations in sodium-cooled systems. In Rickover’s words they were "expensive to build, complex to operate, susceptible to prolong shutdown as a result of even minor malfunctions, and difficult and time-consuming to repair.
[/quote]
(bolding mine)
Hyman Rickover was the father of the nuclear navy, so this was a decisive moment.