Public key encryption is not in any way stronger than symmetrical encryption, it is conceptually more vulnerable. What public key encryption does is solve the problem of key management and distribution which can be a very weak point in symmetrical systems.
Suppose you have a large organization with hundreds or thousands of offices or posts around the world. You need to have couriers constantly traveling carrying and distributing cryptographic keys. Remember the guys traveling handcuffed to a locked briefcase. This is expensive and very vulnerable.
On the other hand traditional, symmetric encryption is one step safer if you do not face the problem of key transmission. If the purpose is to secure stored information or if the keys can be distributed safely then you can use symmetrical encryption. Suppose an employee and his boss are together in the office and the employee will be traveling. They can securely agree on a key they will use to communicate while he is traveling. No need for public key encryption.
Through cryptanalysis of the algorithm. In the case of Enigma, the key flaw was that it would never encode a letter as itself. E.g. if you encountered the string AHRFHIK in a piece of ciphertext, you knew that the first letter of the plaintext could not be an A, the second letter could not be an H, and so on. Together with some more minor flaws, plus the fact that they had some frightfully small mathematicians on their side, the Allies were able to build a special-purpose computer which could use this fact to figure out the encryption key, given enough ciphertext, without actually having to brute-force all possible keys.
Modern crypto algorithms are a lot stronger than that, of course. For mainstream symmetric algorithms such as AES-128, there is essentially no known method of finding the key which is significantly faster than brute-force. Some academic papers have been published on possible attacks, but usually they either work only for a weakened version of the algorithm, or they are completely unrealistic, e.g. you need many petabytes of ciphertext encrypted with the same key, and even then the attack only gives you a tenfold or hundredfold advantage over plain old brute-force.
Despite the fact that they did not have access to public-key cryptography, the Germans usually did a good job of keeping their codebooks secret. So that fact is not particularly relevant to the story of how the Enigma was cracked.
They also used all sorts of dirty tricks to get a “crib”. They’d have the RAF mine a harbour at a known time, relying on the German harbourmaster to dutifully send an encrypted message relaying the information that the harbour had been mined. Of course, this had a high chance of including the date, time, grid co-ordinates and so on, all information they could use to try to work out a way in. Further, a lot of Enigma encoded messages began with the German for “To”. One German operator was also asked to send a test message, so he kept the “L” key pressed down and transmitted it. Of course, Enigma could not map “L” to “L” so the lack of any “L” in the resulting ciphertext gave Bletchley a clue as to what had happened. Other times they got a way in by the German Enigma machine operators mistakenly reusing keys. In other cases, particular regulations e.g. in the German Navy regarding Enigma settings weakened the cipher further still.
The cryptanalysis of Tunny also relied on similar operation shortcomings. Bletchley Park cracked Tunny, a much more complex cipher than Enigma, without ever laying eyes on a machine. They deduced what form the machine took merely by being in possession of two similar messages encrypted mistakenly with the same key and thinking hard about how it could have been produced, before building a replica themselves. For all the emphasis on Enigma, it was cracking Tunny that was truly amazing.
Of course, with modern ciphers, even getting lots of ‘cribs’ won’t help you. Even if you had a thousand files all encrypted with the same AES-128 key, and you already knew the corresponding plaintext for 999 of them, it is still highly unlikely that even the world’s best cryptographers would be able to determine the key and/or decrypt file #1000.
The conceptual drawback of public-key encryption schemes compared to symmetric-key is that it is theoretically possible to completely deduce the private key (and hence completely obliterate the security) using no more information than the public key. However, for any sane key length, this is so difficult that it might as well be impossible. It is, in practice, much, much easier to compromise the transmission of a symmetric key than it is to crack a public key.
If you intend to keep your encryption algorithm secret, then you need a staff of top notch experts to test it. If you make it public you can get an even higher level of testing almost free. You risk vulnerabilities being made public but it is worse to have vulnerabilities you don’t know about. As others have indicated, secure cryptography never relies on keeping the algorithm secret.
A couple if other things that helped with cracking enigma: one operating manual suggested adding an arbitrary nonsense word to the beginning of messages, “sonnenschein” ( sunshine) was given as an example. As a result, many messages ended up starting with the word sonnenschein. Many messages also ended with heilxhitler. Because of the “no letter is ever itself” flaw it was possible to quickly check the messages to see if these were possibly used.
Another vulnerability was that the whole fleet was using the same daily key book, so you only had to crack one message to get a full day’s worth of traffic for the whole fleet. If the subs had used individual keys, then the would have had to identify themselves in plain text, and just knowing which sub was transmitting could be important info.