Europeans were not used to swords that were two-handed and yet were used with such speed. Most smaller edged swords, such as cutlasses and sabres, were used one-handed. Two-handed swords were generally great, heavy things that relied nearly as much on sheer weight as cutting power. The Japanese, on the other hand, were using what was essentially a form of low steel (or at least a rough carbon-iron alloy), which allowed their blades to hold a keener edge. Consequently, katana were lighter and sharper than European two-handed swords, allowing them to be swept around in a very tight, quick arc. Two-handed power behind an exceptionally sharp blade, with the speed of the deftest rapier.
James Clavell is the author of Shogun (amongst other books), and although his characterization of Japanese culture is often derided by Japanese historians, he does get a lot of the sense of history right. He describes one particular scene in which English and Dutch sailors are horrified at the ease with which a samurai draws his katana and decapitates a peasant in the blink of an eye. No European could have matched that feat. (Of course, a musket ball at 10 yards is just as deadly and marginally safer than going toe-to-toe with someone who is perhaps at least as well trained as you.)