The line-em-up-and-shoot-em-down style of fighting evolved for a few simple reasons. First, the smoothbore musket was comparatively cheap and easy to mass produce compared to a weapon like the rifle or the crossbow. Second, it was easy to use. Third, when equipped with a bayonet, footsoldiers could be employed as both light infantry (missile combat) and heavy infantry (shock weapons).
But there were problems with the smoothbore musket. It had piss-poor range and accuracy, so the the troops had to be deployed in a formation which maximized firepower–the line. The troops also had to be kept close together because it still took a little bit of time to redeploy from the offensive line to the defensive square, which was how troops defended against cavalry.
If the troops spread out and fought independently, they became skirmishers, which were more annoying than effective and which were also highly vulnerable to cavalry. While it is true that the Americans occasionally put these tactics to good use in the Revolution, once confronted on the battlefield they weren’t shy about lining up just as the Brits and Hessians did. They were also a lot less shy about running like hell when the Brit and Hessian lines closed to killing range, because the Americans were often outclassed in drill and rate of fire.
If a line fired too soon at an advancing line of bayoneted muskets, few hits would be scored and the advancing infantry won a few extra seconds to close the distance to a better range and get off a more effective volley–or charge with their bayonets. So often, one line waited for the other to approach within a range which looks ridiculous by todays standards. By the standards of the 17th-19th Centuries, it wasn’t ridiculous, it was efficient and deadly.
As the range and rate of fire of the standard infantry weapon improved (and the quality of the average soldier improved as well), cavalry became more vulnerable on the battlefield and the standard line/square tactics gave way to skirmishers–spread out troops who operated more independently. By the Franco-Prussian war of 1870 the number of skirmishers deployed often outnumbered the number of troops in line formation.
This sort of fighting, by the way, is far from “civil,” if you look at it in the right way. John Keegan in A History of Warfare points out that tribal groups almost universally practice forms of combat which are far more ceremonial and far less dangerous. As societies evolve into more complex organizations, their warfare also evolves into a system designed to more effectively kill and maim than to merely intimidate. Those gaudily dressed troops of the 1700s may look foo-foo, but at the time they were the most effective killing machine yet devised.
) I think that would have given them a nasty surprise.