The etymology of ‘viking’ is confused and much-debated, but there is some evidence that the word ‘Viking’ refers to the act of sea raiding, and a ‘Vikingr’ was a person who embarked on a sea raiding adventure. Therefore, there is a bit of selection bias at work. As OP noticed, the information comes to us through the accounts of their victims. The question essentially asks: “Are people who engage in piracy really that bad?” Yes, sea-raiders and pirates really are that bad, but not every member of the population was engaged in piracy.
If we limit ourselves to only discussing the Northmen who actually engaged in raiding, they were unequivocally nasty people. You have to have balls to get in a tiny boat, sail the North Atlantic, find some Christians, and plunder the $#!* out of them. For the people engaged in this practice, bloodshed, slavery, theft, and extortion were the order of the day. And keep in mind, these people were brutal even by the standards of eighth-century Europe, which was… Um… Not a nice place.
Not all Northmen were raiders. As in any other society, the portion of people who actually go out and fight must be supported by a population of farmers, tailors, shipwrights, and smiths (aka 'normal people). We know with great certainty that the Scandinavians were enthusiastic merchants and colonizers who often intermarried with native populations. Scandinavian colonists founded settlements all over Europe and assimilated with the local culture or created hybridized cultures (such as the Normans, who were Gallic Scandinavians).
Setting aside the question of raiding and the accounts of their victims, we must overcome two indisputable facts about Scandinavia: (1) The region was poor in resources such as iron, and (B) it was cold as $#!*. This suggests to me that the “average” (eg non-pirate) Northman probably had a lot in common with the average peasant anywhere else in Europe. That is to say: they spent most of their lives engaged in subsistence farming, sitting around being bored out of their minds as they tried to survive long, dark winters without succumbing to famine. They probably spent most of their time praying for short winters and full bellies.
The social organization of the jarls tells us a bit about how their patronage systems worked. The chieftain ‘jarl,’ was also referred to by the kenning ‘ring-giver.’ Wealthy men were expected to share their wealth with their people, and in exchange the people provided him with their food, services, and loyalty. The easiest way to get rich in those days was to raid some Christians. Therefore, it’s my conclusion that the pirate ‘Vikings’ were probably the metaphorical “1%” of the Scandinavian world, in that they engaged in the fighting and the raiding and brought back their plunder to buy food and services from the (again, metaphorical) 99% who spent their lives farming and fishing and doing other non-violent things.