I cant deny their musicianship.
But good lord, on “Hold the Line” the lead singer looks like Ron Jeremy, and on “Africa” and “Roseanna” the band looks like a glorified cast from “Revenge of the Nerds”.
Im hard pressed to think of a “nerdier” or “dorkier” band than Toto.
I know some are going to say Weezer but they at least intended for the “nerd” look.
Well, it’s peak 1980 or so, I’ll give you that.
Toto is what happens when some pretty good musicians come together as professionals to make an album. Kind of like the Eagles or Boston. There’s not a lot of flash or charisma but, because most of them were session musicians before they hit, they were enormous professionals working in music.
It’s kind of like the Garfield comic. It’s someone who knows what they’re doing setting out to write some hit songs and make a living at it. There’s nothing wrong with that. Not at all. But it lacks a lot of the ‘rock star’ myth so many people seem to expect.
I take it you’ve never seen this:
REM
‘… the band looks like a glorified cast from “Revenge of the Nerds”’
Devo
Really seems like DEVO wins this without any real competition.
Whip It
Other contenders for runner-up:
The Buggles
They may be Giants
Weezer
Mike Flowers Pops - Light my fire
Wonderwall
Both members of Steely Dan look like accountants and for the most part always have.
I agree with the mentions of Devo and They Might Be Giants – though, to be fair, both bands intentionally cultivated a vibe (weird in the case of Devo, nerdy in the case of TMBG) in their look and their music.
For a successful band that was unintentionally / unironically dorky, I nominate Rush. 
FWIW, I’ll point out that, in 1978, when they recorded “Hold the Line” (and, when they presumably recorded the video you’re referring to), singer Bobby Kimball and guitarist Steve Lukather were both only 21 years old. They were babies! 
One last nominee – not as successful as Toto or Devo or Weezer, but he did have several top 40 hits:
He totally had the pencil-necked geek vibe down.
Kraftwerk
Sonic Youth
I just watched the music video. Yea, nerdy. But the thing I noticed most was the globe of the Earth spinning backwards. 
You sure about that?
I forgot about them (probably on purpose). I wonder if Art of Noise was successful enough to qualify for consideration.
Another one, Laurie Anderson had a dorky phase in the early to mid 80s.
The Proclaimers.
Frank Black (from The Pixies) solo stuff is really nerdy.
And of course Weird Al is 1000% nerd.
Probably disqualified because of his complete lack of succes at the time of the initial recordings of his music in the 70’s, and the questionable nature (and extent) of the success of the re-releases, but unsurpassed in dorkiness: Mark Gormley. Without You - Mark Gormley - YouTube A man whom pop-history inexplicably and unjustly overlooked, and who represents greatness in a non-traditional way. Please note that his demeanor, presentation, dress and power-stance is entirely un-affected and not meant ironically in any way. This is genuine.
Matthew Wilder. Not nearly as successful as Toto, but a heck of a lot dorkier.
The thing about being cool: if it’s obvious that you’re trying, you’re not succeeding.
