This is a topic that I’ve thought about for a while now. I hear about an artist or hear a song by them, but which album do I listen to first? The first, the most recent, or the most popular? If I listen to the most recent or popular, chances are if I go back and listen to their previous albums, they won’t be as good and won’t represent what the band is today.
I’ve tried all of these approaches, and I haven’t found anything optimal. For classic rock, I’ll usually stick with a Greatest Hits album. For active bands, I’ll get their most recent and work backwards, usually.
What do you do? Does it matter whether the artist is new, retired recently, or retired decades ago?
I get the album that has the song I heard and liked. If the whole album is good, I usually start acquiring their catalog in chronological order, unless research tells me to do otherwise.
In general, I think it’s fun to see how a band can evolve over time while still retaining their own signature sound. I was able to listen to the Beatles in chronological order at 9 years old just as a I was starting to buy albums, so I heard them change from a jangly high energy pop band to a really thoughtful and accomplished set of composers and musicians, and heard it all over a year or two just as I was becoming educated enough to appreciate what I was hearing. I still get a kick out of it when I can do that with some new group or artist.
With absolutely no further information about the band, I tend to default to their second and third albums as a representative sample of solid work. Most of the bands I like have one of their strongest albums there.
But, since I usually do have further information, I try to figure out what album(s) is(are) considered classic by fans of the band, and start from there. Then I tend to move chronologically from earliest to most recent.
I always go to allmusic.com and find their best-rated and most comprehensive greatest hits album. This is especially helpful for pop or country singers, who might be more “singles artists”. If they don’t have a greatest hits album yet I’ll go for the album with the most songs I know it, then use that rule to go for the other albums in descending order of familiarity. (That was an odd sentence, hope it makes sense)
For rock artists I’ll be more inclined to buy a full album first - my Iron Maiden knowledge started with Dance Of Death but from there I went with Edward The Great which is a best-of. Similarly, my Beatles fandom started with Sgt. Pepper’s and The Beatles but from there I went with 1.
Basically, if there’s a good hits album available I find that’s the way to go 99% of the time, then judge what songs on the hits album I like most, and get the albums they belong to.
I get the album that has the piece that I’ve heard, preferably a “Greatest Hits” album, to get a sense of their other work, then proceed from there.
Generally as I can afford them on Ebay. 
I usually end up buying the most recent. If I like it, i think it’s really interesting to then go back and listen to their previous albums to see how they’ve evolved from one album to the next.
BTW, i’m a person that generally listens to the whole album. Even in the age of itunes i like to buy the whole album and listen to all the songs in the order the artist intended. I very rarely (if ever) buy a single song.
Does the artist choose the layout, or the producers?
I vaguely remember reading of a jazz artist who hated a particular song she had to record.
I was going to put a caveat that maybe i’m reading too much into it, and they don’t even have much to do with the order of the songs (in some cases).
… but i like to think that the album is exactly as the artist wanted it. Naive, probably, but its what i like to think.