I’m sorry, I can’t keep it brief when I talk about classical music. The subject is too large and my passion for it irrepressable. I beg your indulgence, or skip down a bit.
Well, first off, I’d like to offer some practical advice. Look for free and inexpensive ways to explore classical music first, and that will help you to decide what you need to buy, and what you don’t.
I don’t know specifically what part of Ontario you’re in, but first off, listen to some CBC Radio 2 while it still offers classical music. (There’s a whole other thread…) Check your local library and see what’s in their collection. Are you near enough to a University that you could get a research reader’s card? Some university libraries allow you to sign out CDs, others, like U of Toronto, will only allow you to listen in the library. At any rate, that may be worthwhile.
I’d also encourage you to check for opportunities to hear classical music live. There are many fine orchestras throughout Ontario, and lots of chamber music groups, festivals, etc., etc. If money is an issue, see about volunteering - particularly for things like the Elora Festival, the Guelph Spring Festival where they need local volunteers. Live classical music is different from the artificial perfection of modern recording. Precision may be harder to achieve, but it’s easier to follow the thread of a musician’s approach to a piece. If you are near enough to a University that has a music faculty, student recitals are usually free. Hit or miss, I grant you, but it’s kinda cool to be able to say in 15 years that “I heard Russell Braun when he was in third year.”
There are so many ways into classical music, that it’s hard to know where to start. Assuming you have access to a lot of different music by a lot of different composers (either through purchase or borrowing), now what? I like to focus on the different ways pieces of music can connect.
Consider - there’s the period in which a composer was writing, and that’s how a lot of people look at classical music. It can be misleading, though - Ives wrote using ideas that no one else would think of for years to come; Bach wrote dance suites based on dances no one had danced to for decades. Sibelius wrote in a late romantic style long into the twentieth century.
For fun, think about following other connections. Think about following a particular instrument through its development - the piano, for example. How did Mozart and Haydn approach the instrument, and what was different about Beethoven’s piano works? Not to mention the leap to Chopin, and from there to Liszt, and from there… There’s a whole tapestry of threads to follow within just the development of that instrument.
Classical guitar is sometimes referred to as being in the trailer park of music history because it was such a specialty instrument - difficult for non-guitarists to write well for, and coming into its own at the same time as the piano. (side plug - Naxos records features budget recordings by lesser-known orchestras - I don’t recommend them for most of the standard rep., but the guitar series is superlative, and is the best collection of classical guitar going! Okay, that’s only my opinion, but it’s fervently held.) CG has its own heroes, its own composers, its own nationalisms and its own folk influences.
The number of ‘threads’ to follow is huge - instrumentation, nationality, cities (consider following what was written in Vienna, or Paris, or Venice at various times), groups (‘Les Six’, the ‘New Vienna School’, ‘The Impressionists’, etc.), literary or artistic influences, particular performers (why did Glenn Gould record the Haydn and Sibelius that he did? How did that fit in his mind with the Bach for which he was famous?), take the same piece and explore different performances of it (how is Die Winterreise different when it’s performed by Dietrich Fischer-Dieskau, or Thomas Allen, or Olaf Bär, or Dave who is in 4th year of a Bachellor of Music and is doing this for his Grad Recital and who is in way over his head, or Hans Hotter at 65?), influences of Folk music, influences of Popular music - the list is endless, and the repertoire is huge.
So, start from what you like, and add to it one small step at a time - say, from a string quartet by composer X to a woodwind quintet by the same composer. There is enough music out there to thrill and inspire you for more than one lifetime, but don’t let it overwhelm you - just let it embrace you.