First, one of the basics is to find a spice store. Spices are cheaper and fresher there. If it’s a good spice store, they’ll probably have jars you can open and sniff, so you can get an idea what you’re buying. I sometimes go to WorldSpice in Seattle, but there’s a shop in downtown Olympia that’s just as good. Note the prices. (It’s cheaper than you think!)
Once you have a spice store you can buy from, then you can play with spices for not much money.
The Basics according to Fish, listed in order of importance:
B1. Salt. Using salt properly is the third-most important thing you can learn to do as a cook. (Learning how to manage heat is the second. Keeping the kitchen clean and food stored safely without contamination is first.) It might help you to get a wide-mouthed container like this or like this that you can scoop a measured amount of salt from. If not, no worries.
B2. Get a pepper grinder. You may not believe the difference this makes. Ordinary black peppercorn is perfectly fine. You don’t need the gourmet stuff to start with.
B3. Garlic powder. Yeah, you could cook with garlic cloves, and I often do, but garlic is a pretty all-purpose kitchen basic. It’s used in just about every cuisine in the world except Scandanavia (and there’s no helping them :)). It doesn’t have the kick of whole garlic cloves, but if you let it cook in and rehydrate, you’re in business. (Not to be confused with garlic salt — you want to add flavor separately from adding salt.)
B4. Onion powder. Sure, you could cook with an actual onion. Nothing wrong with that. But sometimes you want to add onion flavor without adding a whole bunch of liquid and solid matter to your dish.
B5. Chili powder. Storebought chili powder is kinda … blah. But in a spice store, you can usually pick up some chili powder with a little kick to it. Since many different cuisines use chiles and various peppers, a little of this is always handy. If you wanna get fancy later on, you can get ancho pepper (powder from the dried pablano pepper, one-star heat) or chipotle pepper (powder from the dried jalepeño pepper, two-star heat) or cayenne pepper (three-star heat). Me, I go for the cayenne (see below).
Basic Herbs. Herbs are dried leaves. You can cook with fresh herbs but they’re not as forgiving, they’re more expensive, and they don’t keep well. Dried take longer to cook — you have to give them time to soak up liquid and dispense their flavors — but they will do just fine until you get the hang of it. Listed in order of importance according to the Fish rating system:
H1. Basil. One of the quintessential Italian-food herbs. If you’re cooking with tomatoes, you have to use basil. I think it’s in the Bible somewhere. Honest.
H2. Oregano. Great with fish, chicken, meat, vegetables, in tomato sauce, and on pizza. The other quintessential Italian-food herb.
H3. Sage. Sage is what you taste in most stuffing, and in breakfast sausage (apart from pepper and fats). Use with poultry, stuffing, or on red meat for that sausagey flavor.
H4. Thyme. Pretty damn useful all around. It’s got a nice and subtle flavor that’s not overpowering. Soup, chicken, sauce, mushrooms, seafood… yeah, thyme is fun.
H5. Rosemary. It gives you a nice Italian or French flavor for chicken, fish, pork — white meat, really.
H6. Dill Weed. This is good stuff in chicken salad, egg salad, on fish, and on veggies.
H7. Tarragon. This is a great little herb with a mild, sweet flavor to add to eggs, chicken, or fish.
Basic spices. Spices are roots, bark, berries, seeds, dried fruits, or other parts of the plant besides leaves. Spices are usually dried and ground (although if you buy them whole and grind them yourself they stay fresh longer).
S1. Cumin. This is the #1 ingredient in chili powder, Mexican food, Indian food, and yummy food everywhere. It’s also #1 on the Fish rating for usefulness.
S2. Cinnamon. Not just for toast and cookies. Cinnamon is a small component in many cuisines (including curry). What we call “cinnamon” is probably actually cassia, which is just as good in different ways.
S3. Cayenne. Also called cayenne red pepper. It’s a great spice with some heat to it, useful in everything from Cajun to curry to tacos.
S4. Ginger. Although you could cook with fresh ginger root, sometimes powdered ginger is easier. I use a little of it in curry, in stir-fry, and anything where I need a little lemony kick.
S5. Paprika. This is good stuff for chicken, eggs, marinades, and sauces.
Extra Credit Herbs and Spices
If you don’t have the following, don’t panic. You can do a tremendous amount with the above list. Nevertheless:
E1. Bay leaf. Stick one of these into your spaghetti sauce while it simmers, or into your stew, and you get a wonderful fragrant addition.
E2. Cilantro. Great for adding a little spicy flavor to Mexican food. Some people get a soapy taste with cilantro. I would ask before using a lot of this.
E3. Mustard powder. Use this stuff to make salad dressings, mayonnaise, barbecue sauce, or stick it on meats. You know… mustard.
If you bake: nutmeg, cloves, allspice.
If you like curry: coriander, cardamom, turmeric.
If you like Asian cuisine: Sichuan peppercorn, fennel seed, anise seed.
Because this is the Straight Dope I fully expect someone to come along and say I left out something absolutely essential (fenugreek, sumac, lavender, chives, mace, marjoram, parsley, saffron) or that there’s another use for the spices above that I didn’t list.
But that’s the beauty of cooking with spices, you see. I consider certain spices essential, because that’s the kind of food I like to cook. Someone else will have a different approach to the whole thing. 
The other herb I recommend not using dried–parsely–is also practically free in most stores. But, honestly, dried cilantro in something like a pico de gallo would absolutely kill the dish. I’d recommend leaving it out or even substituting fresh parsley. But dried will ruin the dish.