Recommend a solution to my voice recording needs

I need a portable voice recorder. I plan to use it to record speeches, both in rehearsal and in performance for later playback and analysis. I will probably want to retain the rehearsal recordings until the actual performance (figure a maximum of 1-2 months, but potentially 15-20 recordings each of up to 10 minutes in length at a time), while I will want to retain performance recordings permanently.

There seem to be a lot of options.

I could get a plain vanilla cassette recorder. I dislike this option because of the size. I simply do not believe carrying a full size cassette recorder is practical and have, thus, eliminated it.

I could get a mini-cassette recorder. This addresses the size concern from above, because it’s smaller. It’s also relatively inexpensive. I’m concerned, however, about the tape factor. I don’t want to spend a lot of time fast forwarding back and forth through a lot of tapes looking for the specific version of the speech I want to hear again.

I could get a mini-disc recorder. These are smaller, more expensive and have the advantage of being digital. I’m concerned about the mini-discs though, because I can’t play them back with anything but the recorder. Presumably, if I can download music to the recorder from the computer, I can upload the voice tracks back to the computer, and then burn the ones I actually want to retain to a regular old CD. I think. Yes? None of the ads I’ve been reading (mostly at Amazon) really specify.

I could get an MP3 player / recorder. They run really bloody expensive, but have the added bonus of allowing me to rip some of my kabillions of CDs to the player as well, and using the actual intended functions. Ideally, though, this whole affair would cost me less than USD200, and MP3 players that include recorders seem to run pricier than that.

So, any experts on the subject? Thoughts on which of the options above I should be looking at? And if you have thoughts on that, are there specific items I should be looking at within a category? My priorities are about:
[li]40% flexibility/ease of use for playback. Bookmarking would be good. Also, the ability to upload to PC in some sort of normal format. [/li][li]40% cost - not just for the unit, but also for whatever media I’m going to have to buy (discs, smart cards, etc)[/li][li]10% having it be useful for other things[/li][li]5% reliability, if I miss recording one speech or one rehearsal because of equipment failure no biggie, but I don’t want to be in a scenario where I could potentially lose the whole catalogue to any catastrophe less than a house fire. [/li][li]5% having it not be a technology dinosaur or proprietary mess in the next week and a half that I can’t get media for without paying a fortune to some guy with -2 feedback on eBay. [/li][/ul]