At home we were bombarded by malware and I switched to Firefox when I heard that it avoids a lot of crap by virtue of not being MS—i.e. really common and therefore a popular target. I haven’t had much problem since then (though I did install a bunch of adware & spyware fighting stuff at the same time). I also installed it on a directory that I have hidden just so that it isn’t easy to find for the casual observer.
Personally, I find the tabbed browsing to be absolutely brilliant. I now use Firefox at work for work and slacking. E.g., I’ve been looking up census data for the master plan update and using tabs made it much, much easier.
Even so, I still use the IE Privacy Keeper because it cleans IE while it’s at it, and recently used programs, documents, temp file, recycle bin, etc. so that junk doesn’t end up building up. Also, I discovered that if you download a clip of judo throws, it can end up sitting in the temp file, so I’d assume a clip of, say, Jennifer Garner is lingerie might do the same, so doing the IE Privacy Keeper, or something similar, might help avoid embarassment.