Most USB flash drives are pre-formatted as FAT-32 volumes, which OS X will read and write to normally without issue, and without giving you any feedback on its format since for most purposes it’s irrelevant. For a backup, however, to preserve metadata and other normally invisible files, you have to format it as HFS+ (journaled). You can do this with Disk Utility. It should be possible to use your flash drive as a backup volume then. Whether this is a good idea is another issue.
I use SuperDuper for bootable backups, which is relatively low cost, but not free; it’s currently $28. You can get really granular with the rules, if you want, though it’s not really intuitive to do so unless you’re a bit of a techie. You can also do some powerful scripting stuff pre- and post-backup if you are technically inclined. This is the best software besides Carbon Copy Cloner for a bootable backup, and I obviously chose it instead of CCC, so I think it’s better in some ways.
I don’t use Time Machine for my computer, since it’s a portable and I’m almost always away from my backup drives during the day. It’s better for me to manually run backups when I can, and I need to make fully bootable backups. I use TM for my wife’s computer, though.
Free software:
Time Machine will do a whole-disk incremental backup. If you don’t want to use the whole drive for a backup (for instance, you have a 3 TB disk backing up a 1 TB drive) partition the drive before the first use, and designate one of those partitions as the backup volume. TM will cheerfully eat up any space you have if you don’t partition the drive since it assumes the whole thing is nothing but a backup drive.
(That’s probably a good assumption to make most of the time, since the risk of failure increases if you’re using it for something else, and increasing the risk of failure is a Very Bad Thing when you’re dealing with shit that’s supposed to save your ass. I totally understand that sometimes people buy a huge drive and don’t necessarily want to use all of it for just a backup. A mirrored RAID disk is pretty damn safe, no matter what partitioning you use and how you use the rest of the space, but a single partitioned disk also used for other purposes is slightly less safe than a disk used only for backups.)
A decent partitioning scheme would be to allocate 1.5x the space of your source drive for the backup volume. That leaves plenty of space for incremental backups of even fairly large files as long as your home drive is about 80% full, which about where you should be running. If your drive is within 10–15% of capacity, you really should think about upgrading your drive. If you’re running way under the max and don’t think you’ll add huge amounts of data at once, you could just set the partition at the same size as the home drive. That’s how I have my wife’s backup set, since she’s using less than a quarter of her drive and doesn’t add files any larger than a dump of 12 MP RAW photos most of the time.
Be sure to exclude any directories you don’t want included prior to the first backup. The default is to back up the entire disk, including other user files, no matter which account you enabled TM on. I set it up on my wife’s computer expecting it to back up only her files, and it synced my account too, taking more than twice as long as I expected on that first run because I have way more files than she does.
For directory-level backup, there are GUI interfaces for the command-line tool rsync. One of these is probably your best bet for free and good software. If you’re not a techie, stop reading here. Look for a GUI front end for rsync called arRsync. Other GUIs for rsync exist, but that’s probably the best and easiest to use.
If you’re comfortable with the command line or scripting, then you can start using rsync directly. If you are generally okay with tech, but not super-familiar with UNIX and need some guidance, open the Terminal and do “man rsync” (without quotes) for some documentation. Depending on file permissions, you may need to invoke super-user privilages (“sudo” command) to make sure all files, including invisible ones, sync over.