Check out the Key West forum at Trip Advisor for more info than you know what to do with.
Sunset at Mallory Square/Sunset Pier - Beautiful scenes, street performers, and booze.
Key West Cemetery - Very beautiful and historic.
Happy Hour - Most places have really good happy hours on food and drink. Check out Alonzos and some of the other places in the area.
Southernmost Cafe - At the Southernmost hotel. Nice place on the beach to grab breakfast or a drink.
Dive bars - Key West has a multitude of them. Half Shell, Schooner Wharf, Chart Room, Green Parrot, some of the best in the country! Personally I find the Duval bars to be way too touristy. Take a walk by the bight and check out the scene there. Many of these places are fine for kids earlier in the day. The Porch is also a fun little place. Sloppy Joe’s is just a tourist trap and from what I understand, not the actual location where Hemingway hung out (that’s now called Capt Tony’s).
Nude Drinking - Look for the Bull and Whistle on Duval. Go to the Garden of Eden bar on the rooftop, order a drink and take your clothes off. Probably want to leave the kids at home for this one.
Food - Key West is one great restaurant after another. Blue Heaven, El Siboney (Cuban), Louie’s Back Porch, B.O.'s Fish Wagon, Garbo’s Grill (food truck), Latitudes (on an island, boat ride included). If anyone’s picky, there’s a Wendy’s, Denny’s and other fast food options on Duval. There are also plenty of dessert places. My favorite key lime pie was at Blue Heaven.
Fort Zachary State Park - Nice beach with rocks you can swim to. You can also just wander around the civil war fort. We had it to ourselves when we visited.
Dantes - Large pool with bar.
Cigars - Plenty of Cuban-style cigars. There are a few places where you can grab a good smoke and get a glass of wine. I don’t smoke and even I enjoy doing this (okay, technically I guess I do smoke then).
The beaches in Key West are very mediocre, but it’s a great town to just explore and chill. There are tons of unique shops, restaurants, and bars to explore. Have fun!