Cable Car Museum is already on my list and… hey! I recognize those houses from ‘Too Close For Comfort’.
I don’t have a car there, and honestly, I tend to get a little frustrated driving in a city bigger than Columbus Ohio.
San Francisco is not a city for driving, even if you are a comfortable driving. The streets aren’t quite as bad as Philadelphia or Boston, and the layout is at least mostly grid, but pavement maintenance in the city is a sometimes thing, pedestrians tend to be highly aggressive about jaywalking, and parking is a nightmare. (I tell people that they are better off parking at a CalTran station in Mountain View and riding the train in rather than try to find someplace to park in San Francisco proper.) Fortunately, it has a reasonably good public transit system (at least, for the areas that a tourist is likely to be interested in) and the geography is well suited to transit, provided that you don’t mind a fair bit of walking. Just be aware that things that look close together in two dimensions might be well separated in three; don’t think a hike up from Union Square to Coit Tower will take you only twenty minutes.
Stranger
I wouldn’t even go that far, really. I spent a summer living on the outskirts of the Tenderloin, near the theater district, and enjoyed it tremendously. Lots of good, cheap food to be had - excellent curry is everywhere. Also, I can’t say enough good things about Dottie’s True Blue Cafe: http://www.yelp.com/biz/dotties-true-blue-cafe-san-francisco You will see huge lines there for weekend brunch - that’s because the place has the best, biggest, cheapest pancakes you will ever find. Anywhere. Seriously - go to Dottie’s.
Excellent, you’ll enjoy it. Those houses were also in the beginning of shudder Full House.
San Francisco is an excellent city for walking, right up there with new York and Boston. If you want to go to Golden Gate Park or the zoo you need public transportation, and some places might be a bit far, but most of the places you want to go are easy to get to. You must be okay on hills, though.
I’d recommend visiting the Castro District (get lunch at Chow’s) just to see it. Just jump on a street car on Market and go away from the bay. At the very end of Market is the Castro.
Do avoid the Tenderloin. Totally go to Ocean Beach and the Cliff House. The new aquarium in GG Park is awesome.
Go to the Exploratorium. Then go there again to see the stuff you missed the first time.
City Guides walking tours are usually excellent. They take about 2 hours with lots of stops, warn of particularly steep hills, regale you with naughty / historical / interesting anecdotes and it is free (donation suggested but not required). Their tours cover most of the city, often focusing on the Gold Rush or Earthquake (most tours start in the downtown area or close by).
If you are coming to the Castro anyway, The Castro walking tour is so much more than The Castro Since the Seventies. Before it became the Castro, it was Eureka Valley for almost 100 years. If you don’t take the tour, you’ll walk about 4 blocks, peak in a couple of stores, and miss out on most of what the area offers. And the tour includes a few minutes inside the Castro Theater; the exterior only hints at the beauty inside.
As others pointed out, it is a compact city, made for walking and the neighborhoods are small. It doesn’t take much walking to get from one place to another. The people are friendly, and most will help with directions and suggestions.
I was going to recommend Alcatraz. I don’t generally like touristy type things, but I found the Alcatraz tour really interesting.
If you’re looking for restaurant recommendations, I really enjoyed Bar Crudo. You have to like seafood, though.
My recomendation/challenge : Come home not saying you have to live there!
I’ve never been able to do it!
Have a great time 
PS: Second the Giants Ballpark, whatever the corporate name is now. Was lucky enough to have been there a few years ago in blue sky perfect SF no humidity summer.
Good Lord, it just wasn’t fair! 
We just got back from 2 days in San Francisco! Now, we had a 4 year old and an 18 year old with us, so keep in mind that we had to please them too. Also, my husband and I had been there 10 years ago, so some things we didn’t go back to this time for time reasons (so no Painted Ladies, Lombard Street, Fort Point, or even Cliff House, which is so awesome!).
We did Alcatraz, which is totally amazing. Even my four year old wore the headset and loved listening to everything (although she would NOT stand inside a cell, just in case it closed on her accidentally). We wandered Chinatown and the shopping around Union Square, which was fun. We hit the Musee Mecanique, which was just amazing. It’s free, but we dropped $10 playing and watching the games, and wanted to do more!
We walked out along the bay to Fort Mason. We drove across the Golden Gate Bridge, then went up on the hills above it to take pictures, which was really cool. We walked around the Palace of Fine Arts, but didn’t do the Exploratorium. It’s one of the things that both times we’ve been we wanted to do but time/money constraints always make us skip it. We need to go again! We ended our day at Golden Gate Park. We pretty much hung out at the Playground for my little one, but I would have loved to explore more!
Lucky you, I love San Francisco. Driving is a pain and parking even more so. Don’t bother with cars - I once had to pay $24 for 1 hour of parking. It’s pretty easy to walk around SF and the blocks are fairly small so don’t be put off if someplace you want to go is 10 or 20 blocks away.
I love Golden Gate Park. There’s a few museums and a Japanese Tea Garden. A bit further north is the Sutro Baths, the ruins of a bathhouse from the 1920’s (?). It’s fairly empty during the week days and good for a few solitary hours.
Chinatown is lively and crowded, but lots of fun. Just explore the shops and if you’re there during the weekend and see Happy Happy Happy Man, tell him I said hi. If you’re on Grant Ave between Jackson and Pacific, go to Golden Gate Bakery for an egg custard. There’s usually a line (but it moves fast) and the egg custards come straight out of the oven. I’ve scalded my tongue on those things more than a few times, but it’s always worth it. City Lights bookstore is just north of Chinatown.
If you’re free on Saturday mornings, there’s a farmer’s market at the Ferry Building. They serve lots of prepared food as well and the people watching is great. You can always catch a ferry to Sausalito or elsewhere from there.
At night, head to the Mission District to walk around. Lots of bookstores, boutiques, and restaurants are open at night and it’s nice to just walk off dinner by browsing local stores. If you happen to pass by in the daytime, check out Tartine Bakery for a salad and pastries.
The Tenderloin looks scary, but we weren’t harassed at all when my boyfriend and I wandered there by accident before a concert. Lots of hookers just standing around with their pimps/drug dealers standing by the doorways. Watch out for needles on the street.
Fort Point is pretty cool, if you like that sort of thing; it’s a Civil War era fort pretty pretty much under the SF span of the GG Bridge. The Golden Gate NRA at the other end of the bridge is also interesting; it’s cool to see where the huge gun emplacements were in the WWII era, and the views way up high over the GGB on a clear day are awesome.
I stayed in a nice hotel in the Tenderloin district when I was there. I read about on the web before I went and was concerned. When I got there, I thought the danger and the general ick factor were extremely overblown and mild. It was relatively nice as a matter of fact and the city is small so all you have to do is walk a little and you are in a completely different neighborhood. The sheer number of homeless people in San Francisco is staggering though and that puts some people off but that never bothered me that much but it is interesting. They never bothered me much except to ask for money in an assembly line fashion. I got the impression that San Francisco proper is simply too expensive and too geographically compact to have any truly bad areas except for the homeless population but I used to live in New Orleans which is another unique American city and a tourism hot-spot and New Orleans really does have dangerous areas.
Welcome to the cool, gray city of love. You can spend a lifetime here and then do it again and have a great time. Why you need an RPG store is beyond me (rocket propelled grenade? or role playing game? either is not needed during your short stay.)
Visit North Beach while going to Coit Tower.
My family has been in SF since the gold rush, and nobody has yet done it all (not even counting the gay lesbian scene.) Plenty to do.
The Looff carousel in San Francisco is not in Buena Vista park, but rather Yerba Buena center near the Museum of Modern Art. I think there is another carousel (not a Looff) in Golden Gate park near Keyser stadium, or at least it used to be half a century ago. The one at the beach went extinct with Playland. Sniff.
How funny, I was also going to mention Golden Gate Bakery for its egg custard tarts! SF’s Chinatown is a very vibrant, bustling area. It’s very fun, and I’m glad to see the OP including it in his itinerary.
Tartine Bakery is scrumptious, as well!
Have fun, SF is one of my favorite cities in America!
When you’re in Chinatown, check out this place: Fortune Cookie Factory. It’s teeny but neat. They have chocolate fortune cookies.
You could do a whole beats tour, or just pop over about 50 yards to the Vesuvio Bar on Jack Keroac alley for a drink. http://www.vesuvio.com/
You could get the literary map of San Francisco and use that as a walking guide to check out various places like where Hunter S Thompson used to live during his Hell’s Angels days or where Dashell Hammett hung out. Actually there is a memorial marker on the spot where the guy whose name escapes me at the moment is gunned down in the Maltese Falcon. Map is here: http://www.826valencia.org/store/shop_sf_lit_map.html