I would also highly reccomend the Bronx Zoo. You can easily kill an entire day there.
You can take the #2 or #5 subway (whichever is closer to where you are in Manhattan) to East Tremont Ave in the Bronx, which is about three blocks from the zoo.
If you’re afraid of the subway you can take the BMX11 bus, which goes directly to the zoo from Madison Ave.
I recommend checking out the goCityKids website for NYC – it has a lot of great ideas for things to do with kids in the City. (I subscribe to a weekly email newsletter put out by that site, and I got an email from them just today saying that it will become a paid membership service as of June 1st. The charge will be just 95 cents a year – well worth it, I think, for such a great site. In any case, you can check it out for free for the next week or so!)
Having lived in Manhattan with my husband and kids from before the kids were born until the oldest was five, I speak from experience when I say that NYC is a very kid-friendly place to live or visit. (My oldest is now 13, so it was seven years ago that we moved away, and I still miss it!) I agree with many of the recommendations here – the American Museum of Natural History, the Children’s Museum, the Met (especially the Egyptian mummies & armor, as another poster mentioned), the Bronx Zoo, and Central Park, especially.
When we lived in Manhattan, we often went “playground hopping” in Central Park. Most of the playgrounds are located near the perimeter of the park. We would start at the Diana Ross Playground at 81st St. & Central Park West – catty-corner from the Museum of Natural History – and then work our way around the park, spending time in as many playgrounds as we could until we ran out of energy! You might think that one playground would be pretty much the same as another, but in fact they are all quite different. It was a lot of fun.
A couple miscellaneous comments on some of the suggestions made so far:
We did not have a car when we lived in Manhattan, so we always took the subway to the Bronx Zoo. I’m a worrier, but I never felt unsafe doing that with the kids.
-Playspace on the Upper West Side was a great place, but I’m pretty sure it has closed down.
-I personally love Ellis Island, but I think it would be a little too advanced for a three-year-old.