Waite Hoyt was born in Brooklyn and made it into the Hall of Fame primarily with the Yankees (Brooklyn had been consolidated into New York city the year before his birth, even though Brooklynites still considered them to be separate as far as baseball was concerned). He also had a couple of brief stints with the New York Giants and Brooklyn Dodgers.
Sandy Koufax was from Brooklyn and played for the Dodgers before they moved west, but actually became a star after the move.
Johnny Murphy, from NYC, was the ace reliever for the Yankees in the 30s (one of the first real relief specialists).
Phil Ruzzuto was originally from Brooklyn.
Though he only became a Yankee a few years ago, Alex Rodriguez is New Yorker.
Baseball-reference.com has a list of players by birthplace.