You undercounted John Pertwee and Sylvester McCoy by 1 each and overcounted David Tennant by 1 (even accepting the “year of specials” as a full season). So that changes the averages for old and new to 3.7 (3.2 if you arbitrarily and conveniently discount Tom Baker) and 2.5 respectively.
ETA: Actually, I’m not sure it’s fair to count McCoy as leaving after three seasons. The series was canceled (or put on a very long hiatus for extensive retooling). He wasn’t actually replaced until the 1996 movie (in which he had a brief appearance). If he’s removed from the calculation, the “old” average goes up slightly.