Not quite. He kills people who the law could punish (and in some cases has punished) to prevent future deaths. For instance, the kid from around episode 2 or 3, who Dexter targeted after the kid was released from juvie becasue he recognized that the kid would kill again. He let the kid go because he had kiled to avenge being raped, but then targeted him again after a second body turned up.
The cases were closed, as suicides, and the coincidence of their having the same therapist would be unlikely to be enough to convince the department to re-open the case or to convince a judge to issue a search warrant.