Castellano, like Brando, made an offer they could refuse. Not only did both men ask for exhorbinant paychecks for the second movie, but Castellano insisted on having his girlfriend brought in to do script rewrites. The producers ultimately passed. Too bad, because having the Pentangeli character be Clemenza would have made the storyline even better.
Incidentally, one of the hitmen that tried to kill Pentangeli is an uncredited Tony Sirico (who now plays Paulie Walnuts on The Sopranos) in one of his first film appearances.
Duvall goes along with Brando and Castellano in not getting back in the sequels. His was in GF3, though.
The DVD commentary makes a good yarn about Pentangeli (and there’s even unused footage about it) that has the black armband he’s wearing at the party represent the funeral for Clemenza. Frankie takes over the Clemenza crew after C dies. That’s the continuity.
The reason given for Castellano not coming back was that he wanted a personal associate to write his lines. And money was a factor, too. Brando’s was totally money. So was Duvall’s in 3.