Whether mass is conserved (in special relativity) just depends on how you look at it. Mass is the norm (i.e. the ‘length’) of the four momentum vector P. For an isolated system P[sub]system[/sub] and hence it’s norm |P[sub]system[/sub]| is constant, i.e. the rest mass of the system considered as a whole is conserved
Let’s say the system is made up of a number of n particles with momenta P[sub]1[/sub], P[sub]2[/sub], …, P[sub]n[/sub], then
P[sub]system[/sub] = P[sub]1[/sub] + P[sub]2[/sub] + … + P[sub]n[/sub]
however the sum (which I just call M for convenience,not to imply that it’s more proper to think of it as the mass of the system)
M[sub]system[/sub] = |P[sub]1[/sub]| + |P[sub]2[/sub]| + … + |P[sub]n[/sub]|
(where P[sub]1[/sub], etc, are the rest masses of the individual particles) will not neccesarily be equal to |P[sub]system[/sub]| and is not necessarily constant, i.e. it is not conserved.
In many ways most prefer to talk about the conservation of|P[sub]system[/sub]| in terms of the conservation of energy (it’s just the statement that the system’s energy is conserved in it’s rest frame) rather than describe at as the conservation of mass. On the other hand it’s a much of a muchness: |P[sub]system[/sub]| is conserved and M isn’t conserved so if you choose to talk about the conservation of mass in terms of the former then mass is conserved and if you choose the latter it isn’t conserved.