No, it is supposed to be the case that if a challenge is made it is because the ballot was objectionable somehow, such as not clearly indicating who was being voted for, so it would not be attributable to either count. If the challenges are lifted, they have to look at the ballot again and if there is a clear vote, add it the to correct tally.