If we are talking about an average Neanderthal then reach and weight could be an issue. Neanderthals were shorter than sapiens, and most wouldn’t have reached the heavyweight limit of 200 pounds.
Where they would have excelled would have been wrestling rather than boxing. They could have brought their superior strength to bear more easily. Many Neanderthal skeletons show fractures equivalent to those of rodeo performers, which has caused some scientists to speculate that they hunted by bull-dogging large animals to the ground. Someone who can pin a musk-ox wouldn’t have a problem with any modern human.