OK. First of all, our primate ancestors were not primarily hunters. Our hunter-gatherer contemporaries are usually not primarily hunters. Our species is not, and has never been, primarily a hunting species. And besides, why would only males hunt? In every other carnivorous mammal species, the females hunt just as much as the males, in fact more because they must feed their babies as well as themselves. It makes no sense.
So, male hunting fails as an explanation. Even if we DID hunt all the time, it would fail because just about every mammal species there is has larger males than females, and the ones that don’t have larger males have equal size males and females.
So, the question is not “why are human males larger than human females”, the question is “Why are male mammals larger than female mammals”.
And the answer to that is clearly found in the mammalian reproductive system. It requires a lot of resources months of time for females to produce offspring. It requires about 10 seconds and a few drops of protein for males to produce offspring. So, it is possible for one male to mate with many females at once, but a female can generally only mate with one male at a time…or more precisely, it doesn’t matter how many males she mates with, she will have the same number of offspring.
So therefore, males will attempt to mate with as many females as possible, while females have no such advantage. Now, we can see that if males attempt to mate with many females, there will be a shortage of females to mate with. So, the males will struggle amongst themselves in various ways. One of those ways will be physical. And once we have physical competition, then we have intimidation…males will evolve structures to intimidate other males. So, since males are struggling, the largest and strongest males will tend to have the most mating opportunities, and hence the most offspring…and hence the next generation of male offspring will be larger.
But I hear some of you asking, don’t males have to contribute resources to the offspring too? Well, not usually. In almost all mammal species the males contribute exactly nothing to parental care. And guess what? In species where the males DO contribute to parental care they tend to be about the same size as the females. Since both contribute to the offspring, both must invest equal resources…and so there is no advantage for the males to mate with multiple females.
Species that do not have a mammalian reproductive system generally do not have larger males. Many birds have larger females than males. And in those cases, generally the females lay eggs in the nest of a male, and leave him to raise the chicks. So, the FEMALES must compete for scarce males.
So, why aren’t there more mammal species where the males take care of the offspring? Because of internal gestation. Once an egg is laid, either parent can be left with it, or perhaps it can be left behind by both. But a mammal embryo is left implanted inside the female. It is much much easier for a male to walk away from the embryo than for the female. So, male mammals have generally stuck the female mammals with parental care, because they can. The female could leave the babies with the father, but usually he is long gone by the time the babies are born.
Summation: Mammal males are larger than mammal females because mammal males must compete for the chance to reproduce.
But generally, if the males fight each other for females, the males are larger.