Hesitant though I am to argue with a former CPA, I don’t think your explanation is correct, Rubicon. Under your explanation, the bonus is being taxed correctly (but at a higher percentage rate than the average of the paycheck, i.e. at the marginal rate).
Other posters have explained why the bonus is actually being overtaxed. Using your example, if someone gets an extra $1,000 one week, it is not that the extra $1k is taxed at the marginal rate for $52k. The issue is that the withholding system looks at the paycheck of $2k for one week, multiplies it by 52 to get an annual rate, and taxes the extra as if the person earns $104k per year. So the first part of the bonus may be taxed at the person’s marginal tax rate, but the rest of it can be pushed into even higher tax rates.
At the end of the year, when you submit the tax return it becomes clear that the extra $1k should only have been taxed at the marginal rate present at $52k, so the amount extra taken at higher tax rates is eligible for refunding.