I know some people who have worked there (and IMF). I am not intimately familiar with it and someone who has been an actual employee may check in, but this is basically true. If you are a foreign national (those agencies are required to have some percentage of foreign nationals as employees), you do not pay U.S. income tax. I do not understand the legalities of this, or the rationale. I think there is some kind of reciprocity deal at the national level that is supposed to compensate for this. I think that employees who are U.S. citizens might be considered 1099 contractors or something and have to pay taxes, but they are not subject to withholding, IIRC.
The IRS has about five pages on this, and the guidance appears contradictory to me, but the gist is: “Subject to the provisions of section 1 of the International Organizations Immunities Act, remuneration paid for services performed within or without the United States by an employee for an international organization as defined in section 7701(a)(18) is excepted from wages and hence is not subject to withholding of U.S. federal income taxes.”
However, this says that the World Bank may issue extra payments to US citizens to cover federal, state and local income taxes.