Well, I got to feeling frisky last night and installed the new power supply.
Blessed silence! (well, not quite, but it is much much quieter and no “eeeeeap!” noise).
I left the computer on overnight just to make sure, and it was still very quietly whirring away this morning. I am cautiously optimistic that the new power supply has solved the issue.
Thanks guys!