So I get pronounced noise in my headphones when I’m using them on my computer (probably because their driver size is small, much more sensitive). This is probably because of RF emissions, right? So does anyone have any SOLUTIONS to this problem? Put a Faraday cage around my PC? A soundcard instead of onboard audio? Fold myself in half?
What kind of noise is it? Even onboard audio solutions aren’t really susceptible to RF interference. 99% of the time, noise is caused by an unmuted input on the soundcard. This could be a microphone port, Line-in, CD-In, Aux-In, etc. I’d suggest you download and install the latest drivers for your soundcard, then mute/disable everything except the essentials, usually Master, Wave, and Directsound. Possibly MIDI/Synth too. That should cure the problem.
A high frequency clicking noise (50-60Hz?) is heard when the mouse is moved, windows are moved.
If your mouse is wireless, try a corded mouse. When you move a wireless mouse, it sends out EM pulses to tell your computer to move the pointer on the screen.
I’d like to reiterate my suggestion about muting inputs. The kind of RF interference generated by devices found in a typical home ofice, including mice and such, should be well below the level of audibility in a properly configured system. It becomes audible when inputs are active because the signals are so much weaker and are then amplified before being dumped to line-out.
You haven’t got a “Windows Theme” switched on that has sounds for these events?
Dumb question I know, but had to be asked. 
-This is not a Windows Theme event.
-I muted the microphone, AUX, line-in, and mono in. Still has that sound. Will disconnecting the physical line-in plug do anything?
I had a similar problem with a brand new machine I bought. The noise only stopped once I moved to a new apartment, and hooked it up to a new, different power source. I still have all the same components nearby, but no noise. Maybe try a different power outlet (if it’s on a different circuit), or hooking up at someone else’s house. That way you can at least determine if power is a factor.