I’m looking to put 1GB of memory in a new PC. Other than “upgradability”, is there any reason to choose 2 512MB dimms over 4 256MB dimms, or vice versa?
My gut instinct is that you’ll get (slightly) better performance using all 4 dimm banks instead of just two.
You would only gain in performance if you had quad-channel memory memory controller; IIRC, the only “mainstream” chips that would allow this would be a dual Opteron machine - and if you need a dual Operton, you need more than a miserly gigabyte of RAM. 
Almost every other motherboard uses either dual channel memory or single channel; note that with the Athlon 64 chips, there is only a slight performance gain by going dual channel; I don’t know about Intel’s stuff.
RandomLetters is right, most current motherboards can only talk to one or two dimms at a time, so having memory in four dimm slots isn’t going to gain you any performance.