The reasons for not rebuilding the Temple are many. The lack of a red heifer, mentioned by CMKeller, is a technical restriction (and, Arnold, the “red and white” cows ain’t red heifers, they’re red-n-white heifers.) In fact, a year or so ago, there was a big commotion because someone claimed to have a red heifer… which caused quite a stir in the Orthodox community, as it would be the first pre-requisite for rebuilding the Temple.
The heifer turned out to be blemished, and I venture to say that most modern Orthodox breathed a sigh of relief.
And here’s the why, and I’ll offer two reasons, a theoretic and a practical.
First, the theoretic. Legend/prophecy associates the Third Temple with the Messianic Age. Therefore, a large number of Jews feel that humans should NOT rebuild the Temple, it will be God-directed or Messiah-directed. (BTW, there were also a small number of Jews who felt that Israel should not have become a state, for the same reason – they should wait for the Messiah to do it.)
Furthermore, IF the Temple were rebuilt, there would be no excuse for the Orthodox not to reinstitute sacrifices, and 99.9% of all Jews (regardless of denomination) think that we have evolved past the point of needing sacrifices again. (There is that tiny minority that study how to do sacrifice, for when the Temple IS rebuilt, but those are what you call nuts.)
Second, the practical. The Temple Mount is held by Muslims, and the Dome of the Rock (second most holy site in Islam) is there. Building a Jewish Temple would mean destroying the mosque, and that would mean full fledged, bloody war to the death with our brethren the Muslim Arabs. No one wants to see that happen, and so there is no effort towards rebuilding the Temple. Indeed, about every six months there is some archaeological hooplah because the archaeologists want to dig somewhere and the Muslims fear that it will undermine the Mosque. Herod’s tunnels were the last such example, which caused riots in Jerusalem because the protesting Moslems were fearful that Israel was trying to tunnel under the Mosque and subvert it.
So, most Jews shrug their shoulders and say, this is a problem that the Messiah will resolve.