I heard from my father, a former Marine who guarded Camp David, that it would not be legal to have the troops of another countries army guarding an embassy. Since the Marines are not part of the army they get around this rule. This might not be the law in all countries but maybe in enough to make it the tradition. No cites, just piggy-backing this on to the OP in hopes that someone has the real answer.
Rumored by whom?
Marines, possibly. 
Except the Amphibious Seabees get there first so that the Marines have somewhere to land. But we’re used to the Marines claiming to be the “first to fight”. We happily guard their stuff while they’re off continuing the fight.
Actually, the Marines do have their own dedicated transport vessels (which double as aircraft carriers). I’m not sure who actually commands the Bohomme Richard.
USS Bonhomme Richard LHD6
Capt. Stephen Green, USN, Commanding
The primary mission of USS BONHOMME RICHARD is to embark, deploy and land elements of a Marine landing force in amphibious assualt operations by helicopter, landing craft, and amphibious vehicle.
http://www.lhd6.navy.mil/
Only the Marines would dare ride around in a vessel nicknamed “The Bonny Dick”.
Think that’s bad, my brother served aboard the “Queer Barge” (USS Kearsarge CV33) in the mid 60’s. They decommissioned her in '70 and there’s a new one now LHD3.
To top it off he was an Admiral’s staff weenie, I remind him of that every chance I get.
More likely because historically they were the one US land force that was deployable and deployed overseas as a matter of course (because of the Navy link). I don’t think the Army ever left the North American continent until 1898.
Alessan, the functions of national ceremonial guard are divided between both an Army unit (3rd “Old Guard” Infantry Regiment) and a Marine unit. The Old Guard’s most visible postings are the Guard of the Tomb of the Unknowns at Arlington, and the cavalry troop for state funerals; the Marines’ as you mentioned is the White House. Seems historically the War Department and Navy Department were both called upon to provide a unit of land troops for local defense and ceremonial duties and through custom and usage each got locked into its particular billets. They even divide music duties, with the Old Guard providing the fife and drum corps and the Marines providing the full band. In any case in the military role they would have always operated as a joint command (in the old days, probably with the Marines covering the riverfront and the Old Guard the land approaches).
As someone already mentioned, the US Marines are quite unusual in their size relative to the normal Navy and Army and in their standing as a service in their own right. Most other countries have “Naval Infantry” be an integral part of the Navy, and even in the ones with distinctive Marine services such as Britain and Spain it is closer to the pre-WW2 US model. One thing that is relatively recent, Monty, is having Marine Generals as major Theatre/Joint Command supreme commanders such as SACEUR, STRATCOM, etc. (Gen. Pace, current Chairman of the JCS is the first Marine in the billet in almost 60 years of the organization). Used to be that only generals/admirals from the full-department services would be appointed, and it only “broke” in 1985 – first time a Marine other than the Commandant or Assistant Commandant wore 4 stars (and the Commandants alone were the only active Marines at 4-stars from WW2 until 1969). Part of it had to do with intersevice politics, part of it with intraservice politics, the very small pool of generals mandated by law, and the specialization in frontline combat over the large-enterprise-management orientation expected of a theatre “cinc”.
Hmmm . . . And all this time I was under the impression that the Marines were created so the Navy could have men to dance with and someone to open the lids on jars. . .
OK, I’ve exercised restraint long enough.
Marine: seagoing bellhop
A Marine is just a Seabee w/ a light duty chit.
USMC: Uncle Sam’s motherless childtren.
You can always tell a Marine, you just can’t tell 'em much.
Marine leadership: The guy w/ the loudest mouth.
How do sailors keep Marines from sleeping w/ their GF’s when they’re at sea?
Leave a can of Lincoln wax on the bedside table. 
Okay. I get most of those jokes. What’s the deal with the wax? Do I want to know?
Lincoln wax was, maybe still is, the Marines favorite shoe polish. They’d rather spit shine their shoes than screw.
You forgot my favorite:
Q: Why does the Navy refer to their undersea ships as “subs”, not “submarines”?
A: Because there is nothing lower than a Marine!
Sure, the Marines are the best.
Just ask 'em.
It’s Kiwi wax now. I don’t even know if they still make Lincoln wax.
Ok, so an airman, a sailor and a Marine were caught with alcohol by the Saudis, which is a big no-no, and were sentenced to 50 lashes each. But the Saudi prince in charge of the punishment wanted to show mercy, so he offered each man one last request.
The airman, being the smartass typical to that branch, asked for a Koran to be strapped to his back prior to the whipping, thinking that the Arabs would spare him in order to not desecrate their holy book.
“Impudent infidel,” said the prince," we anticipated just such a profane trick from you, and instead you shall be scourged wearing your own Bible!"
So the Bible was strapped to the airman’s back, which, in any case, did little to protect him as the whip soon cut through it and lacerated him badly.
The Marine’s turn was next, so thinking quickly, he asked for two Bibles to be strapped to his back, and almost all the lashings were done before they were cut through and his back was only slightly lacerated.
When the sailor was asked for his last request, he said “I request not 50, but rather 100 lashes!”
“Ah, you are truly a brave man,” said the Saudi prince," and in recognition of this I will grant a second request. What shall it be?"
“Strap the Marine to my back!”
The Marines were the first to go to the brown suede boot as their everyday boot. With the introduction of the ACU the army did also. The army used to use the brown suede as their desert boot only. No more boot polishing in either branch.
Spoken like a true leg. . .
Airborne units still wear black spit-shined jump boots in Class As.