When an American soldier is given a command by an officer, he responds “Yes, sir!” Since Japanese does not have an equivalent to “sir,” how does a Japanese soldier respond to an officer? Simply “Hai!”? “Hai!” + the officer’s rank?
The consensus amongst Japanese speakers in my office is that an order would be acknowledged simply with hai or ryokai (short for ryokai shimashita, loosely “I understand”). None professed any particular knowledge, though.
They (“hai” or ryokai") would seem to me to be the commonest kinds of response in anime. There isn’t a good Japanese equivalent of “Sir”: you would address people by name, or by a title/honorific, e.g., “sensei” (used for teachers, doctors, lawyers, etc.).
Technically speaking, the modern Japanese Self Defense Forces aren’t a military organization. In practical terms, they are, but you’ll get jumped all over like a Bush-lover in a Pit thread if you say as much in Japan. The Imperial Military had different ranks and organization than the modern one. Branches of the military have standardized terms somewhat; the pre-War Rikugun (Army) and Kaigun (Navy) had different terms for some of the same weapons and equipment, from what I’ve heard. There are still probably a lot of slang expressions or non-standard terms and patterns that are unique to branches of the SDF, though. I’m far from an expert on this topic, but I’ll tell you what I know. Ironically, I seem to know more about it than my Japanese wife 
Acknowledgments are usually ryôkai (了解 lit.; comprehend) and a variation of hai that sounds more like a chopped-off ho. You would properly address anyone, regardless of higher or lower than you, by his rank, just as you would in the armed forces of most countries. The equivalent of Private Pyle in Japanese would be 一等兵パイル (Ittôhei Pyle) assuming he’s Private First Class. Major Kusanagi from Ghost in the Shell, an anime with strong real-world armed forces influence, is referred to as 少佐, (Shôsa) which is her rank. I didn’t catch her actual name the first time I saw it since she’s almost never referred to by name.
There’s a rank structure at this website that shows current equivalent ranks in many major armed forces. Comparisons are probably not really exact. The listed ones are best-fit terms, I’m sure. There are differences between US and UK ranks, after all, and one is directly descended from the other.
Japanese runs to formality and politeness to begin with, and military forms of expression in any country I’m aware of are highly structured with a default level of formalistic politeness. Linguistically, I would be surprised if there are any really big deviations from the norm in Japanese, but since I don’t know anyone in the SDF personally, and haven’t seen many media featuring natural non-scripted language, I could be wrong. I might not pick up the subtleties anyway since I’m not fluent enough to catch more than broad accents and relatively large deviations in word choice.
I must go back and watch again the anime series Zipang, sine (apart from the time travel) it’s a realistic story involving people serving in the Self-Defence Forces (on a modern cruiser) and in the Imperial Japanese Navy (including the battleship Yamamoto) – including interaction between the two groups, because of the time-travel element.
The bolded should be “Yamato.” Yamamoto was an admiral, I believe.
You’re right – I was confusing the two, perhaps because Admiral Yamamoto is on board the ship Yamato in the anime (as he was in real life in 1942, when the story takes place).
They use a very clipped “HAI!” or “Ryokai” which can be translated as “Roger”.
“Hai” is more used when accepting orders of the shouty - drilly type. “Ryokai” is more used for one on one instructions or orders and particularly on the phone.
I only remember hearing a curt Hai! whenever I noticed the JMSDF members in formation while I was stationed in Japan.
Regarding the ranks, here is a nifty site explaining them (there’s a funny misspelling of Petty Officer). The members of the JMSDF wear Navy-style uniforms with Navy-style rank insignia, the members salute both their officers and officers of allied nations’ navies, so I think it’s pretty safe to say they function as a navy.
With all deference to Sleel’s experiences, I’ve never had anyone get uptight if I describe the SDF as “military.” People are often curious how my own background relates to the way Japanese forces operate. A few weeks ago in Kyoto, I had to unravel the Japanese equivalent of my last rank for a group of people we met in a bar.
Except for the members themselves, who pride themselves on being a modern, professional military, and conduct themselves as such!
Well, like I said, I don’t know anyone who is actually in the 自衛隊. Every time I’ve ever directly compared it to the military of other countries, or pointed out that it’s one of the largest and most powerful armed forces in Asia, I’m told basically, “Oh, no, no, no, our forces are only for self defense. They’re not a military. It’s completely different. You don’t understand the situation.” Whatever, guys.
I’ve just been re-watching a few episodes of Zipang, and I’ve found that there’s another possible response to a direct order, which is the simple “Ha” (= a relatively informal “yes”). It’s pretty common as a response on the 21st century warship “Mirai”, but it’s also used in episode 4, when an Imperial Japanese Navy officer responds to an order from Admiral Yamamoto to investigate this strange warship, carrying the Japanese naval flag, that was seen before the Battle of Midway.
So “Ryokai” seems to be a response particularly used in a battle situation, where mutual communication is important, and “Hai” is still formal, but “Ha” is informal – though it can carry the connotation of the recipient thinking a little about how to carry out the order, rather than carrying it out as a knee-jerk response.
“Rajaa,” the romanization of “Roger,” as in “Roger that!” can also be heard, at least in anime.
Reminds me of Trekkies who insist that Starfleet isn’t a military force, even though it’s blindingly obvious that they are 
Diceman: Didn’t both Kirk and Spock refer to Star Fleet as the military?