[On preview, I see that some of my post is redundant with earlier posts, but I’ll go ahead and post it anyway.]
The standard American style is for periods, commas, and semicolons to be placed inside the quotations marks, as in your first example. Other (larger) punctuation marks, such as exclamation points and question marks go inside if they’re part of the original quotation, and outside if they arent’t.
I’ve heard that the reason for the distinction arises from the problems a small mark like a comma or period, sitting out by itself at the end of a line, could cause pressmen, back when type was set by hand. Perhaps, but I’m dubious.
These days, many people argue that it is illogical to put a comma or period inside the quotes when it wasn’t in the original. I suspect that this argument has become more prevalent in the age of computers, when a misplaced comma or period, and what is inside and what is outside of quotation marks, can have a serious impact on the proper functioning of a program. But outside of writing code or technical manuals, this argument carries no weight, IMO.
For me, it just looks bad. These marks usually cuddle nicely at the end of a word, and the eye doesn’t expect to see them hanging out all alone outside the quotation marks. It’s just ugly.
Of course, depending on who or what you’re writing for, there may be established style rules. If you don’t have any imposed, find a set you like, and stick with them.
British usage apparently differs from American, but I’ll refrain from attempting to speak authoritatively on this, not being British.
(BTW, why is this an embarrassing question?)