Is there an umbrella term to cover the punctuation marks ‘?’ and ‘!’
(And I want to add a ‘?’ to the end of that first sentence, but it just looks so wrong.)
Yes, there is. It’s “punctuation marks.”
And yes, there should be a question mark at the end of your sentence. To avoid the wrongness, I would have written it, “Is there an umbrella term to cover punctuation marks like the question mark and exclamation point?”
Of course there should be a question mark, but given the way I’d phrased the question, I couldn’t bring myself to put a question mark after an exclamation mark in quotes.
As for calling them ‘punctuation marks’, thanks for the sugggestion, and I guess I was imprecise in my OP. Your suggestion covers a whole range of other symbols, such as comma, semi-colon, quotation marks. I’m looking for a term which refers only to question and exclamation marks. My first thought was ‘interrogative’, but that doesn’t work for the exclamation marks.
The phrase “terminal punctuation mark” seems both appropriate and popular if you’re willing to toss the period into the mix. I doubt there’s a broadly-used term for “terminal punctuation marks, not including the period.”
Tall terminal punctuation marks.
Aren’t they “the interrobang” when they’re used together!?
Perhaps, but an interrobang is a non-standard punctuation mark in its own right.
Not exactly – an interrobang is a single mark resembling an exclamation mark and a question mark overlapping at the dot, not just the two marks side by side.
You’d probably have to include the ellipsis (…) as well.
But isn’t an ellipsis supposed to end with a period if it’s terminal?
In that case, the ellipsis is not terminal…
Actually, that was my point (that the ellipsis is not terminal), but I phrased it rather poorly, didn’t I? 
Just really really sick.
Going up a level of abstraction, punctuation marks in general are also logograms (aka ideograms) - symbols that convey a whole concept or whole word.
I always call them end marks.
There are different standards. The Chicago Manual of style calls for four dots, but others don’t. What always feels weird to me is using punctuation marks after abbreviations like A.D.; that just seems a bit weird (I still use them, of course).
Valete,
Vox Imperatoris