All I know is what I see on TV.
Sometimes, I see the defense giving closing arguments last. Sometimes, I see the prosecution doing it. And sometimes, I see one and then the other and then the first one again.
Does this vary by state? Does it depend on whether it is a criminal or civil trial?
It seems to me that the prosecution ought to be given the last word since it has the burden of proof. But what does the law say?
Generally, the prosecution does get the last word, for the very reason you describe: they have the burden of proof. However, they don’t always have to. They often have a certain amount of time, and they can use it all prior to the defense, or they can split it up and get the last word.
I thought the defense got the last word. Yes, the prosecution has the burden of proof but since you are innocent till proven guilty I thought deference was paid to the defense where possible.
Since the burden is on the prosecution, they are given more opportunity at the last to make their case. Also remember that at the end of the prosecution’s case, the defense can make a motion to dismiss the case on the grounds that the prosecution hasn’t established a case against the defendant. So everyone gets their chance, just at different times during the trial.
The prosecution gets the closing statement IIRC.
One exception I just ran across: Minnesota. The defense gets the last word there, apparently, or did as late as 1999: there was a bill on the floor to let the prosecutor have a rebuttal after the defense’s cosing argument.
The judge does. Always 
In US courts the prosecution is usually given the last word, plaintiffs in civil cases get the last word. This is rebuttal, and nothing that does not “rebut” is technically allowed. Other countries, and I believe the UK is among them, allow a criminal defendant to have the last word.
… just to expand a touch on these last few posts that mentioned rebuttal…
In most places (in the US anyway) the Prosecution goes first with their closing argument (or the Plaintiff in a civil case). Then the Defense goes with their closing argument. Then the Prosecution gets to do a rebuttal, which is generally supposed to address only points made by the Defense in their closing.