I’ve actually been meaning to post a mini-rant about this, because many posters here don’t seem to know this “rule.”
The first thing to keep in mind that setting words in type is not computer programming. The latter has to be rigidly logical and precise, the former is intended to convey information to the reader in the easiest and most convenient manner. Conventions in aid of that process have evolved over several centuries of producing printed material, but unfortunately many of them seem to be losing ground to the (unrelated) rules of computer programming, HTML, XML, etc., which more people learn these days than typography conventions.
The reason to put an open quote at the start of each new paragraph is simply to remind the reader that you are still quoting. Without that reminder, the reader might assume that the new paragraph is you speaking again. Yes, if that were the case, there should have been a close quote at the end, but it’s easy to assume one was there when it wasn’t.
And obviously, if you’re quoting multiple paragraphs from the same source, you don’t close paragraph 1, open paragraph 2, and so on, because that is the convention for quoting a different source.
The argument against following programming style is that if you use only one open quote, the reader has to search through the next few graphs and find the close quote to figure out where the quoted material ends. It’s distracting. And it can be particularly hard if you close the quote in the middle of a paragraph and add some of your own text at the end of the graph.
So please, for the sake of your readers, follow the proper typographic style, not programming practice, and put an open quote at the beginning of each graph, and a single close quote at the end.
It’s the right way.
(BTW, commas and periods go inside quotes even if they’re not part of the quoted material. I hate seeing a period outside a quotation mark, hanging out there all alone and lonely.)