I say read them all, but I will tell you what my husband did. He had read the first two, and maybe the third one years ago (he was studying German at the time, and wanted to practice – the library had copies in German so he picked them up). Recently our niece, who lives with us, wanted to watch all the movies in anticipation of OotP coming out, so we watched one a night and my husband really enjoyed hanging out with us and watching them. He went and saw the movie with me as well. With Tiff and I filling him in on little plot points, and the two of us talking about what might happen in book 7 non-stop, he decided to skip ahead and read books 6 & 7 along with us, because he “just needed to know how it ends”.
Yeah, he missed a lot of little things, even with us filling him in, but he was fine, since he would never be a die hard fan anyway. He also skimmed all Quidditch and relationship stuff, because he didn’t care about that.
So it can be done if what you are wanting is to just know how it ends. But you’ll miss a lot of the delightful little details JKR puts in her books, a lot of the foreshadowing (the whole Aberforth thing seemed way too coincidental to my husband until I told him the bits mentioned in previous books, for example), and some of the charm. So depends on why you are reading them and what you want to get out of it!