Groan!
Definitely the wetsuit, it’ll extend your season and increase your safety margin.
A problem with cold-water kayaking is that to be safe, you have to dress warm enough to be in the water, even though you generally won’t be in the water.
I particularly like wetsuits with “smoothie” panels, which protect from wind chill. The “smoothie” is porous next to your skin but nonporous on the outside. The extreme of this is called a “steamer”, which you let some water in to begin with and your body heats it up and you can go in and out of the water many times without losing much heat. Great for windsurfing, probably overkill for kayaking except in cold weather.
Find a good place to learn and practice Eskimo rolls. The hardest part of this is being strapped in, upside down, under water, and OK with that. But when you need one in a hurry, it’s a REALLY nifty thing to be able to do.
Don’t discount the fun of a little white water. I’ve never whitewater river kayaked, I’ve only been terrified by what I’ve seen kayakers do when I’ve been whitewater rafting. But I have taken touring kayaks in the Great Lakes and at the coast in North Carolina. When the waves are right (2 to 4 feet and not too gnarly) it’s terrific fun to surf them, and surprisingly easy. You don’t go perpendicular to the wave, the way you’d expect, but put the craft parallel to it. As soon as the wave starts pushing you (sideways), do a high brace, putting the paddle blade on top of the wave, and lean into the wave. The paddle will sit on the top of the wave like it’s a rock, and this is a very easy position to hold as you go screaming toward the shore. Screaming like a little girl, if you’re anything like me.
The fun part is the little surprise when the wave finally slips beneath you. That high brace paddle blade is no longer sitting on a rock, and sinks in, and you roll over. (This is a great time to have your Eskimo Roll mastered, but if not, just pop out and stand up, grab an end handle, and wade to shore with the craft between you and the shore.) The SECOND time, you’ll have learned to rebalance and relinquish the high brace just before the wave slips under. 
Sea kayaks (aka touring kayaks) are great in shore break, when it’s not too big. The first time I tried it, I was really impressed as I watched the bow of the boat pierce the wave and slip right through it as though it wasn’t even there! Unfortunately, not so my chest. WHACK. So, the second time, I made a point of leaning forward to break the wave with my thick skull.
Have fun, and good luck!
Oh, kayakers always have whistles. I guess it’s safety gear, but I’ve never had reason to use one. They don’t work well when upside down under water, for example.
Important note: be sure NEVER to carelessly tuck the skirt strap/handle under the skirt. When upside down, under water, and not OK with it, it is rather reassuring to reach forward and actually find that strap, to pull the skirt off and exit the boat. As a thick-skulled individual, I make a habit of always grabbing that handle to reassure myself, before picking up the paddle. So far, that’s worked.
