Emergency Blanket. Made of aluminum and folds down to the size of a small wallet. Line the bottom of your sleeping bag with it. I’ve done some cold weather camping in Yellowstone. My problem was keeping my back and feet warm in the tent. I throw a pack underneath my feet so it doesn’t touch the ground.
You can also heat up water and pour it into a Naglene bottle (don’t do this with cheap plastic canteens), wrap it in a spare shirt, and put it at the foot of your sleeping bag. This can keep your feet toasty all night. (We used to do the same thing with a hot rock in the coals of a fire, but this assumes both that you have a fire, and that you’re patient enough to make sure that none of the coals get stuck in a crack in the rock and set your bag on fire.)
There are other techniques to stay downright warm even in lack of proper equipment, such as wattlework windbreaks, warm coal beds, and chimney beds–I’ve spent nights in the desert with freezing temperatures with nothing more than the clothes I’ve had on and been quite comfortable–but I assume that this is more work than you want to put into it, and at any rate are against the grain of modern “no trace” backwoods camping. If you’re interested in that sort of thing, though, Olsen’s Outdoor Survival Skills is a good introduction that isn’t based on military-type survival and evasion mentality.
Stranger
OK, just wanted to inject a little reality into the conversation. I’m an Okie and I’ve camped in the Arbuckles in January. You’re not talking Arctic or Rockies conditions, here.
As an example, the weather in the area for this weekend will be in the upper 50s during the day and the lower 40s at night. Windchill can be bad, but probably won’t drop the temp below freezing.
The highest elevation in the Arbuckles is around 1400 (or 1700, my cites varied, but 1400 is closer to what I’ve seen) feet; average in Oklahoma is 1300 feet. Those are some dang mountains, now! 
Unless you’re planning a hiking trip, you’ll be camping in the campgrounds about 50 feet from your car, and within a few hundred feet of lav buildings. There’ll be rangers within a few miles; if the weather’s decent, they’ll be by your campsite to check up on you. (Lots of kids and airforce brats go to the Arbuckles for parties, although not usually in January.)
Check the weather before you go, because in January you could be looking at temps anywhere from 20 to 60 degrees. (Daily averages: high 53, low 31, avg 42 ; from here. I’d worry more about rain than cold, myself. It will probably be windy, so be sure you have a windbreak layer.
Definitely do the groundpad. The first time I went (during a very cold winter, with actual snow on the ground at the time [fairly rare]), I had a cheesy, slumber-party sleeping bag with no pad. That got cold, damn sure. No life-threatening danger, but I didn’t sleep too well. A good sleeping bag is next priority. The tent thing depends on how windy/rainy it is. We often just do the bivy-sack thing.
Be aware that you’ve got a good chance of being in a fire-ban zone, so don’t count on big roaring campfires. (Take your own wood, though, because there’s not any down there.) If it’s a limited fireban, you can still use charcoal in the grills. When we went in January, it was too honking cold for the rangers to be out, so we built a woodfire in the grill and cooked escargot. Fun times!
Oh yeah, and there’s no liquor allowed in the park, so keep any booze under cover in case rangers drop by. If caught, you’ll lose it and probably pay a fine.
That’s all I can think of at the moment. You’ll have fun! Don’t forget the moonlight, midnight hikes - the best part of winter camping.
Get a hunk of Ensolite (closed-cell foam, you can buy a sleeping pad made of the stuff for a few bucks) and cut a hunk big enough to sit on. Take it with you for sitting on your snowbench when you’re cooking, or kneeling when you are digging your shelter, or standing on when…you have to stand there for whatever reason. Works like a charm! Reusable, pretty indestructible and if you lose it it’s no pain in the pocketbook.
I spell towel “HAT” 
Second (fifth?) the advice on peeing as soon as you feel that you need to. You will feel sooooo much better afterwards it’s amazing. And there’s something kinda neat about crawling out into a still winter night, tramping a few steps away from shelter and piddling on a tree.
Pooping, on the other hand (well, not on the hand exactly but you know what I mean), is a little bit of an adventure the first few times. We had to pack everything out (yes, everything), so the ole bathroom kit included a couple of Hefty bags, paper, campsuds and a little bottle of hand sterilizer. And baby wipes. My god babies butts have it good with those things. Anyhow, totally dropping trou in the snow is a balancing act.
Wow. I got more advice than I expected. It seems though that I was expecting it to be a bit colder than it actually will. I figured it would be on average 10 to 15 degrees colder than here, and there would likely be snow. I looked up the climate on Weather Underground (and then read redtail23’s post, which agreed with my findings) and it seems the average high for late January is around 50 while the average low is around 32. Even the record low is only 15. That’s pretty cold, but nothing extreme. Comparing it to the climate here, it’s about 5 degrees colder on average, although for some reason we have colder record lows. Snow is rare. I can expect pretty much the same as here.
So I just have to make sure I don’t end up in the same situation as last time I tried to winter camp, especially since this time I won’t be near a car. The plan is to go hiking to wherever we sleep. It won’t take much to be more prepared than last time though. I don’t think I even had a sleeping bag. (I wasn’t expecting it to be as cold as it was.) So I basically just need a good sleeping bag and pad, some warm (and maybe water resistant) clothes, and protection from the rain. That should be easy to achieve. You guys gave me a lot of ideas. Thanks.
I’ve winter camped a dozen times or so. Cross country skied in. Deep snow.
Second, third and fourth that you must have a thermorest or other good pad to insulate you from the ground.
A big consideration is the depth of the snow. Is there any snow at all?
Keep your boots warm. Be them hiking boots or ski boots, you may want to sleep with them in your bag. A bit uncomfortable and clunky yes, but much better than trying to put on frozen boots in the morning.
At the very least, keep your boots in your tent close to you.
Use your coat as a ‘pillow’ standard practice there. But as it helps keep you warm, you also keep the coat warm.