FWIW, I definitely had the paranoia that my socks and underwear were vanishing somehow (and I really didn’t think someone was stealing them), but in the end, I decided they’d just been scattered between two many different laundry bags, etc, and a few thrown out. So, you should definitely test out the more sinister theories! But if not, even if it seems impossible, it has happened to at least one other person…
Gather some poison ivy. Carefully, with gloves on, rub the leaves on the inside crotch of the panties. Set them on top of your dresser. Be careful not to forget and put them on yourself. If they disappear, chuckle to yourself.
That gets my vote too. I had a cleaning lady years ago that “borrowed” my shoes all the time. It was the oddest thing. I’d look for a particular pair and find them no where - then they’d suddenly appear again - but the strap was adjusted to fit a larger foot.
I started to pay attention and several items of clothing disappeared over a 3-month period. I fired her and the thieving stopped.
Actually, I wouldn’t say anything else to the cleaning lady if you suspect her. If she is the culprit, you have no idea what else she could be doing, if she’s not you’ll have damaged a bond that you’ve had for years. There are tons of options you can get from places like this that will let you find out if anyone is entering your house and taking them. Just set it up to point towards your panty drawer and check it in a couple of weeks.
There’s the explanation - your local P.D. has the motto “Serve, protect, and collect panties”.
I think the only thing you can do at this point is to just stop wearing them.
What areas does the cleaning lady cover. Would she have any reason to go into your dresser? Does she dust the front of your dresser?
How dare you interrupt our wild over reactions and dangerous trap suggestions with a sensible and inexpensive alternative that would not only diffuse the whole situation, but would work well with Spring and Summer approaching? 
…next you’ll be fighting ignorance, and who knows where that might lead?
That’s what a friend of mine did when she was having the same problem. It turned out that the culprit was her landlady’s son. He was using his mother’s key to get in the house.
What was cool was when he got busted. He was doing the indignant outrage bit, then she played the tape. Sweet.
This is way better than the lip augmentation thread 
Reported.
I don’t know the age and gender of your children, but I’d do some snooping around their drawers and closets.