The OP describes me to a T. I generally get up and read for a bit. I did last night in fact. The key for me is to distract my brain from telling me how rotten I am.
I also use the Benadryl trick.
I happen to know that one of the triggers for my insomnia is alcohol. Not always, but usually. I took the month of October off from drinking and found that I started off sleeping better, but after a couple weeks I reverted back to being awake at 3:40am. Can’t win, sometimes.
There are a lot of good suggestions for insomnia. But what I am thinking has been mentioned less. I always sleep thru the night but over 2 years ago I was basically stuck doing 2 people’s work at my job. I was so stressed - at first I was unaware of the issue - that I was having issues getting to sleep.
It got so bad that I stopped going to movies as my body would treat anything with comfort and low light - a dark theater - as an excuse to drop off.
As you try the methods mentioned above is there something that is worrying you? It could be job, family, or loved one. It wasn’t until I broke out in a skin rash that I figured out my job was pushing me and moved to lessen my load.
Good luck and keep us posted on your progress.
Don’t lie there. Get out of bed, get a glass of water, eat something very small, do something relaxing.
That’s because the shorter version of the Serenity Prayer is “f**k it!” 
I definitely have this problem, and two things work for me, neither of which is especially healthy:
But they say staring at a screen is counter-productive when it comes to sleep, so that’s maybe not the best idea.
I have reason to dislike Ambien.
Have you tried an anti-anxiety med? I have had good results with Xanax, while my wife prefers Clonapam.
Here are my go-to’s:
• Use one of the sleep relaxation videos on YouTube: soothing music and beautiful photos of nature help. Some of them last for hours. Soothing and just barely interesting enough to keep me from worrying but not so interesting, they keep me awake.
• Take imaginary walks around my childhood neighborhood. I loved that neighborhood and felt very secure there, but any serene place you remember well would do.
• There’s an OTC I use that contains herbal ingredients and was formulated specifically for waking up in the night. I was sure it wouldn’t work, and to my surprise it does.
Best of luck. You snooze, you don’t lose!
I keep an old wi-fi radio on the nightstand tuned in low to Seeburg 1000 for this reason. It is, by definition, background music, and it serves that purpose admirably.
Insomniac here. I understand there are several things that signal wake mode to one’s brain: light, movement, substances (e.g. caffeine) and thought. Avoiding them is a good thing if one wants to sleep.
My gf used to fall asleep fine, then wake up at 3 am and be unable to fall back to sleep because of “thinking”. Counting backwards would sometimes help, but usually the alarm would go off at 5:30 with her still awake.
I read about Lavender essential oil as a sleep aid. Some decent studies showed that, especially in women, lavender in a diffuser had a positive effect. I bought a diffuser and a bottle of oil. She has had a nearly amazing improvement in her sleep. It’s been about a year and she wouldn’t dream of not running the diffuser at night.
Molly Brandenburg has the most soothing voice for sleeping. Tuck your phone under your pillow with the volume down low. Can’t miss.
https://www.parcast.com/gone
It’s exactly the same with children who wake in the night. If you hear them crying or whatever go to them and settle them down; maybe pick them up and cuddle them. Whatever you do, don’t turn the light on, read to them, or take them downstairs to watch TV, because subconsciously they will remember and do it again.
The mantra is ‘be boring’.
OP again. A couple of you have mentioned stress, and yes, my life has a great deal of stress right now–my wife’s health has been a big issue for a long time and not getting any easier, our finances are a problem, I have a lot of responsibility for my young grandson, there’s some job uncertainty…well, yes, there’s stress. It doesn’t help!
I have looked into the podcasts…thanks for the suggestions. I rather like the Sleep With Me podcast, but I do think it might actually keep me awake – I found myself intrigued by the content (perhaps against all odds). I might try some of the soft music suggestions too.
I also especially liked the idea of not looking at the clock–which is easy as I don’t actually have one. just have to avoid looking at my phone. It was disorienting the other day not to look at it–I wanted to KNOW, dammit–but on the whole it was probably a good idea not to look. I eventually did drop back to sleep.
Still a slog, but thanks for the suggestions and the kind supportive words.
I’m late to the party, but as a life long ruminating insomniac, I have some suggestions for the OP (which may have already been mentioned, since I didn’t read through all of the replies). The only things that have worked for me, after trying everything, are:
A sound machine. There are many types, but mine has the option of playing the sounds of soft waves, crashing waves, babbling brooks, waterfalls, rustling leaves, etc. I switch it around from time to time, but crashing waves is my go-to.
The Dowdow. It’s a little device that projects circles of soft light onto the ceiling, that you are supposed to breathe along with. You inhale when the circle is getting bigger, and exhale when it gets smaller. The timing changes and the speed at which the circle gets bigger/smaller slows down to the pace of breathing that makes you sleep. It’s hard to explain, but IME, it does work.
Other than the above tips, the only other things that have helped are booze and medications! In my worst episodes of insomnia, I’d just embrace it—get out of bed, make food, watch tv, or whatever, but just accept that this night, I’m not going to sleep. It kind of takes the pressure off to just go with it. I hope you find some things that work for you!!
I stayed up late crying over a sad comic strip. I woke up early because the local mosque seems to be running pledge week.
I discovered long ago that simple counting up by X for various X “de-loops” my brain.
It’s so well embedded as a technique in my brain that all I have to do is count up by 5s and by the time I reach 100 or rarely 200, my brain has shifted and I’m settled down. It’s like chanting “om” but for a nerd.
Another brain-shift trick is to alternate gently tapping index fingers to thumbs. The key thing is to pause between taps. So tap right finger and thumb. Bring them back to the start position. Be sure you’re thinking about what you did. Then start the left ones. Avoid going into auto-mode. You have to consciously think about each one.
I need to have something to occupy my brain, or I tend to get to fretting, or chewing my mental cud.
A podcast or audiobook works quite well. It has to be something interesting enough to focus on, but not so fascinating that it itself keeps you awake. If I get up during the night to use the bathroom, I’ll always put the podcast back on. I’m still on a quest for the perfect headphones for the purpose - wired vs something like Air Pods (as I’d lose those the first night), but bluetooth so I don’t have to be connected to the phone, and NOT the sort that wedges itself into the ear canal itself. This does not appear to exist in the real world.
As far as the cause of the nighttime awakenings: Listening to something will help you recover from them, but if you’re not awakened at all, that’s even better. Some things that come to mind are:
During the year of my late wife’s illness, every night I would fall asleep within moments of head hitting the pillow, and every night I would wake up at 3:30. On the dot.
So I guess I also was a stress-related middle of the night insomniac.
My personal version of counting sheep is to mentally play Beethoven’s 7th Symphony. I have no idea how when or why I started this, but it’s my thing and it works.
Off topic… Whenever I see the subject line I first read “waking up in the middle of the night and urinating”. Which is something I do more often.
As someone who has suffered from both insomnia and serious anxiety, my advice to you will be similar to what RioRico has stated above in post #3. Lie down, close your eyes, and try to imagine something nice, something that makes you feel good, some image of your best self or as you would like to be. I used to collect insects, and I would at times imagine myself all alone on a field on a sunny day, enjoying the solitude in pursuing some beautiful butterfly. More recently, I have imagined myself lying in bed with an imaginary wife, who resembles (in appearance but not in personality) a certain Internet celebrity, generally as she looked in 2010. No really original fantasies, just the same images of us lying in our PJs, falling asleep nestled close to each other, with our little dog crawling in between us. You can imagine whatever relaxes you and gives you a sense of ease.
Another suggestion, one which has helped me through times of anxiety, is to watch some ASMR videos before bed. ASMR (Autonomous Sensory Meridian Response) is the phenomenon where some stimulus triggers that tingly feeling around the back of your head, the nape of your neck, etc. There are lots of ASMR content creators on Youtube, who make videos meant primarily to trigger that sensation, but which can be used in general to relax you. Some of these videos can be creepy, in bad taste (my comment enters YMMV territory here), or scary, and people do react positively or negatively to different things. But there are a lot of good personal attention videos where the ASMRist pretends to do something like give you a massage or some other kind of care. One good channel to try might be that of The Lune Innate, who is a Reiki master. This is not meant to be an endorsement of the unscientific art of Reiki, but I have watched her videos just for the calming factor; they’re full of nothing but personal attention, nice words to the viewer, and generally pleasant sensations. But to each their own.