You’re going to have to get over at least a little bit of your self consciousness, because it will be hugely beneficial for you to ask for help.
So, first - go. Everyone above is right, getting into the habit of going to the gym on a regular basis is the most important.
Once you’re there, the people at the gym will show you the basics of how to use the equipment - that’s their job. Most gyms will give you a basic intro-to-the-equipment free, if you ask. This will be true for both the cardio machines (treadmill, stationary bicycles, elliptical machines, etc.) and the weights. While the books are going to be very useful, your gym will probably not be exactly like the pictures in the books (close, but not exactly), so the tour/demo will be very useful.
For classes, pick something that sounds interesting, show up early, find the instructor, and tell her/him that you’re new. A good instructor wants to know. They might also ask if you’ve ever had (or currently have) any injuries and what other type of exercise you’ve done. They may also go over a quick rundown of what to expect, show you what equipment you might use that day, show you a good place where to sit or stand so that you can see them during the class. During the class, they may check in with you from time to time to see if you’re doing ok. If you’re not, be honest, they will likely have adjustments that you can make to whatever it is you’re doing. While you’re in the class, try to participate as much as you can, and ask for help (it’s ok to ask after class as well if you’re more comfortable doing that).
No matter what you decide to do, write things down. If you end up using weights, write down what exercise you did, how much weight you lifted, and how many times you lifted it. If you do cardio, write down what machine or exercise and how long you did it. If you’re in a class, write what class it was and anything notable that class.
You should wear clothing that you can move in and isn’t going to get in your way. Bring water, a towel or two, possibly toiletries (it depends on the gym and whether you’re heading straight home afterwards or not), and a notebook. Before or after dinner is something that you’ll need to figure out through experimentation. (I can’t exercise when I’ve eaten, but I know some people who can’t unless they’ve had a meal or at least a largish snack first.)
Good luck.