The way a typical surge protector works is it has MOVs (metal oxide varistors) between the hot, neutral, and ground wires. When a surge that is high enough in voltage comes in, the MOVs basically “switch on” and create a short circuit, which snubs the surge. The MOVs are typically destroyed when this happens.
Not all surge protectors are created equal. Some have more MOVs than others. The cheapie ones may only have a single MOV, where a good one will have MOVs between all three wires. A really good surge protector will have some sort of visual indication (typically a small light) that lets you know if your MOVs are still good. As I said, MOVs are typically destroyed when they clamp down on a surge, so if you don’t have any sort of indicator on your surge protector, then you have no way of knowing if the MOVs already stopped a surge and were destroyed or not.
Surge protectors will have a joule rating. This is how much energy they can dissipate from the surge. The higher the number the better. A typical power strip type surge protector will have a joule rating somewhere from a few hundred to maybe a couple of thousand. A fancy whole-house surge protector (which gets installed at your breaker box) may be rated up to ten thousand or more joules, though I have seen some that weren’t much better than a power strip.
The thing is, a lightning bolt can contain a few billion joules. A direct strike from a lightning bolt is going to blow any surge protector that you can typically afford to buy into smithereens. Hence jnglmassiv’s advice to unplug the surge protector. While there are ways to protect a building from direct lightning strikes, the ones that really work are extremely expensive and are typically far out of the range of your typical home owner (google things like Ufer ground, Halo/Ring ground, and Faraday cage if you want more details).
While there isn’t much that you can do to protect yourself from a direct strike, when lightning hits power lines, it creates a spike/surge that can travel for miles. While the energy does get spread out and dissipated, the freaking huge amount of energy in a lightning bolt means that even at the distant edge of where the surge can reach, it can still easily contain more than enough energy to completely fry your electronics.
Several miles from the strike, you can still get surges that are several hundred volts. Sure, it’s a tiny fraction of the several million volts that were in the original lightning bolt, but when your electronics are expecting 120 volts, a voltage that is double or triple that can really cause some damage. This is where turning off the surge protector helps.
High voltages can arc across air gaps, so the closer your surge protector is to the original lightning strike, the greater the chance you have of the surge’s voltage jumping across the switch contacts. Again, completely unplugging the power strip gives you much greater protection. Electricity arcs about 1 cm for every 30,000 volts or so, which means that the very short distance between switch contacts can be bridged by only a few thousand volts, where it is going to take a much higher voltage to jump from the wall outlet to the power strip if the plug is a foot or more away from it.
Again, lightning is so ridiculously high in voltage and energy that it already arced across several miles of open air (that’s what a lightning bolt is, just a big electrical arc), and it’s not uncommon for lightning from a direct strike on your house to jump several feet from one place to another inside your house. So again, nothing can truly give you 100 percent protection from a direct strike, but unplugging the surge protector is going to make it very difficult for anything except a direct strike to cause damage to your electronics.
The MOVs in your surge protector also don’t immediately turn on when the surge hits. They have a minimum clamping voltage. In this case, the lower the number the better. A good surge protector will be around 300 to 400 volts. A cheapie might be more like 600 volts. This means that if you are far from the lightning strike and you get a surge that is 250 volts, the surge protector won’t do anything to protect you from it, even though your devices are expecting 120 volts and could easily be damaged by 250 volts. This is a case where just turning off the surge protector will prevent a smaller surge/spike like this from getting through and causing damage.
TL/DR: Switching the surge protector off helps, unplugging it completely is much better.
ETA: And if you are shopping for a surge protector, you want the highest joule rating you can get within your price range and the lowest clamping voltage. Typically look for 500 to 1000 joules, 400 volts or lower. If you are looking at a whole house surge protector, those will also typically have an amp rating, which you will want to be 50,000 amps or greater (which is the fault current from the surge, not to be confused with the normal operating current rating of your circuits, which will typically be 15 to 20 amps, or your entire house, which will typically be 150 to 200 amps).