(I am referring to the electric grid of the U.S., but I supposed the same can be said for most other countries.)
Was reading this story about “preppers” who believe our nation’s electric grid will go down for an extended period of time, resulting in mass chaos. Is this likely? Do they have a point? Or are they just taking advantage of (some) people’s paranoia?
Our electric grid is very old, and not just vulnerable to deliberate attack, but vulnerable to failure from accident or poor maintenance as well.
There were some stories a bit ago about how deep russia managed to hack into parts of our grid, but there are quite a number of physical vulnerabilities as well. There are substations that have no active security at all. There are natural gas plants that can be messed with, and what scares me the most, thousands and thousands of miles of high tension lines that are just strung out across completely uncontrolled areas.
If I were wanting to take out the power grid, it would not be that hard to do quite a bit of damage with just a couple people and some pretty nominal funding.
I agree with this.
I also think that the grid is particularly vulnerable to cascade failures, at least in the northeast and southwest. The grid could easily take out a huge chunk of itself just because of a normal everyday failure, with no intentional sabotage involved.
For those who don’t know what a cascade failure is, imagine you have four power companies, A, B, C, and D. A can’t produce enough power to supply all of its customers. B can, but doesn’t have enough excess to help out A. C produces enough for itself and has plenty of excess. D is like B, producing enough for its own customers but no excess. With all of these tied together, the excess production from C can be used to help power A’s customers, so all is well. But let’s say that a major generator in C breaks. Now, C can’t supply enough power for both its own customers and A’s customers, so C’s other generators overload and trip offline. B and D can’t support the extra load either, so they trip offline, and A couldn’t power all of its customers alone either so they trip offline as well. A, B, C, and D all go dark because of one failure in C. This is a cascade failure, where the failure cascades to other systems.
Every time we have a major cascade failure, folks try to improve the grid so that it can’t happen again. Using the above example, if a major generator in C breaks, instead of having the failure cascade to all of the other systems, just isolate A, where the overload actually is, and A goes dark. But now B, C, and D have enough power to keep running, so they don’t go dark. This prevents the failure from cascading all the way down the line.
Exactly how this is done gets very complicated and is a bit beyond the level of a message board post, but basically you need systems that can react very quickly to faults.
While power companies have made major improvements in systems like this in the past few decades, you don’t really know how well they work until you get a major failure. The last major cascade failure in the U.S. was in 2003, when high current loads caused high voltage lines to sag from the heat (it was a very hot day, which also contributed to the failure) and the lines basically shorted out. This single failure cascaded all the way through the northeast U.S. and up pretty far into Canada.
We won’t know how good the newer protective systems are until we get another big failure like this.
You wouldn’t have to do much intentional sabotage to get a similar type of failure, at least not in the northeast and southwest, where power systems are strained pretty close to their limits on hot summer days.
Doomsday scenarios tend to rely on people being idiots, but people aren’t idiots. They aren’t going to run in circles screaming and panicking. They are going to do what they can to get the power back on. After the 2003 cascade failure, power was restored to a lot of people in just a couple of days. The entire northeastern U.S. and parts of Canada did not descend into chaos, and civilization in these areas did not collapse, though admittedly in some areas there was some significant looting and some chaos. It took a few weeks to get power restored to everyone, but people managed to cope.
The folks who sold portable generators also made a lot of money. 
Taking down the grid probably wouldn’t be that difficult. Keeping it down for everyone isn’t going to happen, though, at least not easily. Someone could theoretically do a lot of damage with an EMP attack using nukes, but then, if we’ve gotten to the point of using nukes, we’ll have more important things to worry about than our electrical grid. Anything less than that, and the damage to the grid is going to be repaired fairly quickly.
The best protection is the people in charge. I met the guy that single handedly stopped that cascade failure from going further into southeast Ohio, and prevented it from going further. If he hadn’t been there, or hadn’t been paying attention and known how to react to what he was seeing, it could have spread even further.
My concern is more all the long high tension power lines around with no protection. I really don’t want to get too far into it, for many reasons, but I would think that damage to those towers could be catastrophic and lengthy to repair. There are some in places that I could take down with a hacksaw. Not that a hacksaw would be fun to use for that sort of project, but I wouldn’t be interrupted for the duration of sawing away for hours and hours either.
Those high tension wire corridors make excellent rifle ranges. Those things go on a long, long way.
How many remember the attack on the substation in California?
Did you read this part of the OP’s article?
How much do you want to bet that a company which is focused on profit does NOT stockpile those extremely expensive transformers?
http://energyskeptic.com/2015/power-transformers-that-take-up-to-2-years-to-build/
The only way you are going to get an EMP attack powerful enough to take out large transformers is with nukes. If we’re using nukes, then we’ve got a lot more to worry about than just our electrical grid.
I happen to design control systems for a living. Most of the stuff I design doesn’t go into power systems (it’s mostly industrial control), but it’s the same sort of stuff. Viruses and hacking is a concern. The thing is, if you hack into a control system, you can wreak a lot of havoc and shut things down, but with a power system, you probably aren’t going to do any permanent damage. All you’ll do is cause things to shut down. This is more of a concern in the places where I work, which do things like extremely hazardous chemical production. A phosgene release, for example, would be a much bigger problem than just shutting down the power.
A lot of control systems manufacturers have become a lot more aware of security issues in recent years. For a while, a lot of control systems were getting fancy and were connecting to all kinds of things. This was great for company management. Managers half way around the world could get live reports off of the control systems and could see what was going on. The problem is that when you have a connection to the outside world, you can potentially get hacked. These days, operating systems are getting more robust. My own company uses a proprietary home-grown operating system that only about half a dozen people in the entire world understand, so it is almost impossible to hack into (security through obscurity does actually have some benefits). Commercial off-the-shelf (COTS) operating systems are getting more robust. Many of them have digital signatures, so if anyone tries to hack into one, any task they try to load won’t match the signature and won’t get loaded. Many control systems are also going to an “air gap” model, where the stuff that does the actual control is physically separated from the outside world. If there is no physical connection at all to a network from the outside world, it is physically impossible for anyone to hack into it.
The weak link in control systems has often been the human-machine interface, or HMI. The easiest way to make a pretty HMI is to use Windows operating systems, and Windows is most definitely not a secure OS. Again, an air gap between your HMI systems and the outside world helps a lot.
Even if someone were to hack into the actual controls, most controllers can be reprogrammed back to their original state fairly quickly. You also aren’t going to be able to cause much physical damage. You could shut down the grid fairly easily, but without any physical damage, the grid would also recover fairly quickly.
I worked for Ohio Edison a few decades ago. If they stocked any large transformers anywhere, I wasn’t aware of it. Those big transformers don’t fail very often, and when they do, the power can usually be routed around the failure until a replacement can be built.
There are thousands and thousands of substations all around the U.S. Sure, you could break into a substation and cause all kinds of damage, though if you don’t know much about high voltage electricity you could also easily kill yourself in the process. But you are probably only going to make a small area go dark. If someone is simultaneously attacking thousands and thousands of substations all across the U.S., then we’ve got a lot more going on than a simple isolated terror event.
As for solar flares, those are potentially a problem. Power systems are protected somewhat against solar flares and they try to shunt the energy harmlessly into the ground, but industry experts are aware that the current protection systems aren’t enough. A solar flare probably isn’t going to make the entire U.S. go dark though. One hit in 1989 and large areas of Quebec went dark. Most of their customers had power restored in less than half a day though.
In the 1850s, the world got hit by a very large solar flare. This was called the Carrington Event. If we got hit by a flare of that magnitude, a lot of the world probably would go dark. How long it would stay dark is debatable, and we won’t really know the answer to that unless it actually happens.
A couple of years ago, a group of electrical utilities planned to jointly stockpile transformers.
Interesting, and good to know.
Thanks!
Grid Assurance is the company they set up.
Would a cyber attack on our “infrastructure” be similar to what we (in conjunction with others) did during the Stuxnet attack? That was something that attacked control vulnerabilities to industrial controllers (Siemens?)?
Stuxnet attacked Siemens programmable logic controllers (PLCs). One of the things that Stuxnet was able to do was take control of the variable frequency drives (VFDs), which control the speed of very large motors. Stuxnet would reprogram the VFD’s communication so that it could no longer be controlled, and would then change the speed of the motor. Depending on what that motor was being used for, that could be very, very, very dangerous.
Back when I first started doing PLC programming (late 1980s), Allen Bradley (Rockwell) basically owned the U.S. as far as PLCs were concerned, and Siemens owned the rest of the world. Rockwell is still the biggest in the U.S. with I think somewhere around 60 to 70 percent of the U.S. market, and Siemens is still the biggest worldwide, with something like 60 to 70 percent of the market outside of the U.S. Schneider is third in both the U.S. and the rest of the world.
I have personally used PLCs by Rockwell, Siemens, and Schneider, among others, like GE and a bunch of smaller vendors. I even have PLCs by Rockwell, Siemens, and Schneider in the development lab where I work every day. Back when I worked for Ohio Edison (late 80s/early 90s), they had standardized on Rockwell PLCs in the plant where I worked.
PLCs are used in industrial automation, but they are also used in power systems and all kinds of stuff, like breweries, wastewater treatment plants, municipal water systems, all kinds of stuff. They are even used to control park rides, like roller coasters, swings, teacup rides, etc. They control the motors that control how fast the rides move, and control the safety systems that shut down the ride if they detect something bad. A lot of Disney rides are controlled by Siemens S7 series PLCs, the same series that Stuxnet infected.
If you can manage to hack into one of the major PLC brands like Rockwell, Siemens, Schneider, GE, ABB, etc. you can do some major damage to the U.S. infrastructure. This is why Stuxnet was a really big freaking deal for the controller industry.
Some questions: are these controllers (PLC’s?) similar or share functions with electrical contactors or motor starters for three phase power in any way?
How subject to “wear” are PLC’s? Any “wearable” parts?
How difficult is it to take remote control (hack, etc) of these devices and subject people to harm since they control so many automated services?
Stuxnet seemed to be a successful attempt to slow down Iranian uranium enrichment that was allegedly a joint effort between the Israelis and Americans. Did this in event in fact represent some kind of watershed opportunistic moment for other hackers (etc) to exploit?
A PLC is an electronic controller. A typical PLC has a CPU (its “brain”) and then a bunch of I/O cards that plug into it. Since we are talking about Stuxnet, here is a Siemens S7 series PLC:
Here is one that is actually hooked up and running:
http://www.control2k.co.uk/media/1050/p02_06k.jpg
The big thick purple wires are for communication to other PLCs or to computers running an HMI (human-machine interface, like operator consoles). Slangily, a lot of people (including myself) call this purple wire “Barney hose” or “Barney cable”, since it is purple. The Barney in this case refers to Barney the Dinosaur. The blue wires go to actual sensors and valves and such.
For comparison, here’s a Rockwell PLC:
And here’s a Schneider PLC:
The PLC and all of its I/O cards are all solid-state electronics and have no moving parts. The parts that they use are all rugged, and are designed to work over industrial temperature ranges, so they are pretty solid and reliable. Consumer grade electronics are made to last about 7 years under a fairly restricted temperature range. Industrial equipment is designed to last about 15 years under a much wider temperature range and is also designed to tolerate a lot more dust and vibration. PLCs don’t wear out very easily.
PLCs run ladder logic, which you can picture as basically the software equivalent of having racks and racks of old fashioned relays, except they are all software simulated. A ladder logic program looks like this:
PLCs can also be programmed in other ways, but explaining all of those would probably take a few more fairly long posts. But basically, just picture a PLC being able to be programmed to do very complex control algorithms (PID feedback control, for just a simple example). The PLC directly reads the sensors out in the field and directly turns on motors and valves and all kinds of stuff. PLCs are definitely not restricted to motor control.
A PLC can be connected to a VFD. A VFD is used to control motor speeds. Your typical AC motor runs at whatever the line frequency is, so an AC motor connected to 60 Hz AC power is going to run at some multiple of 60 Hz (exactly what multiple depends on how many poles, or coils of wire, are inside the motor). A VFD, as the name implies, varies the frequency. If the VFD puts out 30 Hz, an AC motor will run at exactly half the speed that it would at 60 Hz. If the VFD puts out 10 Hz, then
the motor will run at 1/6th the speed, etc.
A VFD can be single phase or three-phase.
This is what a typical small VFD looks like:
http://www.gohz.com/content/images/thumbs/0000114_75-hp-vfd-single-phase-to-three-phase-vfd_300.jpeg
Here are a couple of VFDs in an I/O cabinet:
https://university.listenlights.com/wp-content/uploads/2018/02/cover-image-1.jpg
Here are some much larger VFDs:
These look to be 3-phase. The cabinets are probably a bit taller than I am if you are having trouble picturing the size.
This diagram shows how a VFD typically connects to a PLC.
http://www.grupsautomation.com/gallery/SIEMENS%20VFD%20V20%20%20APPLICATION.PNG
Here is another example:
The VFD is the big box on the upper left. The PLC is in the middle on the right. Looks like a Rockwell SLC series PLC to me.
PLCs are very difficult to hack into, mostly because they aren’t general purpose computers. They have custom communications protocols, and the programming protocols are typically not released to the public. Before Stuxnet, most people didn’t consider hacking PLCs to be a realistic threat. Now we know better.
To the OP, I suggest reading Ted Koppel’s book Lights Out where he spent considerable time investigating the US electrical grid and it’s weak points.
According to him, there are some very old (transformers? switch equipment?) assemblies operating in the NE that have no readily available replacement, and are so large they were moved in on special railroad cars. Again, according to his book, these would take months to years to design/build replacements, and even when done, we would have trouble reinstalling as the railroad tracks don’t exist any more. If I understood it correctly, loss of this type of equipment could leave millions without power for many months. He projected substantial loss of life for these scenarios.
Experts in the field may know better, but reading the book left me with very low confidence about our grid’s robustness, and our ability to recover from a large scale equipment failure. My wife and I are (sort of) preppers, not the camo and ammo kind, but more of the fresh water and food type. After the book, we took pains to increase our “survivor” larder from a 2 week supply to 2 months. We’re continuing to increase it over time.
From the book, the short take on how to survive if you really think an EOTWAWKI event is coming? Join the Mormons. They seem to be acquiring vast grain and seed storage bunkers, and are apparently prepared to manage without normal infrastructure (phones, electricity, etc.).
Read the book. It’s definitely unnerving.
Unless you can cause the equipment to burst apart, of course…
Those are called schnabel cars and they are fascinating (if you’re patient enough) to watch.
Should a kind of EOTWAWKI event occur, I promise you that the local government will come knocking on all the prepper’s doors when it becomes obvious that the preppers seem to be doing fine living off their supplies while the rest of the population is starving. The authorities will be friendly at first, but should you refuse to cooperate and “share” your bounty, they will return with guns and you will be jailed on “hoarding” charges.
A large organization like the Mormons will be an easy target for the government authorities looking to feed themselves. Mormon supplies will be quickly confiscated “for the good of the country”.
You can see plenty of those on the roads.