Ohio company at heart of blackout
Toronto Star - November 20, 2003
A Homer Simpson-like chain of errors at an Ohio power company triggered events that plunged 50 million people into darkness August 14, a U.S.-Canada task force has found.
FirstEnergy Corp.'s failures ranged from the simplest procedures — such as cutting trees growing too near its power lines, or telling its neighbours that it was having problems — to the more complex, such as having emergency systems fail without setting off any alarms.
And although it broke a string of industry rules, U.S. Energy Secretary Spencer Abraham conceded that they are all voluntary and said he'll push for a mandatory U.S. reliability rules.
But Canada's Natural Resources Minister Herb Dhaliwal ducked the issues. He said electricity is a provincial jurisdiction. And Ontario does have mandatory rules, and penalties for breaking them.
The task force refutes assertions by U.S. politicians that the blackout originated in Canada, and says there's no evidence it was caused by sabotage.
Its interim report released yesterday gives no such comfort to FirstEnergy, a holding company with seven operating Ohio utilities.
FirstEnergy managed to turn a local problem into an international catastrophe in part because it failed even the simplest of tasks. For example, it neglected to trim trees growing under its transmission lines. In summer, hot weather and heat caused by heavy power flows makes wires expand and sag. If the wires touch trees that have been allowed to grow underneath, the line shorts out and fails.
Utilities are supposed to assume some lines will fail — at some point — due to weather, storms or accidents, and have systems in place to nip problems in the bud. But FirstEnergy's computerized alarm and monitoring systems started to fail about 2:15 p.m. The company's information technology department knew of the problem, and was trying to correct it, but never alerted the operators routing power through the wires. They remained unaware that they no longer had a safety net.
Meanwhile, demand was growing, and as it did, one transmission line and then another began to fail.
FirstEnergy is supposed to be monitored by a regional agency called the Midwest Independent System Operator (MISO) that oversees utilities in the Midwest. But MISO monitoring systems had also failed August 14, so it didn't know things were going wrong at FirstEnergy.
FirstEnergy only woke up to the problems at 3:45 p.m.
At that point, blacking out an area around Cleveland and Akron might have eased demand on the system and reduced the stress, but no action was taken. Nor did FirstEnergy alert other utilities or neighbouring system operators they were in trouble.
FirstEnergy's lax response to the budding emergency was not surprising.
The company "relied upon on-the-job experience for its operators in handling the routine business of a normal day," the task force says. "But (it) had never experienced a major disturbance and had no simulator training or formal preparation for recognizing and responding to emergencies."
Dave Goulding, who heads Ontario's Independent Electricity Market Operator (IMO), says his controllers routinely start to make contingency plans if they see a problem developing, trying to figure out their next move in advance should the situation deteriorate further.
But the task force says FirstEnergy didn't go through that exercise as a matter of course, so it was caught flatfooted as one failure piled on the next. In any case, the report says, anything FirstEnergy did once they realized the problem might have been too late.
With its system already staggering, another key FirstEnergy transmission line failed at 4:06 p.m., which sent power flooding on to other heavily loaded lines.
At the time, power was flowing through Ohio on its way to northern Ohio, Michigan and Ontario. When the power couldn't find its way through central Ohio, it surged east and west, around both ends of Lake Erie, some of it crashing through Ontario.
The surge caused circuit breakers to trigger on more lines as they became overloaded, breaking connections between Ontario and the northeastern U.S. from the rest of the continent. Unfortunately, Ontario and the northeast were importing power at the time. Local generators couldn't meet the demand once imports were cut off, and were already under stress from the power surges on the transmission lines.
As demand far outstripped the ability of generators to meet it, voltages became unstable and generators had to disconnect themselves from the grid to protect their equipment from damage.
With huge chunks of the transmission system knocked out, and generators disconnecting in droves — a total of 92 plants shut down in Ontario — the province joined the northeastern states in the dark.
Despite the blackout's U.S. origin, Dhaliwal said the solution is not to sever Canada's power links with the U.S., saying imported power often keeps the lights on in Ontario.
Goulding, of the IMO, said what's needed is a new "electricity reliability organization" that can set and enforce standards on both sides of the U.S.-Canadian border. He sits on a steering committee that's already planning such a body.
Tom Parkinson, chief executive of Hydro One, which runs Ontario's main transmission system, says the report won't affect his company. It spends $700 million a year on maintenance.
"It's business as usual for us. We fully comply with the rules now," he said in an interview.
But Ontario's Energy Minister Dwight Duncan said the province can't be complacent.
"We have a lot of work to do on the transmission side and we're prepared to take it," he said. "There are significant challenges on that side."
David McFadden, a Toronto energy lawyer who sat on a working group that was part of the task force, said the report shows the importance of mundane issues, such as tree cutting.
He rejected the notion that deregulating electricity has pushed some companies to put profit ahead of reliability. "You'd have this sort of thing happening every year" if that were the case, he said. "I don't believe that's what's happening. You had a very serious management issue in one company."
Keith Stewart of the Toronto Environmental Alliance said the report is one-sided.
"They just take the demand for electricity as a given," he said. "They're ignoring the role of conservation, and measures that can be taken to reduce demand."
with files from Robert Benzie