Power Play -- How Its New $5.9 Million Power Plant Brought A Hospital To Its Knees

Copyright 1998, The Seattle Times Co.

Last April, Valley Medical Center in Kent unexpectedly lost all electrical power. Patients had to be rushed to other hospitals, surgeons had to work without electricity and intensive-care nurses had to hand-pump ventilators for critically ill patients. The story of what went wrong hasn't been told - until now.

RENTON - One morning last April, Rally Panelo stepped to the controls of Valley Medical Center's new power plant to make history.

Panelo, Valley Medical's director of engineering, was about to disconnect the hospital from the electrical utility's power grid. Once he did so, the entire 12-building campus would run on its own electricity, supplied by a state-of-the-art co-generation system - a first not only for Valley, but for any hospital in Washington and perhaps in the country.

The occasion was historic, yes, but quiet: The physicians, nurses, support staff and patients in the hospital on April 29 were unaware. And if all went as planned, they wouldn't even notice.

All did not go as planned.

For the first few minutes after Panelo removed a fuse in the power plant's control panel, disconnecting power from Puget Sound Energy, the hospital ran smoothly on what's known as "island mode." But then, in different areas of the sprawling complex, lights began flickering, ventilation fans shut down, medical-tool alarms beeped and computer screens blinked on and off.

In the radiology department, electronic equipment started to smoke. In the main laboratory, a blood analyzer shut down. In the Medical Arts building, surging electricity melted an electrical cable onto a circuit board, shutting down two elevators and trapping the occupants between floors.

After about an hour, a harried Panelo replaced the fuse, which he thought would reconnect the hospital to the grid. Instead, the co-gen plant's computer controls sensed something they didn't like and shut down to avoid potential damage. The hospital's backup diesel generators then kicked in, but after a few minutes, they tripped off too - cutting all power to the 303-bed hospital.

Nurses and orderlies scrambled for flashlights. One nurse raced home to fetch a portable generator so that a surgeon performing a delicate vascular surgery could have light.

As the blackout extended into early afternoon, staffers chipped in to pull off the largest hospital evacuation in state history. Almost miraculously, no one was harmed as patients - some of them in critical condition - were shuttled to other hospitals in and around Seattle.

The details of exactly what happened that day at Valley have never before been reported. In fact, even the hospital's management didn't understand what had transpired until The Seattle Times discovered that the blackout had been caused not by a simple, minor malfunction - as officials were told by the man in charge of the power plant - but by fundamental problems that remain to be solved.

A close look back at the incident raises questions not only about whether Valley's $5.9 million plant can reliably supply power on its own, but about the decision to build it in the first place.

Burgers and Lotto

Valley Medical Center works hard to get attention.

Tucked in the Kent Valley - a congested traffic artery from the heart of the region's medical community on Seattle's First Hill - the public hospital uses creativity to help keep residents of King County's southern suburbs close to home.

Steered by its brash, high-profile administrator, Richard Roodman, Valley over the past decade has launched big-budget promotional campaigns, started offering McDonald's hamburgers and state Lotto tickets for sale and continually upgraded its physical plant, all while attracting more patients and keeping fees in check.

Roodman, a former door-to-door salesman, has achieved national prominence as a promotions expert who at one point as Valley's administrator became the highest-paid public official in Washington.

He's known for innovation. So when one of his senior managers proposed in 1995 that Valley could take a cutting-edge action that would also save the hospital millions of dollars, Roodman was receptive.

Ome Almeda, the administrator for support services, based his pitch to Roodman on this prediction: The federal government will soon deregulate electrical utilities, and when it does, power rates in the Northwest - low compared to most of the country - will rise. Utilities here will look to sell their power to markets where prices are higher, such as California.

Almeda - and, ultimately, Valley's elected board of commissioners - ignored indications to the contrary. For every energy expert who warns that deregulation will spark spiraling electricity prices in the Northwest, there are several others who predict rates here will remain flat for the next few years and rise only at the rate of inflation after that.

Those experts point out that there is a limited number of transmission lines for freely moving electricity around the country, which limits the amount of power-peddling that can be done. Also, they note, deregulation invariably attracts more suppliers into a competitive marketplace, and that usually drives prices down. That's what has happened in the transportation, telecommunications and natural-gas industries.

Yet Almeda managed to convince Roodman and the board that building their own co-gen plant made eminent sense. While it wouldn't supplant Puget Sound Energy entirely, it would take on a significant share of Valley's electrical demand.

"There was really an attractiveness and viability to co-gen power," Almeda said. "It had reliability and safety, and also with the ongoing changes in the utility industry, you could control your expenses."

Few of Almeda's peers who manage large physical plants in the Northwest saw such compelling promise. In the 1990s, at least four other Washington hospitals and the University of Washington had already conducted feasibility studies on co-gen plants, which use an alternative fuel source - typically, natural gas - to generate electricity and steam used for heat. Each found the economic viability of self-supplied electricity to be marginal, at best.

"Given low rates, and after you factor in acquisition costs and operations and maintenance expenses over the life of the facility, very few of these projects pencil out," said Dan Swanson, facilities director for Catholic Health Initiative West, a Tacoma-based chain of eight hospitals. "If these plants could truly save you dollars, everybody would be doing it. It'd be a no-brainer."

`Questionable assumptions'

A study done for Valley by Independent Power Associates of San Francisco reached a different conclusion. The IPA study advised the hospital to build a 3.2-megawatt, natural-gas-fired power plant for $2.88 million.

The study predicted Valley would save enough in electricity costs to pay for the project in five years, assuming electricity rates rose 5 percent a year through the year 2020.

Outside experts with no connection to the project snicker at that assumption.

"I'd be amazed to see a 5 percent increase on a regular and sustained basis," said Dan O'Brien, energy consultant with San Jose, Calif.-based Salas O'Brien Engineers. "That's just not going to happen."

Almeda believed otherwise. He arranged a five-day trip for board president Carol Anderson, a real-estate property manager, and vice president Donald Jacobson, a plasterer, to go with him to tour co-generation plants at Wellesley College in Massachusetts and Locktite in Connecticut.

Jacobson said he left on the trip "probably skeptical to start with, wondering whether this could really save us any money," but returned a true believer, ready to back the proposal. With him and Anderson behind it, the board decided to go ahead.

Word of Valley's plans caught Puget Sound Energy, the electricity supplier, by surprise. The utility requested a copy of the IPA feasibility study and retained a consultant of its own, SFA Pacific of Mountain View, Calif., to review it.

SFA Pacific found numerous "questionable assumptions" in the IPA study. The utility's consultant doubted that the plant could be built for $2.88 million, disputed that the hospital's electricity rates would rise 5 percent a year for 2 1/2 decades, noted that financing costs were ignored and suggested that the operating and maintenance expenses cited by IPA were understated.

Using what it considered more realistic assumptions, the utility's consultant concluded: "It would take more than 25 years to pay back all of (the hospital's) initial design and construction costs, and a co-generation plant would add significant operational risks."

Almeda and Roodman viewed the utility's criticism as a patronizing attempt to scuttle a good idea.

"What I came away with is they just tried to kill the project because this hospital provides them $1.2 million or $1.3 million in (annual) revenue," Almeda said. "The only thing I can say is no utility company would encourage people to develop this kind of project.

"This is a first in this area. I can only presume that if we were able to do this and make it work, then, it's not going to be too long before other people look into it."

Puget Sound Energy tried to turn Valley officials around with this proposal: The hospital could spend $1 million to erect an electrical substation, which would allow the utility to route power directly to the campus and would save the hospital more than $200,000 a year for five years.

Almeda and Roodman told the utility its offer was "unacceptable," and pushed ahead, soliciting project bids.

According to the bid call, all bidders had to agree to supply natural-gas-fueled generators manufactured by Jenbacher Energiesysteme of Austria, whose representatives Almeda and board members Anderson and Jacobson had met on their trip to the East.

Diamond B Constructors of Bellingham offered to design and construct a larger facility than originally suggested - 3.6 megawatts - equipped with ultra-sophisticated computer controls and backup systems. The new price: $4.7 million, just for design and construction. Feasibility studies and other development costs would add $1.2 million more.

Even at the higher price, Steve Kay, Valley's chief financial officer, concluded that the plant would pay for itself in less than nine years. He recommended the board accept Diamond B's bid.

On March 11, 1996, they did, unanimously.

"I think the process is a great illustration of how a public entity should come to a decision," Roodman said. "Only time will tell if this is a great decision or an average decision. I think it's unlikely this is a poor decision."

Hospital wins an award

On Sept. 8, 1997, Valley celebrated the opening of its new power plant. Hospital officials, commissioners and representatives of Jenbacher Energiesysteme all wore hard hats and high-voltage smiles.

Local reporters were invited; none showed. But the plant quickly drew the attention of a national trade publication, Power Magazine. Senior editor Robert Swanekamp, himself a licensed power-plant engineer, was curious about why a hospital in the energy-rich Northwest, of all places, would open its own plant.

Not since the energy crisis of the mid-1970s had institutions or factories thought seriously about delving into the technically intricate realm of self-generated electricity. And no hospital north of California, where steep power rates are the rule, had ever pursued it.

Even in the Golden State, where some 50 hospitals operate co-gen plants, only small generators are used, and the plants are designed strictly as a source of supplemental power.

"When the utility goes down, the co-gen goes down," said Keith Forsman, senior tariff analyst at San Francisco-based Pacific Gas & Electric.

Swanekamp, the magazine editor, visited Valley in early 1998. Almeda told him the hospital's plant was saving more than $50,000 per month on energy bills - a figure he derived by comparing monthly utility bills from before the plant began operating, excluding what it cost to build and operate the facility.

A few weeks before the blackout, Swanekamp awarded Valley the magazine's 1998 Powerplant Award "for running in defiance of conventional wisdom" and "for aggressively embarking on strategies many industries are just now contemplating."

In his story on the award, Swanekamp reported that the hospital had experienced several utility power outages since the plant opened, and that the co-gen plant, operating in island mode, had helped the hospital ride through the outages.

Interviewed recently about those claims, Swanekamp said he got the information from Almeda. But the claims are false: Records show there were no outages at the substation that serves the hospital's neighborhood for all of 1997 and 1998 before the April 29 blackout.

Almeda says Swanekamp must have misunderstood him.

The revelation - combined with other things he has since learned - has made Swanekamp much less excited about Valley's power plant these days.

"Co-generation can reduce electric bills, improve energy efficiency and enhance the environment, but like any technology, it must be properly implemented," he said.

The technology works best when used as a supplement to utility power and when operated by experienced power-plant professionals, he said. Valley's plant is run by Panelo, a mechanical engineer. He directs a crew that includes an electrician, a boiler operator, a marine engineer, an air-conditioning specialist and an electrical engineer - none of whom had previously operated power plants.

"Hospital staffs are accustomed to running small boilers and testing emergency diesels once a week," Swanekamp said. "Generating electricity in a power plant 24 hours a day, seven days a week, is a far more complex endeavor."

It is a lesson Valley officials learned the hard way.

Pulling the fuse

On April 28, Puget Sound Energy dispatcher Bob Ailment notified Panelo that the utility needed to conduct some work on power lines near the hospital. Given that Almeda had bragged in professional circles that the hospital had already successfully gone on island mode, Ailment asked Panelo to be prepared to do it again for a short time while the work was being done the next morning.

Panelo agreed to do so, but he was concerned. In fact, no one had ever successfully islanded the plant for an extended period, especially at midday with the campus drawing a full load of power. What's more, Almeda and Panelo were puzzled about several previous instances in which all four Jenbacher engines had briefly shut down for no apparent reason.

At 10 a.m. on April 29, utility-systems operator Bob Durham called Panelo to request that the hospital go into island mode. Panelo surprised Durham by telling him that he had already islanded the hospital a few minutes earlier. Panelo said later he wanted to leave himself enough time to alert the utility to abort the line work if he was unable to island successfully.

Following steps provided by Diamond B, the plant builder, Panelo had used an unorthodox procedure to go into island mode. Instead of opening the utility circuit breaker, Panelo removed a fuse from the master controller, a sophisticated computer whose primary function is to track the variables that must be in sync for the plant and utility to run safely together.

To reach the fuse, Panelo had to unlock and swing open the front panel of the master controller and yank the fuse from a row of fuses. This action simulated a utility power failure and signaled the controller to open the circuit breaker.

When Puget Sound Energy officials later heard about what Panelo had done, they were astonished.

"It's like reaching under your dash and pulling a fuse to turn off your car radio, instead of using the control knob," said Jeanne Harshbarger, a system-protection engineer for the utility. "You could bump and damage other fuses, and you don't know if you're tripping off the windshield wipers, too."

Almost immediately, strange things began to happen.

Outpatient-services manager Steve Dofelmier was writing a report on his computer in the radiology-services wing. His computer screen began to flicker. He shut it down and walked to the cardiac-catheter lab, concerned about sensitive medical equipment, and found that instruments there had flickered, too. He had no idea why; despite his position, no one had notified him about the islanding.

In the main hospital lab, a hematologist asked laboratory-services supervisor Sharon Hu if she knew about any electrical testing going on that morning. The technician asked because her blood analyzer had inexplicably shut itself off. Hu said she knew of no such tests.

Meanwhile, in the operating-room wing, recovery-room supervisor John Bishop noticed things seemed a bit quieter and stuffier. The ventilation fans, with their omnipresent background hum, had stopped working. Bishop reported the outage to the maintenance dispatcher.

At about 10:15, maintenance workers Mike Bymers, John Graham and Pat d'Ambrosio were winding down their morning break in the engineering shop when their two-way radios began squawking. Graham was directed to the cardiac-catheter lab, where someone reported a funny smoke smell. Bymers headed over to the main lab to help out with the blood analyzer. D'Ambrosio hustled to check on a report that an elevator door in the main hospital building was inexplicably opening and shutting.

Elsewhere on the campus, engineering specialist Bill Bess got a call to check on people stuck between floors on two elevators in the Medical Arts Center. After helping extract the passengers, Bess climbed to a rooftop electrical panel to reset the elevators. There, he discovered a circuit board and a 1 1/2-inch-wide electrical cable hopelessly fused together.

In 22 years at the hospital, he had never seen anything like it: "The ribbon cable melted right down on the control card."

As all of this was unfolding, maintenance worker John Massimino stood by in the hospital basement monitoring a panel of computer screens and electrical indicators. Nearby sat two diesel-fueled emergency generators poised to provide electricity to critical parts of the hospital should all else fail. Massimino heard workers all over the campus communicating animatedly via two-way radio about the electrical problems.

Before long, Massimino experienced firsthand what all the chatter was about. The basement lights, the computer screens and the control-panel indicators shut off in unison, then turned back on.

"Everything went dark, I mean completely black, at least 10 times when I was down there," he said.

Blackout

At 11 a.m., a Puget Sound Energy supervisor called to tell Panelo that the line work was completed. Panelo was now free to synchronize the co-gen plant with the utility's power grid - a necessary step in reconnecting the hospital to the grid.

Panelo went about reversing the unorthodox steps he'd taken earlier. With the four Jenbacher engines supplying the hospital with increasingly erratic power, he replaced the master controller fuse. According to the instructions provided by Diamond B, there should have followed a five-minute delay, during which the controller would automatically synchronize the plant and utility.

Instead, less than a minute after Panelo replaced the fuse, the controller tripped off the Jenbacher engines. Within 10 seconds, the hospital's two diesel-fueled emergency generators, one 16 years old and the other 30, tripped on.

Within a few minutes, the older generator short-circuited and stopped producing power. The other, unable to carry the supply burden alone, soon overheated and shut down.

Massimino, the worker stationed in the basement, now sat in blackness he knew would linger. He got up and left the basement to see if he could help out elsewhere.

At this point, Almeda, who was in a meeting when Panelo took the co-gen plant into island mode, was summoned by an assistant and hustled to the basement. There he met Bymers, Bess and d'Ambrosio.

Almeda decided to try to restart the newer emergency generator. With his men illuminating the area with flashlights, Almeda dropped to his knees to reach the circuit breaker. He grasped the plastic handle with both hands and turned as hard as he could. It wouldn't reset.

He stood up and pressed on the handle with his foot.

Nothing.

Almeda again dropped to his knees, grasped the handle and wrenched on it. It snapped off in his hands.

"I turned it so hard, it broke," Almeda said. He laid the handle down on the floor, where it stayed untouched for the next several hours, everyone careful to step around the symbol of the last chance of avoiding an extended blackout.

Back at the control room of the co-gen plant, Panelo focused on restoring utility power. He tried resetting switches, to no avail. He left the plant building and, carrying a broom handle, jogged over to a knoll on the southeast corner of the campus where the utility circuit breaker was housed in a gray metal box.

Panelo opened the box and contemplated a square, green button on an electrical panel. He realized his actions in the next few moments could restore power to the hospital - or get him killed.

He knew that by pushing the green button, he would reconnect the circuit breaker linking the campus to 12,470 volts of utility power. But he had no idea what was causing the breaker to continuously trip open. Pushing the button could be dangerous.

He stood to one side of the refrigerator-shaped cabinet and, wielding the broom handle like a sword, poked at the button.

"I used the broom because, if that thing blew, I thought maybe I'd lose only my hand, instead of getting blown away."

Panelo closed the circuit breaker with the broom handle, drawing a massive flow of electricity into the gray box - but only momentarily. The breaker kept tripping open. Each time Panelo poked at the green button, lights dimmed in stores and offices for several blocks around the hospital.

Surgery without power

While Almeda and Panelo tried futilely to get electricity flowing again, other Valley staffers contributed human energy.

Judi Hubbard, administrator of patient-care services, and Jim Distilhorst, administrator for medical and quality services, set up an evacuation command post under a skylight in the hospital lobby.

Surgery scrub nurse James Wilson headed to a supply room to fetch an armload of flashlights. Recovery- room supervisor Bishop, a former M.A.S.H. unit director in the military, found ways to help complete the five surgeries in progress with no electricity.

The most delicate case - a vascular procedure on a woman's major arteries - was being handled by a team led by surgeon Michael Burke. Batteries kept the operating table lit and anesthesia machine running. Bishop sent in assistants with flashlights to illuminate the perimeter of the room, while Wilson raced home to fetch his portable Honda generator, which was used to supply power for Burke's headlamp.

An hour and a half after the blackout, Burke completed the surgery.

In the critical-care wing, nurses and technicians took turns hand-pumping ventilators supplying oxygen to seriously ill patients. Arrangements were made to transport the most critical patients by helicopter.

Through the afternoon, all 102 patients were evacuated to 14 other area hospitals.

Meanwhile, Panelo and Almeda frantically tried to find a way to restore power.

The breakthrough came when Panelo reached Jenbacher customer representative Thomas Jaud by phone in Portland. Jaud suggested disabling a relay inside the master controller which - unbeknownst to the hospital staff and utility technicians - was continuously tripping open the utility circuit breaker.

However, some mislabeled schematics added to the confusion. It took several hours to locate the correct relay. Panelo finally reconnected the hospital to the grid, restoring power, at 4:30 p.m.

Board didn't get full story

Valley officials blame the blackout on a string of simple malfunctions and unfortunate errors. Almeda says procedural changes and adjustments to the co-gen plant's controls, along with improvements to the hospital's emergency backup generators, make a recurrence unthinkable.

The plant was back up the day after the blackout, and the hospital islanded for the second time during a test on Nov. 23.

"There is no reason for us, after implementing all the changes recommended, not to go on island," Almeda said.

But hospital staffers, utility engineers and electrical inspectors familiar with the plant aren't so sure. They say Almeda has repeatedly misrepresented how the plant has been operated and kept them minimally informed about what went wrong on April 29.

Immediately after the blackout, Almeda reported to Roodman, Distilhorst and the five-member board of commissioners that the co-gen plant had "operated perfectly" during islanding and "had nothing to do with what happened."

Until a Times reporter asked them about the electrical glitches that preceded the blackout, hospital managers held to the erroneous understanding that the islanding had gone smoothly.

Not until prompted by the Times' findings did Neil Reid, a consultant hired by the hospital to investigate the blackout, look into reports of the co-gen plant producing wildly fluctuating power during islanding.

Reid, senior vice president of Bouillon Inc. of Seattle, now says the islanded plant apparently was unable to handle the simultaneous demands of such major power draws as elevators moving and large roof fans switching on.

Reid is now conducting tests to determine just how many elevators and fans can be activated in island mode without triggering power fluctuations.

`They're not being honest'

Beyond mastering the technical nuances, Valley must also move to repair short-circuited employee morale.

Some hospital staffers are still angry about Almeda's claims to be ignorant of the problems experienced before the blackout - problems witnessed and discussed by scores of people in the hospital - and about the lack of interest from Almeda and Panelo in their first-hand accounts of what happened.

"We haven't had a debriefing of any sort," said John Graham, an 11-year member of the engineering staff. "It did get brought up in front of Rally (Panelo) once and he said, `You don't have to worry about it, because it'll never happen again.' "

Another engineering specialist said: "They're not being honest about what happened and when we ask questions, we're always left in the dark. This is a good place to work, but a lot of things that go on here churn your stomach. It's like the taxpayers are paying for all of this and being led down the wrong road simply to make the hospital administration look good."

Ome Almeda remains steadfast.

"Did I just want a new toy? No, absolutely not," he said.

"I sincerely believe this development of co-gen is really helping the hospital in lowering its operating cost. The real objective is helping the community, not just that Ome wants a toy. I really believe this is good for the community."

While Reid develops the technical fixes, Almeda continues to reassure his bosses that everything that should be done is being done. Hospital officials say they have little choice but to believe in him.

"I'm a hospital manager; I'm not familiar with engineering aspects. Power grids and co-generation are way outside of my field of experience," said Outpatient-services manager Dofelmier. "As a member of a management team, I have to do my part and have faith that my fellow team members are going to do their job."

Panelo, the man at the plant controls - the man responsible, day to day, for averting a potentially tragic repeat of April 29 - assures Dofelmier and others that their faith is well-placed.

"Looking back, I think we gained a lot of knowledge and experience from this event," he said. "I've gained quite a lot of skill, and right now I'm more positive than ever about how to deal with this power plant."

Byron Acohido's phone message number is 206-464-2352. His e-mail address is: bacohido@seattletimes.com