Virtual Reality: At the UW HIT Lab, there's virtue in
Plenty of interfacing is going on inside this therapy room at Harborview Medical Center's Burn Unit.
Lonnie, a wiry 39-year-old framer, sits waist-deep in a metallic tub of water, naked other than a white towel over his lap and a virtual reality headset covering his eyes and ears. Images flow from a laptop down through two cables and into his eyes and mind. A nurse scrubs his scalded left arm under water while a second nurse holds the base of the joystick he manipulates with his right hand.
Inside the goggles, Lonnie interfaces with a frigid world, soaring through an icy cavern with a waterfall sliding down a slope and a river snaking along its floor. He flings snowballs at penguins and snowmen as he swoops and swerves. Each hit sets off a rumbling explosion and a plume of green smoke while some errant shots splash into the riverbed.
Hunter Hoffman, a cognitive psychologist and researcher with the University of Washington's Human Interface Technology Laboratory — the HIT Lab — stands in a corner watching every movement. A monitor shows him exactly what Lonnie sees.
Cleaning burn wounds is necessary to stave off infection, but it is among the most painful procedures a patient must endure. Lonnie, who suffered third-degree burns when an old camping stove exploded, says it feels like getting jabbed with hot needles. But pain requires attention, and the idea behind this experiment — this interface — is to distract Lonnie's attention by immersing him in another place.
It works well for about 15 minutes. But suddenly, Lonnie is ripping it off. Reality has taken over. Hoffman leaves so Lonnie can grit and curse his way through the rest of the procedure in privacy.
The long-running experiment, done in conjunction with Harborview psychologist David Patterson, represents one of the more standard projects that come out of the eclectic, sometimes eccentric, and always interdisciplinary HIT Lab. What we call virtual reality, the lab calls immersive or augmented reality.
Its emphasis is on interface: between computer power and human mind; university-based research and commercial application; the virtual and the real. With almost every project, researchers interface, too — with engineers and computer programmers, technicians and bright students, real-world experts and market forces.
In the 15 years since it began, the lab has had its share of dead ends, but its work is always ambitious. Right now, it is pursuing projects aimed at helping people to see or to walk, surgeons to operate or to locate disease, students to demystify biochemistry, teachers to illuminate environmental concerns, architects and interior designers to make sense of space, and drivers to maneuver more safely.
The projects embody the spirit of the lab's director and founder, Tom Furness, a pioneer of virtual reality. His engineering bio describes his life work as "the concept of virtual interface technologies that prove a circumambience of three-dimensional spatial information to the human using the visual, auditory and tactile sensory modalities."
Furness translates in English: "I want to break the glass. Those last few inches, between the computer screen and your brain, is the fundamental problem I've been trying to address my whole career."
THOSE FEW INCHES represent a giant stride — and if that's not aiming high enough, Furness projects an almost missionary desire that the technology change lives, not just make them easier.
Awhile back, he addressed a group of investors called The Virtual Worlds Consortium. The group, which includes about four dozen organizations and companies such as Intel, Boeing and Fujitsu, helps the lab through annual membership dues and equipment donations. The consortium also puts up the money for particular ideas.
But while the lab is judged by the numbers — patents, technology transfers and state companies that it spawns — Furness spent much of his talk highlighting more ambitious goals.
"I want to make the world a better place," he said, showing slides about war, famine, AIDS and SARS. "I want to help make the world safer and healthier. I want to work on uniting people and solving the pervasive problems of our age."
That sort of thinking is one reason he has long supported Hoffman's pain research, although it is a ways from finding a market. It's also why Furness attracts researchers like Hoffman, a workaholic who toiled without pay for three years until he got funding from the National Institutes of Health and the Paul G. Allen Foundation.
Furness, 60, grew up a dreamer, dreaming the biggest of pictures. He wanted to be an astronaut. But his poor eyesight kept him grounded, so he went to Duke University, studied electrical engineering and went right to work for the Department of Defense during the Vietnam War. He spent much of his 23-year career there simplifying fighter-jet cockpits.
"My job was figuring out how you connect one person to 50 computers with all these displays and switches in the cockpit," he says, "while he was pulling G's and getting shot at."
As chief of the Visual Display Systems Branch for an Air Force laboratory, Furness developed virtual-interface concepts well before the term "virtual reality" was coined. His staff of 50 government and contract scientists, engineers and technicians pioneered advancements that led to the "Super Cockpit," a helmet-mounted display that essentially melded information from all that instrument data with the real world outside.
His work was declassified when the federal government came under fire for wasting taxpayer money. The Department of Defense sent Furness on a series of public show-and-tell presentations. He proved to be a natural showman with his down-home North Carolinian drawl, rocking-chair charm and near-evangelistic belief that computing can and will break out from behind the glass.
"Overnight," he recalls, "it seemed I went from engineer to entertainer." Networks and national publications profiled him and his work, and got caught up in the gee-whiz potential of VR. He began getting calls from doctors, people with disabilities, teachers and others asking, "Can you help me?"
WITH THE MILITARY'S permission, Furness took a sabbatical in 1987 so he could explore ways to spread some of the technology to the rest of the world. Ultimately, he decided to set up a university-based lab. He talked to the Massachusetts Institute of Technology and other top engineering universities before settling on the UW. He liked being in the same town with Microsoft and Boeing, and the university was eager to become more aggressive in finding commercial markets for its research.
The lab opened in 1989 and weathered lean early years. It found a bit of traction with the 1991 patent on Furness' research into a process called Virtual Retinal Display (VRD). The technology involves scanning an image directly onto the retina. The UW's Office of Intellectual Property and Technology Transfer says VRD has led to more than 45 patents issued to the university; it also spawned Microvision Inc., which holds an exclusive license to the technology.
In fiscal year 2002, the HIT Lab, which operates as a research unit in the College of Engineering, was responsible for 14 percent of the patents awarded to the entire university. Researcher Eric Seibel was credited with 11 registered innovations in that period, more than any other person at the university. Seibel topped the list again for 2003.
Besides licensing its technology to existing companies, the lab claims to have spun off or helped start 18 companies in the past decade, two of which are traded on NASDAQ. The totals are impressive when you consider the lab has only 19 permanent employees — and five of those work part time. Most of the researchers are students, from Ph.Ds to undergraduates, but faculty associates, outside experts and industry fellows also come and go with the projects.
There is little infighting or battling for turf, partly because projects and people keep changing so veterans can't get too set in their ways or territorial.
Suzanne Weghorst, an acting director with Seibel when Furness is absent and a researcher who has been with the lab since its inception, says Furness' personality and energy set the agenda and the pace.
"The HIT Lab culture is very broad-based in terms of academic backgrounds and research interests — just like Tom," she says. "Its culture is focused on breaking new ground and 'thinking outside the box' — just like Tom. And the culture is very human-oriented, both in the work we do and in the way we work with each other — just like Tom."
The main section of the lab is on the second floor of the Washington Technology Center, which is housed in Fluke Hall, on the campus' southern edge. The part of the lab where the programmers work is an egalitarian cluster of cubicles circled by a single aisle. Between two conference rooms sit the "Visualization Simulation Lab," filled with various works-in-progress, and a room labeled Pain and Phobia Lab. Posters of presentations cover cubicle walls, and erase-boards hold jotted notes like "imagining lens" and "optimal beam scanning." Off in a cubbyhole is an old augmented-reality kiosk emblazoned with the words "watch television and the rest of your life at the same time."
Veteran researchers such as Weghorst and Seibel work in the cramped cubicles and spend much of their time leading teams and mentoring students. Other than Furness', the only offices belong to two hairy spiders that Hoffman keeps as "tactile reinforcers" for his phobia studies.
Students take on a lot of responsibility. Down a flight from where programmers work on laptops, prepare grants and meet, there are two hardware rooms. One is populated with relics from prototypes past. The other is home to a strange little pair of spectacles resting on a mannequin head. Fitted with a small scanner and a little camera that sends images directly to the retina, it's the latest prototype for a wearable aid that helps people with impaired vision avoid obstacles. The National Science Foundation helped Seibel pursue the project, and he, in turn, supervised two bright students who designed and built it.
The device has Furness' fingerprints all over it, too. He was inspired to develop it after seeing early, if fleeting, success for a man with Parkinson's Disease.
EARLY ON A TUESDAY morning, Weghorst sits at the head of a long conference table, leading a meeting on how to make the principles of biochemistry come alive. With her, besides a longtime lab hardware engineer and a couple of professors, are graduate students from across the globe, from Venezuela to India.
On a speakerphone is a scientist from the Scripps Research Foundation. His lab is producing chalky Jacks-sized molecule models that people can touch and turn in their hands. These models can essentially come alive by adding interface technologies, and Weghorst's team hopes to make that happen. Bill Anderson, a master's student from New Mexico, for instance, has come up with a stylus-like tool, about the size of a penlight, that can be used to touch the models Scripps is building and detect various charges where amino acids bond.
An aspect of the project relies on principles that led to one of the lab's top achievements, "The MagicBook." Invented by Mark Billinghurst, who now directs a sister HIT Lab in New Zealand, The MagicBook is an example of "augmented reality," or the transition between reality and virtual reality. By using goggles that look like opera glasses, a reader can focus on a square on a page, press a button and watch a 3-D object pop up. He or she can then motor around inside the story and even become part of it.
The MagicBook technology allows several users to look at the book (and a particular 3-D object it creates) from different viewpoints. When one person enters the world, others also using the glasses can see him or her as part of it. This sort of immersion and collaboration makes the technology ideal, Billinghurst says, for education, architecture and entertainment.
The molecule book is a basic introduction to protein structures. From a book lying flat on the keyboard, you read the old-fashioned way about amino acids and the various structures that proteins take. The computer screen, meanwhile, shows 3-D colored images highlighting the various principles. It still has bugs, and the team continues to evaluate how ambitious to be. But mostly, the multidisciplinary discussion involves one simple question: Does it help?
That question is asked throughout the lab as students work with lead researchers and mentors on a wide range of topics, from showing Puget Sound currents to technology that may one day aid cancer detection and treatment.
MY FIRST ATTEMPT at performing a transurethral resection is going very badly.
Using both hands, I clumsily maneuver a surgical scope through my patient's urethra. Dr. Robert Sweet of the UW School of Medicine stands at my right shoulder, watching on a monitor above the operating table as I slide the device in and out and change from cutting and coagulating modes by alternating between two foot pedals.
I can feel the tension on the other side of the device as I activate what seems like a blade but is actually an electrical loop. As I begin pulling back on the blade and trimming tissue, I hit parts of the patient's anatomy I shouldn't. Thank God the patient is actually a simulator. Jeff Berkley, president of Mimic Technologies, jokes from the back of the room that this is about the time a lawyer icon should pop onto the screen.
"I would have taken over before this," says Sweet, who's also a urologist at the UW Medical Center. I hand the device over, and he slips the scope back and forth, shearing the walls of the prostate tissue as deftly as if he were coring an apple.
Surgical training is an area where virtual- and augmented-reality technology are slowly but surely making inroads. By using simulators, residents can get more practice and a more objective assessment from the recorded data, Sweet believes. Residents are trained mainly by putting in hours.
But when Sweet was a resident, he wanted more opportunities to practice. So a few years ago, he approached the HIT Lab with an idea for the simulator. Peter Oppenheimer of the HIT Lab provided the graphics while Berkley's company programmed the tactile sensation into it.
While residents almost unanimously welcome the device, Sweet continues to test its effectiveness. He envisions a day when simulators for all kinds of surgery training and evaluation become the norm.
The challenge remains, however, to make simulators for other types of surgery realistic enough to be meaningful. The sense of tactile reality, called "haptic feedback," is critical. Berkley, while at the HIT Lab, developed a suturing simulator that provides a realistic sense of tension when you stitch an imaginary hand.
Berkley's involvement highlights two other lab byproducts — developing talent and spinning off companies. He spent six years at the lab, and while there met Seahak Kim, a Ph.D intern from the Tokyo Institute of Technology. The two worked together on haptics projects and realized their combined know-how would make a company. They created Mimic in 2001.
FEW PROJECTS IN the lab are as easy to grasp as the inside of Hoffman's SnowWorld or SpiderWorld projects or as amusing as the suturing simulator. Most innovations, if they ever come, are subtle — the result of small solutions relentlessly pursued and incrementally achieved.
Making real, marketable and helpful virtual reality is, in reality, often a slog. Yet, Furness' optimistic interfacing permeates the place. In his office he's posted two quotes: "We don't wait for miracles. We depend on them" and "Whether you think you can or think you can't — you're right."
There are continual rumors he's going to retire soon, but he insists he's "more fired up than ever." In fact, he spent the past three months in New Zealand, helping Billinghurst strengthen the HIT Lab there. Then, he and other university officials, including interim President Lee Huntsman traveled to Singapore to work on establishing a HIT Lab there.
One morning, he popped into a staff meeting with the Seattle crew via computer connection. The conference room inside the technology center was filled with lab workers, students and veterans alike. They munched pizza and faced the head of the room where Furness' magnified face hung on a big screen. He could only hear, not see the Seattle crew because of a glitch in the system. But he made do.
A small microphone was passed around the room so each of the 40 or so people at the meeting could describe what was new, whether it was a successful dissertation defense, a project progress report or wedding plans. Furness listened patiently and offered encouragement to each one.
Between his gray hair and beard, wide-rimmed glasses and soothing voice, he looked and sounded like everyone's grandfather. Hoffman, picking up on the blend of benevolence and leadership that Furness' giant face was reflecting, grabbed the microphone and piped up from the back of the room: "I feel like Captain Kirk speaking to a higher being."
Richard Seven is a Pacific Northwest magazine staff writer.