Medical Guesswork
From heart surgery to prostate care, the health industry knows little about which common treatments really work
By John Carey

The signs at the meeting were not propitious. Half the board members of Kaiser Permanente's Care Management Institute left before Dr. David Eddy finally got the 10 minutes he had pleaded for. But the message Eddy delivered was riveting. With a groundbreaking computer simulation, Eddy showed that the conventional approach to treating diabetes did little to prevent the heart attacks and strokes that are complications of the disease. In contrast, a simple regimen of aspirin and generic drugs to lower blood pressure and cholesterol sent the rate of such incidents plunging. The payoff: healthier lives and hundreds of millions in savings. "I told them: 'This is as good as it gets to improve care and lower costs, which doesn't happen often in medicine,"' Eddy recalls. "'If you don't implement this,' I said, 'you might as well close up shop."'

The message got through. Three years later, Kaiser is in the midst of a major initiative to change the treatment of the diabetics in its care. "We're trying to put nearly a million people on these drugs," says Dr. Paul Wallace, senior adviser to the Care Management Institute. The early results: The strategy is indeed improving care and cutting costs, just as Eddy's model predicted.

For Eddy, this is one small step toward solving the thorniest riddle in medicine -- a dark secret he has spent his career exposing. "The problem is that we don't know what we are doing," he says. Even today, with a high-tech health-care system that costs the nation $2 trillion a year, there is little or no evidence that many widely used treatments and procedures actually work better than various cheaper alternatives.

This judgment pertains to a shocking number of conditions or diseases, from cardiovascular woes to back pain to prostate cancer. During his long and controversial career proving that the practice of medicine is more guesswork than science, Eddy has repeatedly punctured cherished physician myths. He showed, for instance, that the annual chest X-ray was worthless, over the objections of doctors who made money off the regular visit. He proved that doctors had little clue about the success rate of procedures such as surgery for enlarged prostates. He traced one common practice -- preventing women from giving birth vaginally if they had previously had a cesarean -- to the recommendation of one lone doctor. Indeed, when he began taking on medicine's sacred cows, Eddy liked to cite a figure that only 15% of what doctors did was backed by hard evidence.

A great many doctors and health-care quality experts have come to endorse Eddy's critique. And while there has been progress in recent years, most of these physicians say the portion of medicine that has been proven effective is still outrageously low -- in the range of 20% to 25%. "We don't have the evidence [that treatments work], and we are not investing very much in getting the evidence," says Dr. Stephen C. Schoenbaum, executive vice-president of the Commonwealth Fund and former president of Harvard Pilgrim Health Care Inc. "Clearly, there is a lot in medicine we don't have definitive answers to," adds Dr. I. Steven Udvarhelyi, senior vice-president and chief medical officer at Pennsylvania's Independence Blue Cross.

What's required is a revolution called "evidence-based medicine," says Eddy, a heart surgeon turned mathematician and health-care economist. Tall, lean, and fit at 64, Eddy has the athletic stride and catlike reflexes of the ace rock climber he still is. He also exhibits the competitive drive of someone who once obsessively recorded his time on every training run, and who still likes to be first on a brisk walk up a hill near his home in Aspen, Colo. In his career, he has never been afraid to take a difficult path or an unpopular stand. "Evidence-based" is a term he coined in the early 1980s, and it has since become a rallying cry among medical reformers. The goal of this movement is to pierce the fog that envelops the practice of medicine -- a state of ignorance for which doctors cannot really be blamed. "The limitation is the human mind," Eddy says. Without extensive information on the outcomes of treatments, it's fiendishly difficult to know the best approach for care.

The human brain, Eddy explains, needs help to make sense of patients who have combinations of diseases, and of the complex probabilities involved in each. To provide that assistance, Eddy has spent the past 10 years leading a team to develop the computer model that helped him crack the diabetes puzzle. Dubbed Archimedes, this program seeks to mimic in equations the actual biology of the body, and make treatment recommendations as well as figure out what each approach costs. It is at least 10 times "better than the model we use now, which is called thinking," says Dr. Richard Kahn, chief scientific officer at the American Diabetes Assn.

WASTED RESOURCES
Can one computer program offset all the ill-advised treatment options for a whole range of different diseases? The milestones in Eddy's long personal crusade highlight the looming challenges, and may offer a sliver of hope. Coming from a family of four generations of doctors, Eddy went to medical school "because I didn't know what else to do," he confesses. As a resident at Stanford Medical Center in the 1970s, he picked cardiac surgery because "it was the biggest hill -- the glamour field."

But he soon became troubled. He began to ask if there was actual evidence to support what doctors were doing. The answer, he was surprised to hear, was no. Doctors decided whether or not to put a patient in intensive care or use a combination of drugs based on their best judgment and on rules and traditions handed down over the years, as opposed to real scientific proof. These rules and judgments weren't necessarily right. "I concluded that medicine was making decisions with an entirely different method from what we would call rational," says Eddy.

About the same time, the young resident discovered the beauty of mathematics, and its promise of answering medical questions. In just a couple of days, he devoured a calculus textbook (now framed on a shelf in his beautifully appointed home and office), then blasted through the books for a two-year math course in a couple of months. Next, he persuaded Stanford to accept him in a mathematically intense PhD program in the Engineering-Economics Systems Dept. "Dave came in -- just this amazing guy," recalls Richard Smallwood, then a Stanford professor. "He had decided he wanted to spend the rest of his life bringing logic and rationality to the medical system, but said he didn't have the math. I said: 'Why not just take it?' So he went out and aced all those math courses."

To augment his wife's earnings while getting his PhD, Eddy landed a job at Xerox Corp.'s (XRX ) legendary Palo Alto Research Center. "They hired weird people," he says. "Here was a heart surgeon doing math. That was weird enough."

Eddy used his newfound math skills to model cancer screening. His Stanford PhD thesis made front-page news in 1980 by overturning the guidelines of the time. It showed that annual chest X-rays and yearly Pap smears for women at low risk of cervical cancer were a waste of resources, and it won the most prestigious award in the field of operations research, the Frederick W. Lanchester prize. Based on his results, the American Cancer Society changed its guidelines. "He's smart as hell, with a towering clarity of thought," says Stanford health economist Allan Enthoven.

Dr. William H. Herman, director of the Michigan Diabetes Research & Training Center, has a competing computer model that clashes with Eddy's. Nonetheless, he says, "Dr. Eddy is one of my heroes. He's sort of the father of health economics -- and he might be right."

Appointed a full professor at Stanford, then recruited as chairman of the Center for Health Policy Research & Education at Duke University, Eddy proved again and again that the emperor had no clothes. In one study, he ferreted out decades of research evaluating treatment of high pressure in the eyeball, a condition that can lead to glaucoma and blindness. He found about a dozen studies that looked at outcomes with pressure-lowering medications used on millions of people. The studies actually suggested that the 100-year-old treatment was harmful, causing more cases of blindness, not fewer.

Eddy submitted a paper to the Journal of the American Medical Assn. (JAMA), whose editors sent it out to specialists for review. "It was amazing," Eddy recalls. "The tom-toms sounded among all the ophthalmologists," who marshaled a counterattack. "I felt like Salman Rushdie." Stanford ophthalmologist Kuldev Singh says: "Dr. Eddy challenged the community to prove that we actually had evidence. He did a service by stimulating clinical trials," which showed that the treatment does slow the disease in a minority of patients.

By 1985, Eddy was "burned out" by the administrative side of academia, he says. Lured by a poster of the Tetons, he gave up his prestigious post. He moved to Jackson, Wyo., so he could climb in his spare time. He and a friend even made a first ascent of a new route on the Grand Teton, now named after them. Meanwhile, he carved out a niche showing doctors at specialty society meetings that their cherished beliefs were dubious. "At each meeting I would do the same exercise," he says. He would ask doctors to think of a typical patient and typical treatment, then write down the results of that treatment. For urologists, for instance, what were the chances that a man with an enlarged prostate could urinate normally after having corrective surgery? Eddy then asked the society's president to read the predictions.

The results were startling. The predictions of success invariably ranged from 0% to 100%, with no clear pattern. "All the doctors were trying to estimate the same thing -- and they all gave different numbers," he says. "I've spent 25 years proving that what we lovingly call clinical judgment is woefully outmatched by the complexities of medicine." Think about the implications for helping patients make decisions, Eddy adds. "Go to one doctor, and get one answer. Go to another, and get a different one." Or think about expert testimony. "You don't have to hire an expert to lie. You can just find one who truly believes the number you want."

More important, the lack of evidence creates a costly clash. Americans and their doctors want access to any new treatment, and many doctors fervently believe such care is warranted. On the other hand, those beliefs can be flat wrong. As a consultant on Blue Cross's insurance coverage decisions, Eddy testified on the insurer's behalf in high-profile court cases, such as bone marrow transplants for breast cancer. Women and doctors demanded the treatment, even though there was no evidence it saved lives. Insurers who refused coverage usually lost in court. "I was the bad guy," Eddy recalls. When clinical trials were actually done, they showed that the treatment, costing from $50,000 to $150,000, didn't work. The doctors who pushed the painful, risky procedure on women "owe this country an apology," Eddy says.

Is medicine doing any better today? In recognizing the problem, yes. But in solving it, unfortunately, no. Take prostate cancer. Doctors now routinely test for levels of prostate-specific antigen (PSA) to try to diagnose the disease. But there's no evidence that using the test improves survival. Some experts believe that as many cancers would be detected through random biopsies. Then, once cancer is spotted, there's no way to know who needs treatment and who doesn't. Plus, there is a plethora of treatment choices -- four kinds of surgery, various types of implantable radioactive seeds, and competing external radiation regimens, notes Dr. Eric Klein, head of urologic oncology at the Cleveland Clinic. "How is a poor patient supposed to decide among those?" he asks. Most of the time, patients don't even know the options.

VESTED INTERESTS
"Because there are no definitive answers, you are at the whim of where you are and who you talk to," says Dr. Gary M. Kirsh at the Urology Group in Cincinnati. Kirsh does many brachytherapies -- implanting radioactive seeds. But "if you drive one and a half hours down the road to Indianapolis, there is almost no brachytherapy," he says. Head to Loma Linda, Calif., where the first proton-beam therapy machine was installed, in 1990, and the rates of proton-beam treatment are far higher than in most other parts of the country. Go to a surgeon, and he'll probably recommend surgery. Go to a radiologist, and the chances are high of getting radiation instead. "Doctors often assume that they know what a patient wants, leading them to recommend the treatment they know best," says Dr. David E. Wennberg, president of Health Dialog Analytic Solutions.

More troubling, many doctors hold not just a professional interest in which treatment to offer, but a financial one as well. "There is no question that the economic interests of the physician enter into the decision," says Kirsh. The bottom line: The conventional wisdom in prostate cancer -- that surgery is the gold standard and the best chance for a cure -- is unsustainable. Strangely enough, however, the choice may not matter very much. "There really isn't good evidence to suggest that one treatment is better than another," says Klein.

Compared with the skepticism Eddy faced in the 1990s, many physicians now concur that traditional treatments for serious illnesses often aren't best. Yet this message can be hard for Americans to believe. "When there is more than one medical option, people mistakenly think that the more aggressive procedure is the best," says Annette M. Cormier O'Connor, senior scientist in clinical epidemiology at the Ottawa Health Research Institute. The message flies in the face of America's infatuation with the latest advances. "As a nation, we always want the best, the most recent technology," explains Dr. Joe Thompson, health adviser to Arkansas Governor Mike Huckabee. "We spend a huge amount developing it, and we get a big increase in supply." New radiation machines for cancer or operating rooms for heart surgery are profit centers for hospitals, for instance (see BW Online, 07/18/05, "Is Heart Surgery Worth It?"). Once a hospital installs a shiny new catheter lab, it has a powerful incentive to refer more patients for the procedure. It's a classic case of increased supply driving demand, instead of the other way around. "Combine that with Americans' demand to be treated immediately, and it is a cauldron for overuse and inappropriate use," says Thompson.

The consequences for the U.S. are disturbing. This nation spends 2 1/2 times as much as any other country per person on health care. Yet middle-aged Americans are in far worse health than their British counterparts, who spend less than half as much and practice less intensive medicine, according to a new study. "The investment in health care in the U.S. is just not paying off," argues Gerard Anderson, director of the Center for Hospital Finance & Management at Johns Hopkins' Bloomberg School of Public Health. Speaking not for attribution, the head of health care at one of America's largest corporations puts it more bluntly: "There is a massive amount of spending on things that really don't help patients, and even put them at greater risk. Everyone that's informed on the topic knows it, but it is such a scary thing to discuss that people are not willing to talk about it openly."

Of course, there are plenty of areas of medicine, from antibiotics and vaccines to early detection of certain tumors, where the benefits are huge and incontrovertible. But if these effective treatments are black and white, much of the rest of medicine is a dark shade of gray. "A lot of things we absolutely believe at the moment based on our intuition are ultimately absolutely wrong," says Dr. Paul Wallace, of the Care Management Institute.

The best way to go from intuition to evidence is the randomized clinical trial. Patients with a particular condition are randomly assigned to competing treatments or, if appropriate, to a placebo. By monitoring the patients for months or years, doctors learn the relative risks and benefits of the treatment being studied.

But such trials take years and cost many millions of dollars. By the time the results come in, science and medicine may have moved on, making the findings less relevant. Moreover, patients in a clinical trial usually aren't representative of real people, who tend to have complex combinations of diseases and medical problems. And patients often don't stick with the program.

Such difficulties are highlighted by an eight-year study of low-fat diets that cost upward of $400 million. Most subjects failed to stick to the low-fat regimen, making it tough to draw conclusions. In addition, the study failed to take stock of different kinds of fats, some of which are now known to have beneficial effects. Many trials fall into similar traps. So it's no surprise that up to one-third of clinical studies lead to conclusions that are later overturned, according to a recent paper in JAMA.

Even when common treatments are proved to be dubious, physicians don't rush to change their practice. They may still firmly believe in the treatment -- or in the dollars it brings in. And doctors whose oxen get gored sometimes fight back. In 1993, the federal government's Agency for Health Care Policy & Research convened a panel to develop guidelines for back surgery. Fearing that the recommendations would cast doubt on what the doctors were doing, a prominent back surgeon protested to Congress, and lawmakers slashed funding for the agency. "Congress forced out the research," says Floyd J. Fowler Jr., president of the Foundation for Informed Medical Decision Making. "It was a national tragedy," he says -- and not an isolated incident. The agency's budget is often targeted "by special interest groups who had their specialty threatened," says Arkansas' Dr. Thompson.

With proof about medical outcomes lacking, one possible solution is educating patients about the uncertainties. "The popular version of evidence-based medicine is about proving things," says Kaiser's Wallace, "but it is really about transparency -- being clear about what we know and don't know." The Foundation for Informed Medical Decision Making produces booklets, videotapes, and other material to put the full picture in the hands of patients. Health Dialog markets the information to providers and companies, addressing back pain, breast cancer, uterine fibroids and bleeding, coronary heart disease, depression, osteoarthritis, and other conditions.

In studies where one group of patients hears the full story while other patients simply receive their doctors' instructions, a key difference emerges. The well-informed patients opt for more invasive, aggressive approaches 23% less often, on average, than the other group. In some cases, the drop is much bigger -- 50% to 60%. "Patients typically don't understand that they have options, and even if they do, they often wildly exaggerate the benefits of surgery and wildly minimize the chances of harm," says Ottawa's O'Connor, a leader in this field of so-called decision aids.

Eddy's computer simulation could help more patients attain appropriate care. His approach is to create a SimCity-like world in silicon, where virtual doctors conduct trials of virtual patients and figure out what treatments work. After getting funding from Kaiser Permanente in 1991, Eddy hired a particle physicist, Len Schlessinger, who knew how to write equations describing the complex interactions in biology. The pair selected diabetes as a test case. In their virtual world, each simulated person has a heart, liver, kidneys, blood, and other organs. As in real people, cells in the pancreas make insulin, which regulates the uptake of glucose in other cells. And as in the real disease, key cells can fail to respond to the insulin, causing high blood-sugar levels and a cascade of biological effects. The virtual patients come down with high blood pressure, heart disease, and poor circulation, which can lead to foot ulcers and amputations, blindness, and other ills. The model also assesses the costs of treating the complications.

Eddy dubbed the model Archimedes and tested it by comparing it with two dozen real trials. One clinical study compared cholesterol-lowering statin drugs to a placebo in diabetics. After 4 1/2 years, the drugs reduced heart attacks by 35%. The exact same thing happened in Eddy's simulated patients. "The Archimedes model is just fabulous in the validation studies," says the University of Michigan's Herman.

STANDARD OF CARE
The team then put Archimedes to work on a tough, real problem: how best to treat diabetes in people who have additional aliments. "One thing not yet adequately embraced by evidence-based medicine is what to do for someone with diabetes, hypertension, heart disease, and depression," explains Kaiser's Wallace. Doctors now typically try to treat the most pressing problems. "But we fail to pick the right ones consistently, so we have misdirected utilization and a great deal of waste," he says. Kaiser Permanente's Dr. Jim Dudl had a counterintuitive suggestion. With diabetics, doctors assume that keeping blood sugar levels low and consistent is the best way to ward off problems such as heart disease. But Dudl wondered what would happen if he flipped it around, aiming treatment at the downstream problems. The idea is to give patients a trio of generic medicines: aspirin, a cholesterol-lowering statin, and drugs called ACE inhibitors.

Using Archimedes and thousands of virtual patients, Eddy and Schlessinger compared the traditional approach with the drug combination. The model took about a half-hour to simulate a 30-year trial, and showed that the three-drug combination was "cost- and life-saving," says Kaiser's Wallace. The benefits far surpassed "what can be achieved with aggressive glucose control." Kaiser Permanente docs switched their standard of care for diabetes, adding these drugs to other interventions. It is too early to declare a victory, but the experience with patients seems to be mimicking Eddy's computer model. "It goes against our mental picture of the disease," says Wallace. But it also makes sense, he adds. "Cardiovascular disease is the worst complication of diabetes -- and what people die of."

Eddy readily concedes that this example is a small beginning. In its current state of development, Archimedes is like "the Wright brothers' plane. We're off the sand and flying to Raleigh." But it won't be long, he says, "before we're offering transcontinental flights, with movies."

The modeling approach allows each of us, in essence, to have an imaginary twin. We can use our twin to predict what our lives and state of health are likely to be with different lifestyles and approaches to care. Companies could create virtual clones of each employee, predicting what will occur with current care or with added prevention or treatment programs. "They can see what happens to such things as the complications suffered by diabetics, the lost time from work, the amount of angina or the rate of heart attacks, the number of deaths, and the cost of new employees if one dies," Eddy explains. "Our mission is that in 10 years, no one will make an important decision in health care without first asking: `What does Archimedes say?"'