KNOWING WHAT WE KNOW : THE SPIRIT OF JOHN WENNBERG

Before I resume my discussion of the role of stakeholders in the American health care system, there are some underlying facts that I take for granted in my arguments that aren’t well known to the public. It’s time for a side trip to discuss those.

A major part of the health insurance reform debate is tied up in the question of what to cover and when. If what’s done for a patient in a given circumstance can be limited to a known set of procedures and therapies then costs can be contained. This presumes that there is some standardization to medical care – or that it CAN be standardized. One of the best kept secrets in health care is… it isn’t, and in current state of knowledge it can’t be.

The law presumes that any community with available resources has an implicit agreement as to what the “prudent physician” would offer in treating a patient – a “standard of care”. The courts differ somewhat as to whether the standard is local only, statewide, or national – BUT the concept stands that medical care offered within the accepted standard of care is appropriate, prudent, and legally defensible.

However.. The medical community knows that having a single well-defined standard is distinctly unusual. Treatment has to be individualized for each patient. Not every patient will consent to what the doctor proposes – and ethically, the doctor advises and the patient makes the decisions. Patients may have other medical conditions that demand deviation from the standard. There are clear differences in local standards of care between communities – what would be treated with medication, what requires surgery, what needs to be treated actively, and what can just be watched. A doctor will be biased toward what he was first taught in medical school and learned on the wards; then is influenced by new information he might get from the medical literature and continuing education, and from drug companies and equipment manufacturers; and by what he can do himself and what specialists and resources are available in his community. Each doctor traditionally reserves to himself the prerogative of the final arbiter in assessing what’s most applicable in caring for his patient. And so the “standards”.... aren't. They became broader and more inclusive as new medicines and technologies are introduced. The profit motive in health care leads to competition. Competition means introducing to the community technologies that have not been adequately investigated – and the investors have to recoup their investment. (But if they’re still investigational, insurance doesn’t pay for them. This issue lands squarely on the patients’ shoulders.)

Just how broad these variations were, was demonstrated by a series of innovative studies published in the 1970s and 1980s by John Wennberg and his associates, of the Department of Epidemiology at Dartmouth College. Wennberg first started out to compare the standards of care for low back pain in Boston and in New Haven, CT. He found that one community had three times the rate of surgery compared to the other, with a commensurate increase in hospital utilization and cost. There was absolutely no difference in outcomes - patient well-being, freedom from pain, and quality of life.

A fluke? An observational error? Well, no... Wennberg and associates went on to study a number of different common conditions that could be treated within standard of care either with surgery or with medication in a number of different communities (the technique he developed became known as “small area analysis”). These included coronary artery disease, prostate cancer, hysterectomy for benign disease. In each case he found similar results - particularly in diseases where there had been no formal clinical trials and thus no clear research to guide physicians. Different localities had different standards of care for the most common major diseases – and the standard followed created major differences in costs (hospital revenues, doctors’ incomes, hospital and pharmaceutical expenses) but no difference in how patients did.

The results were consistent enough over two decades of investigation and dozens of published analyses that the state of American medical practice (as current in the 1970s and 1980s) could be reduced to a few simple statements that are pretty shocking when you first hear them:

• only 20% of all medical practice is validated by randomized clinical trials, metaanalyses (comprehensive systematic reviews of ALL published literature), or other appropriate research generating high-quality evidence;
• only 20% of the rest is LIKELY to be validated – in part because of the expense and difficulty of performing the appropriate research, and in part because nobody will  invest manpower, time, and money to validate old technologies that are generally accepted (nobody is going to invest in proving that aspirin works for headaches);
• ALL THE REST is unproven – shared biases imparted in medical school and at the bedside in internship and residency, educated suppositions as to what ”should” work; AND
• Communities tend to have common prevalent standards of care that may differ widely from those in other communities because physicians of similar training and biases will tend to recruit each other and locate together.

Because acceptable practices vary widely, a patient getting multiple opinions gets multiple answers which may be very different from each other – and all equally right or equally wrong. I tell my patients that a doctor who has only one possible answer to a complex medical issue either thinks he’s God or just isn’t well read.

Wennberg's work stimulated a drive toward developing an objective base of evidence for all medical practice – but because new drugs and technologies have continued to appear faster than the research to validate them can be performed, while the actual evidence is limited and frequently of poor quality, the numbers cited above still pretty much hold today. Since his main publications appeared 30+ years ago, one new specialty has emerged that is almost completely evidence-based – HIV/AIDS therapy. A lot of good research has been done; also, a lot of flawed research, and a lot of research funded by drug or medical equipment manufacturers with inherent conflicts of interest. Positive studies are published prominently in the medical community and reach the lay press with good news of better results for a major disease (which are often not quite ready for prime time). Many papers presented with great fanfare at scientific meetings are never published because the presentations are preliminary; when all the data are in, the analysis done, and the manuscript written, it just didn’t work out as presented. Negative studies of high quality – the hypotheses that didn’t pan out when properly tested - rarely see print. Without seeing what's already been tried and failed, it‘s hard to know what we DO know.

Some studies that need to be done are logistically improbable. Randomized clinical trials need to be well-organized and accurately conducted to be valid. This means that enough patients have to be entered that any real difference between the treatments will be detected; manpower is needed and costs incurred to make sure the data to show that difference is collected consistently and correctly and analyzed properly; and legal requirements are met to protect patients’ rights. Insurers don’t pay for “experimental” therapies – including participation in clinical trials - so other funding has to be in place. Funding by drug companies or equipment manufacturers is suspect for conflict of interest; and government funding (usually from the National Institutes of Health) has become very difficult to obtain as the federal deficit rises and the budget is diverted in other directions.

Few patients will consent to a study that randomly assigns them between a major operation or taking a new medication. Other barriers? Physicians should not enter their patients in a trial if they feel they already know the answer. (This is an ethical issue. The doctor’s obligation is to offer a patient the best advice they know. You only suggest a clinical trial to a patient when you don’t have a belief that one treatment is better than the other.) Some studies have so little possibility of having a positive effect and so much likelihood of major toxicity that, were they patients, doctors wouldn't participate; offering the study is an ethical problem when risks outweigh benefits.  If the study has to do with a rare disease and few patients are likely to benefit from the outcome, it takes low priority when funding is scarce.

Wennberg’s work continues and is easily available on the Web as the Dartmouth Atlas of Health Care.

How can a non-physician bureaucrat somewhere in a health plan determine what should be paid for? Insurers want as little risk exposure as possible, either to medical malpractice liability or wrongful death suit, both of which they could incur by not authorizing what is medically necessary. To limit this, they adopted (and still hold to) the legal fiction that “medical judgment lies with the doctor; we are simply advising as to what we will or will not pay for”. But since most patients cannot pay for services their insurers won’t pay for, and some insurance plans bar such payments even if doctor and patient agree, not paying becomes equivalent to exercising medical judgment. On the other hand, there is only so much money to go around – and if they pay for unnecessary services their business fails. Insurers and reformers alike really DO need a way of establishing what medically appropriate and necessary utilization is.

To meet this need a new industry developed. A variety of players, all with different vested interests, authored “Guidelines” and “Clinical Protocols” and “Criteria” and “Algorithms” for a variety of diseases. (For ease of discussion we’ll call them all “guidelines”.) The authors include medical specialty societies, insurers, utilization review organizations (who review cases for insurers to assess appropriateness of utilization and charges), and for-profit hospitals and providers. These have gradually moved from being based on expert consensus alone to being built around critical and systematic evaluation of the available evidence. Supposedly none of these have legal weight, but are just statements of guidance to “best practices”. Don’t these impinge on a doctor’s autonomy? Yes… but doctors have at least two strong incentives for accepting them. Many states legislated that deviating from established guidelines was NOT in itself malpractice, but adhering to them was an affirmative defense against a malpractice suit. The other is that deviating from an insurer’s guidelines makes you eat the cost of what the insurer won’t pay and the patient can’t; consistent deviation gets your contract with that insurer and access to its patients terminated.

This clarified the situation… some. Most such documents are written to draw limits around acceptable practice but not micromanage behavior within those limits. Guidelines written by insurers and utilization review consultants are in general the most restrictive and micro-managerial, because their vested interest lies in paying out as little as possible. Guidelines written by investor-owned for-profit providers tend to be the broadest because their return on investment depends on providing as many services – especially high-end expensive services - as often as possible; if they directly employ doctors, frequently the doctor’s pay includes an incentive component to maximize services delivered. 

This leaves American medicine and the cause of health care reform in a somewhat dubious situation. Clearly the cost of future reform – like the cost of current insurance – needs to be known or knowable, and depends on offering only those services that are medically necessary. At this point, we DO know what we don’t know… but we don’t know what we know. And how to fill in the many gaps between what we think and what we know is not at all clear.

Of course, doctors, government, and insurers already knew all this. Time for the rest of the American public to be let in on the little secret.