The Effectiveness of Quality Assurance

Pergamon

International Journal for Quality in Health Care, VoL 8, No. 4, pp. 401-407,1996 Copyright Š 1996 Avtdii Donabediui, Published by Eljevier Science Ltd. All righti reserved Printed in Great Britain 1353-4505/96 $15.00+0.00

The Effectiveness of Quality Assurance

AVEDIS DONABEDIAN

Presented on May 30th, 1996, at the Closing Ceremony of the 13th International Conference of the IntcrnationaJ Society for Quality in Health Care, Jerusalem. The author reserves copyright.

401

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

The organizers of this conference intended, from the very first, to adopt as its theme: "The Impact of Quality Interventions in Health Care" and, indeed, we have heard the strains of this theme in its many variations, like the enticing notes of a magic flute, all these many days.

What more appropriate ending to the conference I thought, when asked to be your farewell speaker, than a few parting words about "The Effectiveness of Quahty Assurance". What easier, I went on to think, since this is a subject I have studied and written about during the more than thirty years of my professional life [1,2].

How hasty I was! How reckless! How foolish!

Soon you shall see why.

As the enormity of my task sank in, I stripped it to its bare essentials. I shall speak, I decided, only about one form of quality assurance: that which consists of obtaining information about performance and, based on an analysis of performance in any given situation, leads to modification in behavior: directly, through educational and motivational activities, and indirectly, through adjustments in system design.

Furthermore, I would have in mind, I decided, only clinical care, lopping off all other aspects of organizational performance less central to the patient-practitioner transaction.

But even when so restricted, the subject presents some serious difficulties: in definition, in conceptualization, in documentation, and in presentation.

"Effectiveness" is itself no simple concept. It is to be visualized as a process in a series of steps: introduction; implantation; implementation; modification in behavior; and finally, consequent progress toward health and health-related objectives. It is likely that many of the factors that influence the effectiveness of quahty assur-

ance act continuously throughout this progression. It is also likely that at each stage some factors are more influential than others, and that at some points new factors emerge to become critical. For example, early in the progression, the nature of the intervention and the receptivity to it are dominant factors. At the transition from behaviors to outcomes, the ability to harness most effectively the technology of health care is the more critical variable. Yet, what comes before prefigures what is to come later; and anticipation of what is to come influences what happens at preceding steps.

A similar pattern of modulation and reverberation runs through the many layers of the health care system. At the most general level, there are the societal factors that surround, shape, and profoundly influence the functioning of the health care enterprise. That enterprise is itself differentiated into layers and segments: layers such as the institution, the department, the work group, and the individual, and segments such as the professional and administrative. At each of these levels and in each of these segments, distinctive forces may influence whether or not quahty assurance will be adopted, the form it will take, and how effectively it will be implemented.

The large number of quality assurance interventions, separately and in combination, add another set of complexities to the task at hand.

So does the imperfect state of our knowledge about the effects of these interventions. True enough, there is an extensive literature to draw upon. But much of it is anecdotal; it merely describes what was done, and what seemed to have been accomplished, only in specific locations, during short periods of time. There are very few controlled studies. For example, of the more than 6,000 reports on continuing education gathered by Davis and associates, only 99

402

contexts, and the interventions appropriate to each of these.

In such a study a theory of effectiveness would help, but I know of no such theory. There are, rather, many theories and many competing perspectives. The health care enterprise may be seen as a culture, or a set of cultures, to which the quality assurance effort must adapt, or which may have to be modified if quality assurance is to flourish. Or the health care enterprise may be seen, in a somewhat related fashion, as a system of social interactions in which the example, approval and support of significant others govern behavior. Therefore, it is to this network of social exchanges that quality assurance must be linked [4]. Alternatively, the health care enterprise is endowed with a considerable degree of rationality, so that information and knowledge rule and it is through these that quality assurance must act [5]. Or, perhaps, behavior in the health care system is rational in still another way, that of self-seeking calculation, the advantages sought being economic, social, or professional. Quality assurance must, therefore, aim to contribute to these interests or, at least, not to harm them. Contrariwise, behavior in the health care system may not be as rational as one would like to believe. Rather, it may be governed in part by a variety of psychological and emotional needs, aspirations, and fears [6]. Or, possibly, the health care system is a network of communications, vertical and horizontal; or it is a system of power relationships, or superordination and subordination; or it is all of the aforementioned and other things besides.

In the absence of a unifying theory, one takes refuge in eclectic formulations that draw on several perspectives. The most dominant of these formulations today goes under the name of "total quality management" or some variant of it.

I have before me two reports. One is of an effort to reduce mortality from coronary artery bypass surgery in several States in Northern New England. It flies the banner of "TQM", uses its concepts and methods, speaks its language—and it succeeds. In this case, as in many others, TQM works [7].

The second enterprise, this one in not too-faraway New York State, has the same objectives, but it is conceived and operated by a governmental agency with awsome powers of retribu-

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

were deemed worthy of further analysis. Of these, only two thirds reported a change in behavior, and even fewer spoke of changes in outcomes. Furthermore, the changes observed were often limited to a few of the process and outcome variables studied, they were small, difficult to quantify, and of indeterminate clinical significance [3].

Even rarer than well-designed studies of single interventions are assessments of variants of such methods. Rarer still, to the point of nonexistence, are studies that set out to test competing, theory-based strategies of quality assurance.

To present this empirical material, even after rigorous pruning, would be impossible in a talk such as this and if presented, it would lead to the almost foregone conclusion that: every reasonably established method in the armamentarium of quality assurance has been shown to work in some situations. Precertification and secondopinions work. Reminders, feedback, profiling, benchmarking, guidelines, protocols, indicators, detailing, continuing education in its various forms—they all work. Quality circles, quality improvement teams and similar group efforts work. Financial incentives work; professional incentives too. So do regulatory interventions, administrative controls and professional interventions. They all work. Yet no one method is demonstrably superior in every situation, or in most.

One response to this uncertainty is to use a combination of methods, hoping that a cumulative effect, or even a synergy, may emerge.

Fortunately, the methods at hand do fall into reasonable constellations or sequences that promise mutual reinforcement. Guidelines, feedback, professional persuasion and continuing education form one such sequence. There could also be an interaction between external regulatory requirements and internal administrative or professional initiatives—an interaction that is mutually supportive rather than antagonistic.

Another response to the current uncertainty in choosing what method is best is to postulate that effectiveness depends not on the method alone, but on an interaction between the method and the situation in which it is to be implemented. One looks, therefore, for a kind of fit between method and situation. The study of effectiveness becomes, then, a study of

A. Donabcdian

The effectiveness of quality assurance

in new directions, willing to take justifiable risks [9].

What is not clear is how the appropriate cultural change is to be achieved. Perhaps it occurs, partly, through the play of external forces: such as governmental pressure, professional aspirations, consumer demand, the play of market forces, and so on. All these imply a manifest or subtle threat to the organization; it must adapt or possibly perish.

Perhaps the factor most often mentioned as a feature of a culture, as well as a modifier of it, is leadership: leadership in every sphere of a society and every level of an organization. The chief executive is a leader; so is the head of a clinical unit; so is a manager; so must be someone in the quality improvement team.

Leadership is often associated with positions of authority; the ability to exercise authority, to influence careers, to reward or censure, is an important adjunct to it, even if kept in the background. Power relationships are a factor not to be ignored in the adoption and conduct of quality assurance. But other attributes of leadership matter equally, if not more: the ability to persuade, to motivate, to inspire trust, to set a personal example of commitment to and personal participation in the quality assurance enterprise. Furthermore, most clinicians would like to see in charge of the quality assurance apparatus one of their own; a clinician senior in rank and of unquestioned competence.

In part, this preference is related to still another contextual factor, that of sponsorship.

In clinical practice, sponsorship by the relevant professional association (of physicians, nurses, and so on) confers legitimacy on the quality assurance effort as a whole, and more so on the particular guidelines and criteria that pertain to the details of clinical work. It is a resource assiduously to be sought.

Both leadership and sponsorship imply an underlying structure of socially organized relationships. In addition to these, formal organization of the health care enterprise is an almost necessary requirement for the institution and operation of quality assurance activities. Formal organizations provide the arena within which cultural change takes place and where leadership is exercised. They have the means to set the goals of performance, to investigate success or failure, to identify causative factors and to take appro-

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

tion, held in check, but unmistakable. It is traditional, pragmatic. TQM is beyond its ken.

Yet, it also works—at least as well, perhaps better [8].

Can one make sense of all this? Do you see, now, the problem I have faced?

Fortunately, despite all the uncertainties I have portrayed, there are certain themes that run constantly throughout the literature on "effectiveness", themes partly founded on empirical evidence, partly on theory-based expectations, and partly on informed speculation. It is to these themes that I now turn.

To introduce at least a semblance of order into my presentation, I shall divide these themes, rather arbitrarily, into "Contextual" and "Operational".

The context subsumes the general properties of the situation into which quality assurance is to be introduced and in which it is to operate.

These properties may support or handicap quality assurance, or they may only support it in some forms, under restricted conditions.

Among the contextual factors, one encounters at the onset, the notion of "culture", which includes what one believes and values, how reality is seen and interpreted, how one is to behave and how things are to be done. All these are manifested in how important quality is regarded to be, how it is defined, who is to be responsible for it, and through what mechanisms. The role of government is critical to these matters, as is the role of the health professions, of the organizations that finance and provide care and of consumers, in association or individually.

In a step down from the more general to the more particular, one often speaks of the culture within an organization—the microcosm where the issues I have just mentioned come into play.

It is often said that some forms of quality assurance amount to a "thought revolution", one that requires a corresponding cultural change. Some features of that change appear in the clear assumption of responsibility for quality in the highest reaches of an organization, the diffusion of that responsibility throughout all its parts and layers, a corresponding empowerment of personnel and a less authoritarian form of governance. Furthermore, organizations are distinguished into some that resist change and others that seek to learn, are ready to strike out

403

404

1. There is a demonstrable, consequential, legitimate need The awareness of need may derive, as I have already implied, from the play of external forces, or it may be self-generated, or the two may interact. But, no matter how prompted, the need must be regarded as important and clinically relevant. Often, a reasonable first step is an organized effort, through group discussion, to identify needs, and set them in an agreed-upon order of priority. In general, trivialization is deadly, but sometimes one must seize upon something relatively unimportant that a clinical unit wishes to have done, hoping in that way to demonstrate the potential of the quality assurance enterprise to help and to succeed.

In order to be demonstrable and credible, what is needful must be documented with data— data of unimpeachable provenance and quality.

Moreover, the inference to be drawn from the data must, themselves, be persuasive and compelling. Comparisons may be made with normative standards of acceptable legitimacy, either professionally approved or self-generated. Parti-

cipation in the formulation of such guidelines and standards is said to enhance compliance. It is said, moreover, that comparison with the actual performance of peers or of similar institutions tends to be more compelling, and some believe that setting precise, measurable goals in advance, especially concerning outcomes of care, is powerfully motivating, if the goals fail to be achieved [12,13].

The manner of presenting data is also important. More effective than written transmittal is the opportunity to explain and discuss the findings and their interpretation, and even more so if individual performance is discussed in private with a trusted and respected senior colleague [14].

A genuine conviction that performance needs to be improved is the indispensable first step in the process of quality assurance.

2. Something can be done to meet the need What should follow upon a conviction that something needs to be improved is at least a reasonable expectation that improvement can be made. Loosely, this falls under the now popular, even alluring, rubric of "empowerment".

Empowerment applies at all levels in an organization: executive, managerial and operational. It applies, in particular, to the quality assurance directorate. This is empowered by the appointment of a chief of considerable stature and authority, who belongs in the highest reaches of an organization, where one can participate in and influence, all decisions that significantly impinge on quality. The directorate is also empowered by having at its disposal the necessary resources: human and material. These include the requisite varieties and levels of expertise. They also include time. Nothing vitiates a quality assurance enterprise, revealing its marginality in an organization, more than its being delegated to persons of relatively little authority, or conducted as an add-on to existing responsibilities, in one's own free time.

These observations apply, as well, to groups or teams that undertake, or are asked to undertake, quality improvement tasks. Quality flourishes if everyone is alert to opportunities to improve it, can communicate these, can suggest how improvements are to be made, and can expect serious consideration, leading to action,

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

priate action. Within organizations, the networks of informal communication and interpersonal influence are concentrated and potentiated, offering thereby a ready vehicle for the processes of quality assurance. When the organizational nexus is underdeveloped, or virtually absent, as in the private practice of ambulatory care, some new organizational structure, formal or informal, is usually needed to allow physicians to recruit resources, develop expertise, and offer mutual support in the effort to improve performance [10].

Let me now turn to my second category: namely, the "operational" factors that influence effectiveness. To help me present these in some order, I shall assume, guided by more general models of health behavior, a rather crude progression of steps, as follows [11].

(1) There is a demonstrable, consequential, legitimate need.

(2) Something can be done to meet the need.

(3) That which will be done, or is done, is the right thing, done in the right way.

(4) There are demonstrable, useful results, free of unforeseen, harmful consequences.

I shall go through these steps in order.

A. Donabedian

The effectiveness of quality assurance where appropriate. Thus, one fosters a sense of optimism, even of adventure, in an organization.

If not, one can expect cynicism at first, and later an apathetic resignation in those who remain, while the best depart.

The necessary next step, therefore, is that action be taken but not any action, only actions that are reasonable and approved.

3. That which will be done, or is done, is the right thing, done in the right way

service to patients is a compelling professional goal, the model of governance proposed is an established feature of professional life, and the methods to be employed are largely epidemiological, with some compatible extensions [15].

In most cases, it is best, it seems to me, to emphasize continuities rather than disjunctions, where possible extending quality assurance activities already present in many health care institutions. But that principle does not hold if what already exists is, itself, externally imposed, discordant, discredited, and demonstrably ineffective. It is better, then, to offer as a replacement not another unfamiliar incursion, but, rather, a return to the purer, more authentic traditions of the health care professions.

Much of what seems new in quality assurance is, in fact, eminently traditional. Professionals wish to monitor their own work, led by one of their own whom they trust and respect. They prefer to study patterns of performance rather than to search for individual miscreants. They would much rather look for causes of failure in underlying processes and structure, than in professional malfeasance. If there are failures in knowledge, judgment, or skill, they would want these to be corrected by education and retraining, not punishment. Furthermore, education would be more effective if specifically directed at discrete, verified needs, conducted in person by respected colleagues, and reinforced, where possible, by individual consultation and advice.

All this is persuant to congruence with professional norms. But it also serves a second principle, that of "ownership". Professional sponsorship and leadership are one prerequisite to ownership. And so is personal participation in the quality assurance enterprise: in setting its goals, in constructing its criteria and standards, in carrying out its processes and, where possible, implementing the changes that it prescribes.

Through "ownership", two other related principles are also served. These are "relevance" and "utility". The purposes and consequences of quality assurance must berelevantto the life and work of those who engage in it, or are to be consumers, so to speak, of its findings and consequences. It operates in the domains these consumers recognize as their own, where they work, where they exercise responsibility, where they can bring about change. Ideally, the quality assurance enterprise will do what its consumers

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

Expectations of what interventions, what disturbances in the accustomed life of an organization, quality assurance is likely to make, are perhaps the major determinant of how warmly it is likely to be received when proposed, or how obstinately opposed. Later, the very first actions taken can justify what was hoped for, or either confirm or begin to allay the fears that almost any change in the established order is certain to arouse. At every step, thereafter, with each new undertaking, the need to gain approval recurs, except that past events, one hopes, have gradually built up trust, and fostered an inclination to cooperate.

Much of what makes quality assurance interventions acceptable can be made to fall under the rubric of "congruence", which is the degree of fit between the interventions envisaged, and what I earlier called "culture": the culture of the organization as a whole or, of the subcultures of its parts—among the latter, that of the health care professions being the most compelling.

At the very least, one aims for a compatibility with professional ideals, or, better still, a reinforcement of these. A clear commitment to quality, as professionals understand the term, rather than cost-saving mainly, is a necessary bond. So is the resolve to advance the welfare of patients, to reinforce professional responsibility, and to serve the need for professionals to know, and continue to learn. It helps if what is proposed is familiar in rationale and method. It is less disturbing if the concepts and methods of quality assurance are seen to resemble those of the scientific method, which professionals respect, or of clinical problem-solving, in which they are daily engaged. If could be disturbing to ask professionals to adopt concepts and methods ostensibly borrowed from the industrial sector. And it is unnecessary to do so, since

405

406

4. There are demonstrable, useful results, free of unforeseen, harmful consequences The credibility of the quality assurance enterprise hinges on one thing above all else; that something is done as a consequence of its activities, and that this something is demonstrably useful. Let me call this, somewhat fancifully, the principle of "fruition".

What could be more persuasive than to experience, first hand, the benefits of quality assurance? What could more demonstrably confirm an organization's commitment to it?

On the contrary, what could be more destructive to the entire effort than to observe that quality assurance is a tissue of ostentatious pronouncements, or merely busy-work: onerous, boring, unrewarding and useless.

Even worse, would be to experience the undesirable consequences that one has feared from the start, among them: dilution of professional responsibility, distortion of professional judgment, stereotyping of practice, discouragement of innovation, legal hazard and an ambience of fearfulness that leads to resistance,

evasion, concealment and ultimate demoralization.

These dire prognostications are most often, of course, only the hobgoblins summoned forth by the timid, or the merely manipulative, to justify opposition to legitimate quality assurance initiatives. But, sad experience has also shown that, under perverse forms of intervention, such fears can materialize. Therefore, at every step, they are assiduously to be guarded against.

It is now time to end, but on a more hopeful note.

To my mind, the most important single condition for success in quality assurance is the determination to make it work. If we are truly committed to quality, almost any reasonable method will work. If we are not, the most elegantly constructed of mechanisms will fail.

We shall leave this place, I know, determined to hold the stewardship of quality as a sacred trust. Once again, we dedicate ourselves to that high calling.

It is also fitting that, as we leave this city, we offer thanks for its hospitality, and pray earnestly for peace to reign within it. Permit me, therefore, to do so now, in the words of the sweet psalmist himself, first as he spoke, and then in translation [16]:

tofia, ftec

Pray for the peace of Jerusalem : they shall prosper that love thee.

Peace be within thy walls, and prosperity within thy palaces.

For my brethren and companions' sakes, I will now say, Peace be within thee.

Because of the house of the LORD OUT God I will seek thy good.

And now, dear friends, farewell—and God bless us all.

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

would want to see done in the first place, generate information they would like to have, aim for effects they would want to see realized. In short, it is useful.

Sometimes, quality assurance is useful in solving discrete problems that have troubled a clinical unit. At other times, it serves individual aspirations, for example by revealing and rewarding meritorious performance, otherwise unnoticed. Sometimes, an entire profession, nursing for example, is offered new opportunities for personal self-expression and growth, as well as an avenue to professional recognition— even power. Whenever such utilities are manifest, participation in quality assurance is not only welcomed, it is avidly sought.

To summarize, the quality assurance enterprise, if it is to flourish, should conform to the cultural imperatives of those it wishes to influence. But quality assurance is also a force capable, of itself, to bring about a gradual change in that culture, so that, in time, a greater congruence can emerge. Therefore, the quality assurance enterprise must be in for the long haul.

It must be persistent, consistent, meticulously fair, and it must show results.

A. Donabedian

The effectiveness of quality assurance Acknowledgements: I wish to thank Dr. Richard Baker who not only helped me locate references but, also, by sharing his own ideas, shaped some of my thinking as well.

REFERENCES

Changing the clinical behavior of doctors: a psychological framework. Unpublished.

7. O'Connor G T e / al., A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. Journal of the American Medical Association 1996; 275: 841-846.

8. Hannan E L, Improving the outcomes of coronary artery bypass surgery in New York State. Journal of the American Medical Association 1994; 271: 761-213.

9. Shortell S M, Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health Services Research 1995; 30: 377-401.

10. Groll R, Implementation of quality assurance and medical audit: general practitioners' perceived obstacles and requirements. British Journal of General Practice 1995; 45: 548-552.

11. Becker M H, Editor, The Health Belief Model and Personal Health Behavior. Health Education Monograph, Volume 11: 324-508, 1974.

12. Williamson J W, Evaluating quality of patient care: a strategy relating outcome and process assessment. Journal of the American Medical Association 1971; 218: 564-569.

13. Grimshaw J et al., Developing and implementing clinical practice guidelines. Quality in Health Care 1995; 4: 55-64.

14. Eisenberg J M, Changing physicians' practice patterns. Part II, pages 87-142 in Eisenberg, Doctors' Decisions and the Cost of Medical Care.

Ann Arbor: Health Administration Press, 1986.

15. Donabedian A, Continuity and Change in the quest for quality. Clinical Performance and Quality Health Care 1993; 1: 9-16.

16. Psalm 122, verses 6-9.

Downloaded from intqhc.oxfordjournals.org by guest on November 9, 2010

1. Donabedian A, A guide to medical care administration, Volume 11, Medical care Appraisal.

American Public Health Association, 1969. See pages 116-121 on "Implementation", and pages 122-151 on "Effectiveness".

2. Donabedian A, The effectiveness of quality assurance. Part II, pages 59-128 in R H Palmer, A Donabedian and G J Povar, Striving for quality in health care: an inquiry into policy and practice, Ann Arbor: Health Administration Press, 1991.

3. Davis D A, Thomson M A, Oxman A D and Haynes B, Changing physician performance: a systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association 1995; 274: 700-705.

4. Mittman B S, Tonesk X and Jacobson P D, Implementing clinical guidelines: social influence strategies and practitioner behavior change.

Quality Review Bulletin 1992: 18: 413-422.

5. Batalden P and Stoltz P K, A framework for ' continued improvement of health care: building and applying professional and improvement knowledge to test changes in daily work. Joint Commission Journal of Quality Improvement 1995; 19: 424-^52.

6. Robertson N, Baker R and Hearnshaw H,

407