PB.ANUT-E by American Public Works Association

Performance Measurement in Public Works A nuts and bolts guide for public works professionals

William B. Cook

Performance Measurement in Public Works A nuts and bolts guide for public works professionals ÂŠAPWA, August 2000 American Public Works Association 2345 Grand Boulevard, Suite 500 Kansas City, MO 64108-2641 www.apwa.net

Printed in Kansas City, MO

ISBN: 0-917084-87-X

CONTENTS

P r ef ace I nt r o d u ction

What is performance measurement?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why is performance measurement important?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What’s the history of performance measurement?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What does it take to develop and implement a good performance measurement system?. . . . . . . . . . .

1 2 3 4

C h apte r 1 How do we know if we are ready for performance measurement?

Doing a readiness survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Step One – Answer the tough questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Step Two – Determine your basic level of readiness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Step Three – Checklist your current system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Step Four – Discuss it as a group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Now what? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

C h apte r 2 Where are we headed?

Checking in with your mission, vision and values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guidance from the Public Works Management Practices Manual. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Why is strategic direction important?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Real mission, vision, values definitions and examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two basic approaches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The whole system approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing or updating your mission, vision, values in one fine day. . . . . . . . . . . . . . . . . . . . . . . . . . Do we need a facilitator? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The principle of alignment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C h apte r 3 What approach should we use?

In what context do we use performance measurement?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Accountability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Balanced Scorecard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Baldrige National Quality Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Benchmarking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Best Practices Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Canada Awards for Excellence Program. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparative Performance Measurement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GASB Statement No. 34 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Managing for Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management Practices Self Assessment and Accreditation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performance Budgeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Service Efforts and Accomplishment Reporting (SEA). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategic Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total Quality Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . And there’s more . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 16 16 17 20 20 23 23 24

26 27 28 29 31 31 32 33 34 35 36 37 37 38 39 39

CONTENTS

Chapter 4 Organizing the performance measurement effort

Appoint a performance measurement program manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Start with a performance measurement team and work plan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying programs to measure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing a program mission statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying program performance measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setting targets for accomplishment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Training program leaders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Developing a communication plan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40 40 42 42 42 43 43 43

Chapter 5 Mapping out current reporting processes

Why is mapping important? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Using flowcharts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Exploring possible improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Chapter 6 Getting buy-in

Recognizing the major obstacles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Addressing all the key questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connecting with your employees, labor groups, managers and elected officials. . . . . . . . . . . . . . . . . . Committing to full-disclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Communicating directly with employees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chapter 7 What are the needs of the community?

How do we find out what the needs of our community are?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Customer councils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Customer interviews. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Focus groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Telephone hotlines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Town meetings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

48 48 48 49 49

50 50 50 51 51 51 51

Chapter 8 Getting clear on performance measurement definitions

Many terms to consider. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Fully defining each performance measure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Performance measurement definition options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Chapter 9 Focusing on outcomes

Don’t expect everyone to get it right away. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some things really can’t be measured!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Categories of information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taking it to the next level – the “So what?” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56 56 57 57

CONTENTS

Types of outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 What are the consequences? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

C h apte r 1 0 Evaluating performance measures

Criteria for a good set of performance measures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Characteristics performance measures should possess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The realities of performance measurement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategy alternatives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Limitations of performance information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overcoming the limitations of performance information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C h apte r 1 1 Gathering data

Keeping a handle on data collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Different levels of information for different needs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transforming data into information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sources of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Factors to consider when collecting data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The data collection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using trained observers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

C h apte r 1 2 Developing performance information that has real value

Reporting and using performance information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Who should receive the performance information when?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How can the performance information be used? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performance information presentation and reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

60 60 61 62 63 64

66 66 67 68 68 69 70 70

71 71 71 72

C h apte r 1 3 Linking your performance measures to the budget process, performance appraisals, and pay Linking to the budget. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 What affects peopleâ&#x20AC;&#x2122;s performance? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Linking to performance appraisals and pay. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

C h apte r 1 4 Using performance measurement information to manage

Can performance measurement really help me manage?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Asking the right questions and doing something with the results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

C h apte r 1 5 Suggestions for improving performance information usefulness

Address service quality and outcomes explicitly when reviewing services and programs . . . . . . . . . . 77

CONTENTS

Ask program managers to set a target for each performance measure and assess progress. . . . . . . . . . Include indicators of both “intermediate” outcomes and “end” outcomes . . . . . . . . . . . . . . . . . . . . . Include performance measurement in your training programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Incorporate outcome performance requirements into contracts wherever feasible. . . . . . . . . . . . . . . .

77 77 78 78

C h apte r 1 6 Is there a bottom line?

What is measured and reported gets attention!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Measuring program outcomes and quality is not easy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Is there an answer?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

A ppen d i x A Sources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 A ppen d i x B Best Practices in Performance Measurement

Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

A ppen d i x C Mission, Vision, Values, Goals

One-day workshop agenda. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 What facilitation skills do I need? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

A ppen d i x D Surveys

Why survey? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The American Customer Satisfaction Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What do you want to achieve with your survey? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Types of surveys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What resources are required? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What needs to be considered when writing survey questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rules for constructing a survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Survey design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pretesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

87 87 88 89 91 92 93 94 95

A ppen d i x E Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

PREFACE

I have attempted to eliminate much of the glitz and glitter (if you can call it that) surrounding performance measurement and get to the point. That’s why the format of the book is mostly headlines, lists and bullets. My hope is that this information can be used as a practical guideline from a public works perspective. At the same time, I have attempted to provide all of the important tools needed to embark on a performance measurement effort. As the performance measurement project manager for Snohomish County, WA (24 departments with 2,600 employees), I have personally experienced the full breadth of challenges and rewards that come from such an effort. One thing is for sure, it is no small matter to commit to a performance management and measurement system. After writing Public Works Performance Management: Creating Accountability and Increasing Performance for APWA in 1999, it was clear that a more detailed “how to” guide would be needed for performance measurement. This book is meant to be that guide. This is a place to start. It should also be a valuable reference manual as you proceed. I wish you luck as you begin this process. You will need some real staying power and a firm grasp of why you are doing this. Performance measurement is not for the faint of heart.

About the Author Bill Cook is the Executive Office Administrator for the Snohomish County Executive in Everett, Washington. He is a past member of the APWA Management Practices Advisory Committee, APWA Accreditation Council, and APWA Leadership and Management Committee. He was a self assessment clinic instructor, has assisted many agencies around the county with self assessment, and led the self assessment process for Snohomish County’s public works department. Organizational development is one of his many responsibilities as Executive Office Administrator. In this capacity he has developed and implemented a successful performance management system for the county.

INTRODUCTION

“The significant problems we face cannot be solved by the same level of thinking that created them.” - Albert Einstein Editor’s Note: Superscripted footnote numbers throughout text refer to publications listed in Appendix A - Sources

What is performance measurement? When most of us think of performance measurement, we think of big formulas and a relentless stream of potentially useless information. For performance measurement to be useful in our daily work and meaningful to our citizens, it cannot be overly complex. Therein lies one of our challenges – deciding what to measure, how to connect it to the real world of our work, and not get lost in volumes of data and paper. There is no universally accepted term for measuring an organization’s performance. As a result, many terms such as productivity, work measurement, and effectiveness have been used synonymously with “performance measurement.” Performance measurement in its simplest form is an assessment of how well an agency provides services. From a slightly different perspective, it is the regular gathering of specific information regarding the results of the services we offer. After gathering and analyzing this information, you should be able to answer these questions:

• What are the citizens getting for their money? • What was achieved? • How was the quality of the service? • How were the lives of our citizens improved by this effort? • How efficiently was the work done? Managers of any sports team need to know the running score so they can assess whether changes are needed for the team to win. Public works directors, managers, engineers and supervisors need similar information. Defining what a “win” is can be tricky for most government agencies. For private sector businesses the running score is profit and market share. That’s their main thing. Cost alone doesn’t mean much. Measuring that running score and using it for better performance is what performance measurement is all about.1 So what’s the main thing for a public works agency? Why does the agency exist? What do the citizens expect? The generic answer for the main thing question is – Public works agencies exist to build and maintain permanent engineering works or improvements with public and private money and provide services that support these works.

INTRODUCTION

In Public Works Performance Management, performance measurement was defined as:

per-form-ance meas-ure-ment: A process of assessing progress toward achieving predetermined goals, including the efficiency with which resources are transformed into goods and services (outputs), the quality of those outputs and outcomes, and the effectiveness in terms of their specific contributions to program goals. This is an accurate definition, but it is too complex. There are too many elements in the definition to make it meaningful. Creating a definition that is useful and meaningful is a real challenge. The true bottom line from a citizen’s perspective is outcomes – real results. Outcomes address the all important question of “So what?” If we overlay x miles per year, sweep x miles per month, or process x tons of waste per year, So what? Big deal! What performance measure would answer this question and is it meaningful to all of our employees, elected officials and citizens? Based on this perspective, here’s a simplified definition:

Performance Measurement:

An assessment of how an agency performs at providing services. Why is performance measurement important? The most powerful reason for measuring performance is that citizens are continually demanding more responsive and competitive government. Our citizens’ expectations are high and the demands continue to grow as their knowledge of the agency grows. We need to be able to respond in a way that is believable, realistic, and informative. We need to be able to prove that we are good stewards of public funds in order to build and sustain trust. In addition, the Government Accounting Standards Board (GASB) has adopted a resolution strongly encouraging local governments to adopt annual Service Efforts and Accomplishments (SEA) reporting. This type of reporting is a standardized performance report that provides a means for comparing government performance over time and against other jurisdictions. The standards for SEA reporting have been under development for several years and many governmental financial experts anticipate that SEA reporting, or some form of it, may be required in the future.7 There are a number of benefits that come as a result of measuring performance including:

• Strengthened accountability • Enhanced decision-making • Improved customer service • Agencies enabled to determine effective resource use • Strategic planning and goal setting supported

INTRODUCTION

Why measure performance? It is essential because …

• If you don’t measure results, you can’t tell success from failure. • If you can’t see success, you can’t reward it. • If you can’t reward success, you’re probably rewarding failure. • If you can’t see success, you can’t learn from it. • If you can’t recognize failure, you can’t correct it. • If you can demonstrate results, you can win public support.

- Reinventing Government

What’s the history of performance measurement? The need to measure government performance was recognized as far back as the early 1900s by New York City’s Bureau of Municipal Research soon after the city adopted a formal budgeting system. In 1943, a guide for measuring performance was written by Clarence Ridley and future Nobel laureate Herbert Simon and published by ICMA. Measuring Municipal Activities: A Survey of Suggested Criteria for Appraising Administration suggested types of information that local governments might use to monitor and assess how well services are being delivered. It was not long before the federal government turned its attention to performance measurement with the 1949 report to the Commission on Organization of the Executive Branch of the Government, also referred to as the Hoover Commission. This and a second Hoover Commission worked successfully in the years following World War II to streamline and reorganize the federal government, which had grown much larger and more complex in response to the Depression and World War II. These commissions recommended that agencies measure performance and use the information in budgeting, among other functions. In his work with Japan through the 1950s, “quality guru” W. Edwards Deming pitched statistics as the basic means of finding out what any system can do and then designing improvements, as indicated, to help the system become more productive. Although discussion of performance measurement has waxed and waned over time, its use has been an essential ingredient in several budget and management reform movements of the modern era, including planning-programming-budgeting systems in the 1960s, zero-based budgeting in the 1970s, and management by objectives in the 1980s. Research into performance measurement has continued steadily by state and local governments as well as organizations such as the Urban Institute and ICMA. In the mid-1980s, the Governmental Accounting Standards Board (GASB) began a research project into performance measurement that resulted in a series of reports published under the title Service Efforts and Accomplishments Reporting: Its Time Has Come (19891993). GASB concluded that performance measurement and reporting is rapidly developing and is of value to elected officials, citizens, and management. It also recognized that there is need for further work on developing valid and generally accepted indicators, gathering data, and developing methods to verify and present the information.1 In the 1990s, national associations such as GASB, the National Academy of Public Administration, and the American Society for Public Administration passed resolutions calling for the public sector to use

INTRODUCTION

performance measurement and reporting systems. Several states have enacted financial performance reporting standards for state agencies. One reason for the push is the 1993 Government Results and Performance Act. This law mandates creation and support of inspectors general and chief financial officers to fight waste in selected federal agencies and to improve accountability for financial and general management. Strategic plans must be set, performance goals established, and an annual report filed with Congress on actual performance as compared with goals. Some federal agencies now must show results before new appropriations are made. All of these changes were made so that government could manage for results, not just cite rules and regulations as a defense against action. The attention of many performance measurement efforts has turned to outcome-based management. This approach has been developed and utilized by some U.S. cities over the past two decades. In addition, some states are beginning to use it. Foreign countries such as Great Britain, Australia, and New Zealand are well on their way. Sunnyvale, California, a city of 120,000 in the heart of the Silicon Valley, began the experiment 20 years ago. In each policy area, the city defined sets of “goals,” “community condition indicators,” “objectives,” and “performance indicators.” In a normal political process, most decision-makers never spend much time talking about the results they want from the money they spend. With this system, for the first time they understand what the money is actually buying, and they can say yes or no. At least a half dozen states hope to follow in Sunnyvale’s footsteps. Oregon has gone the farthest. In the late 1980s, Governor Neil Goldschmidt developed long term goals, with significant citizen input. He set up the Oregon Progress Board, comprised of public and private leaders, to manage the process. The Board developed goals and benchmarks through 12 statewide meetings and written materials from over 200 groups and organizations. “Oregon,” the Board stated, “will have the best chance of achieving an attractive future if Oregonians agree clearly on where we want to go and then join together to accomplish those goals.” The legislature approved the Board’s recommended 160 benchmarks, measuring how Oregon is performing on three general goals: exceptional individuals, outstanding quality of life, and a diverse, robust economy. Seventeen measures are deemed short-term “lead” benchmarks, related to urgent problems on which the board seeks progress within five years. They include reducing the teen pregnancy rates, enrolling people in vocational programs, expanding access to basic health care, and cutting worker compensation costs. Another 13 benchmarks are listed as “key”— fundamental, enduring measures of Oregon’s vitality and health. These include improving basic student skills, reducing the crime rate, and raising Oregon’s per capita income as a percentage of the U.S. average. Governor Barbara Roberts translated the broad goals and benchmarks into specific objectives for each agency. Objectives were integrated into the budget, giving Oregon the first performance-based budget among the states. Government agencies, professional associations, think tanks such as the Urban Institute, and researchers in universities have suggested many standards of performance measurement. In an era when “do more with less” has become a common adage directed at all levels of government, performance measurement is becoming an essential tool for addressing questions of improvement in terms of outcomes, results, and accountability.

INTRODUCTION

What does it take to develop and implement a good performance measurement system? As reported in Serving the American Public: Best Practices in Performance Measurement by the National Performance Review, the following findings were made to help government agencies develop and implement a performance measurement system:

Leadership is critical in designing and deploying effective performance measurement and management systems. Clear, consistent, and visible involvement by senior executives and managers is a necessary part of successful performance measurement and management systems. Senior leadership should be actively involved in both the creation and implementation of its organizationâ&#x20AC;&#x2122;s systems. In several public and private organizations studied, the chief executive officer not only personally articulated the mission, vision, and goals to various levels within the organization, but was also involved in the dissemination of both performance expectations and results throughout the organization.

A conceptual framework is needed for the performance measurement and management system. Every organization needs a clear and cohesive performance measurement framework that is understood by all levels of the organization and that supports objectives and the collection of results. Some organizations use a balanced set of measures and methodology to organize measures and align them with their overall organizational goals and objectives. Most have a uniform and well-understood structure setting forth how the process works and a clear calendar of events for what is expected from each organizational level and when.

Effective internal and external communications are the keys to successful performance measurement. Effective communication with employees, process owners, customers, and stakeholders is vital to the successful development and deployment of performance measurement and management systems. It is the customers and stakeholders of an organization who will ultimately judge how well it has achieved its goals and objectives. And it is those within the organization entrusted with and expected to achieve performance goals and targets who must clearly understand how success is defined and what their role is in achieving that success. Both organization outsiders and insiders need to be part of the development and deployment of performance measurement systems.

Accountability for results must be clearly assigned and well understood. High performance organizations clearly identify what it takes to determine success and make sure that all managers and employees understand their responsibilities in achieving organizational goals.

Performance measurement systems must provide intelligence for decision-makers, not just compile data. Performance measures should be limited to those that relate to strategic goals and objectives, and that provide timely, relevant, and concise information for use by decision-makers â&#x20AC;&#x201C; at all levels â&#x20AC;&#x201C; to assess progress toward achieving predetermined goals. These measures should produce information on the efficiency with which resources are transformed into goods and services, on how well results compare to

INTRODUCTION

a program’s intended purpose, and on the effectiveness of organizational activities and operations in terms of their specific contributions to program objectives. Collecting data simply because the data is available to be collected, or because having large amounts of data looks good, doesn’t justify the effort. Organizations should choose performance measures that can help describe organizational performance, direction, and accomplishments, and then aggressively use these to improve products and services for customers and stakeholders.

Compensation, rewards, and recognition can be linked to performance measurements. Very few organizations actually link performance evaluations and rewards to specific measures of success. The desire is to tie financial and non-financial incentives directly to performance. Such a linkage sends a clear and unambiguous message to the organization as to what’s important.

Performance measurement systems should be positive, not punitive. The most successful performance measurement systems are not “gotcha” systems, but learning systems that help the organization identify what works – and what does not – so as to continue with and improve on what is working and repair or replace what is not working. Performance measurement is a tool that lets the organization track progress and direction toward strategic goals and objectives.

Results and progress toward program commitments should be openly shared with employees, customers, and stakeholders. Performance measurement system information should be openly and widely shared with the organization’s employees, customers, stakeholders, vendors, and suppliers. Real time access can be maintained on the organization’s Internet and intranet sites for access by various levels of management, teams, and sometimes individuals. Most organizations use periodic reports, newsletters, electronic broadcasts, or other visual media to set forth their objectives and accomplishments.

This not an end, but a beginning … Effective performance measurement systems take time: time to design, time to implement, time to perfect. Performance measurement must be approached as an iterative process in which continuous improvement is a critical and constant objective. Successful organizations use benchmarking to establish performance targets as part of a continuous improvement process. This can include process mapping and comparing these practices with other organizations considered to be the best. The performance measurement process model developed through this study is provided as Appendix B.

How do we know if we are ready for performance measurement?

“If a diamond needs to be cut and polished, it should be prepared to take the friction that goes with the process.” - Greg Mashapa

You wouldn’t be reading this book if you didn’t have some sense of your agency’s readiness for performance measurement. At least you have an interest in knowing more. If you want to assess how ready you are to take this on, this chapter will be helpful. If you know you are ready (or have been commanded to “engage”), you may want to move on to Chapter 2.

Doing a readiness survey Before you plunge into a performance measurement process, there are a few things you should think about. It is very important for you to step back and survey the “lay of the land” before proceeding. Without doing this, the probability of creating another ‘flavor of the month’ is high. There are four possible steps in doing a readiness survey. Step Step Step Step

One: Two: Three: Four:

Answer the tough questions Determine your basic level of readiness Checklist your current system Discuss it as a group

Step One – Answer the tough questions You may not have immediate answers to these questions, but it is important to begin thinking about them. Give it a try. Take a minute and write down your thoughts to these questions:

1 Is our agency already engaged in some sort of performance measurement process? What other initiatives are underway or are on the drawing board?

2 Why are we interested in a performance measurement system? What’s our motivation? 3 What are we signing up for with performance measurement? How much effort is required? 4 What are the real benefits of implementing a performance measurement system? 5 How do we get buy-in for the performance measurement process? Buy-in from whom? 6 How will our agency’s culture impact this effort? 7 How do we handle those who don’t want to participate? How do we overcome negative reactions from our staff? 8 Do we have the resources needed to do performance measurement? 9 How do we organize ourselves to make this process a success?

How do we know if we are ready for performance measurement?

To assist you in considering the full array of answers for each question, here are some possible answers. Highlight those that apply to your agency.

1 Is our agency already engaged in some sort of performance measurement process? What other initiatives are underway or are on the drawing board? Mapping out all of the initiatives that are currently underway or are on the drawing board can be revealing – and scary! List the initiative, timeline and who’s involved on one page (if you need more than one page, your agency is a good candidate for initiative burnout). This will help put the performance measurement process into perspective and suggest where you may find some challenges. Do any (or all) of these labels sound familiar? • • • • • • • • • • • • •

Accountability Balanced Scorecard Baldrige National Quality Program Benchmarking Best Practices Review Canada Awards for Excellence Governmental Accounting Standards Board (GASB) Statement No. 34 Managing for Results Management Practices Self Assessment and Accreditation Performance Based Budgeting Service Efforts and Accomplishment Reporting (SEA) Strategic Planning Total Quality Management

2 Why are we interested in a performance measurement system? What’s our motivation? • • • • • • • • • • • • • •

Someone thought it was a good idea. Someone went to a seminar and wants a promotion. We are required by management to do it. We need to create more accountability. We need to show that we are good stewards of public resources. We need to find ways to be more efficient. We want to find savings. We want or need more recognition. To find some tools to help us improve our services. Other agencies are doing it; we can, too. Our peers claim that this is what a well run agency does. Let’s show them we are one. To bring us out of the ice age. We need a fresh coat of paint. We need a compass or a guide to help us move to where we want to be.

3 What are we signing up for with performance measurement? How much effort is required? • • • • •

This is a big project. It’s another flavor of the month. It’s going to take too many resources and too much time. This will be a lot of work, but is a necessary part of public service in today’s world. This is a nice addition to our current management information system.

How do we know if we are ready for performance measurement?

4 What are the real benefits of implementing a performance measurement system? • • • • • • • • • • • • •

Save money. Improve efficiency. Improve effectiveness. Will make all of us look good. Helps clarify budget needs. Helps identify operational and management needs. Promotes teamwork and staff development. Encourages interdepartmental coordination. Identifies duplication and wasted effort. Defines real outcomes that answer the “so what” question. Promotes public awareness. Creates a dialogue with our citizens and reports our progress on issues of importance. Improves communication.

5 How do we get buy-in for the performance measurement process? Buy-in from whom? • • • • • •

We do/don’t have buy-in from management – administration, directors, managers, supervisors. We do/don’t have buy-in from our employees. We do/don’t have buy-in from our unions. We do/don’t have buy-in from our council. We need buy-in from everyone to make it work. Buy-in includes allowing those involved in the process to take the time needed to do the work – we do/don’t have it. • Buy-in includes moral support and encouragement – we do/don’t have it. • Buy-in includes a strong commitment to the process – we do/don’t have it.

6 How will our agency’s culture impact this effort? • There won’t be any impact from our culture. • Some employees may fear that this is the first step in a process that will lead to tougher work standards and a forced speedup of work processes or even layoffs. • People will say that we can’t measure what they do. • Some will say we are measuring the wrong thing. • Everyone will embrace this as an opportunity to improve our services.

7 How do we handle those who don’t want to participate? How do we overcome negative reactions from our staff? • • • • • • •

Sell the benefits of the process. We don’t need to overcome them. The worth of the process will become evident as we progress. Storming is natural, don’t worry about them. Some people don’t like any change - they will be uncomfortable no matter what we do. Some negative is okay, we should encourage diversity of thought. The light goes on at different times for different people, just work with them.

How do we know if we are ready for performance measurement?

8 Do we have the resources needed to do performance measurement? • Resources are hard to come by – they include knowledge, understanding, management support, information systems, a project manager, staff time, etc. • Managers and supervisors who already feel that their resources are stretched too thin will be reluctant to tackle new, time-consuming measurement reporting tasks. • We need to carve out the time for this – something else will need to come off the plate or be delayed. • We are going to need to ask for more resources in order to make this happen.

9 How do we organize ourselves to make this process a success? • • • • • •

Assign a project manager. Each division needs to assign someone with this responsibility. Create a “tiger team” to get everything organized and put together. Work plans should be adjusted to include this new work. The people working on this should do it in addition to their normal work. A new resource should be added to handle all this new work.

Step Two – Determine your basic level of readiness Next, you can determine your basic level of readiness by placing a checkmark below on those characteristics that best describe your agency. There are three basic levels of readiness: “have a lot of work ahead,” “getting ready,” and “ready to take it on.” “Have a Lot of Work Ahead” • • • • • •

We are already on overload from organization-wide initiatives. There is a lack of interest in performance measurement. Performance measurement is a once-a-year event. There is little or no community dialogue regarding what the community wants and needs. Our top management doesn’t really support new or innovative processes. There’s a status quo attitude – don’t rock the boat!

“Getting Ready” • • • •

There is an interest in engaging the community in a dialogue on performance and accountability. Top management has recognized the need for performance measurement. The automated systems of the organization provide lots of options for information feedback. There is energy around really having the agency’s act together and being recognized for excellence.

“Ready to Take It On” • • • • •

We have some experience with performance measurement. We have a strong project manager with excellent people, communication, and technical skills. We are preparing for or are engaged in a management practices self assessment. There is ongoing community dialogue – and it has been happening for some time. There is strong support from our agency management, the organization’s administration, and the legislative body.

How do we know if we are ready for performance measurement?

• We have a high degree of employee involvement in general. • There is a desire to be accountable as an agency. • We have made a commitment to ongoing improvement.

Step Three – Checklist your current system

How does your current performance measurement system stack up? Do a little performance measurement of your own; assess it against this checklist. • Our system starts with big-picture goals – and everyone knows what they are. Any manager can set performance goals for work groups or individuals, but those immediate targets will always seem arbitrary unless people understand how they tie into the big picture. Does your agency have a clear, well-understood mission and goals? Are they widely publicized? • Every work group has performance drivers linked to the big-picture goals. How do people’s jobs tie to the overall goals? Departments and work groups within departments need specific objectives that somehow tie to the big-picture goals. • Individual and work group performance objectives are clear – and reasonable. Targets for performance can be set with any measure that makes sense. In some cases, job-specific goals – meetings held with customers, calls answered by the second ring – can be helpful, so long as the individuals involved are also part of a group with larger objectives. Even people whose work isn’t easily quantifiable should be held accountable for performance objectives. If the performance targets aren’t reasonable or are perceived as unfair, of course employees will dismiss them. On the other hand, you may want to develop stretch goals for the quarter or the year in case the work group blows past the “reasonable” targets. • We educate and coach employees continually. It happens often: agencies launch a performance measurement system with plenty of hoopla and plenty of training. A year later, employees have forgotten what most of the measurements mean – and new employees are utterly mystified. The important task of communicating the connection between workgroup objectives and big-picture goals hasn’t continued. • We have a clear and simple system that tracks progress toward our goals. Most agencies have well-developed systems for tracking financial performance, but not many have equally sophisticated methods of tracking progress on the big-picture goals. What is tracked must be communicated in such a way that employees see and can follow the results. One way to track and share this information is through the use of “scoreboards” that are posted on office walls or on employees’ computer screens.

How do we know if we are ready for performance measurement?

Step Four – Discuss it as a group There are many potential approaches for discussing the merits of a performance measurement system. Those who will be involved in the discussion also needs to be determined. At a minimum, the management team should have a discussion prior to a great deal of time and effort going into the design of the system. “Management team” in this context refers to the public works director and all those managers and/or supervisors who report directly to the director. The discussion approaches range from adding the topic to a regular management meeting agenda, to holding a workshop or retreat, to discussing the topic in the broader context of “where we are headed?” One effective approach is a group exercise called “Readiness Assessment.” The purpose of the exercise is to identify where the agency currently is on the topic and where the group is willing to commit to move the agency in the future. Then there is a discussion regarding what it will take to go from where the agency is are today to where it wants to be. This becomes the action plan. The group can be expanded beyond the management team as desired. Multiple exercises can be held with different groups if desired, e.g. each division of the agency, and the results are combined. A 5x5 grid (drawn on a flipchart) is used to explain the exercise and gather the input from each participant. The x-axis assesses the readiness to do performance measurement. The y-axis assesses the need.

4 N E E 3 D

2 1

0 1 2 3 RE A DINESS

The exercise includes the following steps: • The Readiness Assessment grid is placed at the front of the room on a flipchart. • The public works director provides the background and perspective for why this assessment is being done. • The facilitator explains the purpose of the exercise, the process, and how the information will be used.

How do we know if we are ready for performance measurement?

• If needed, the group can have a facilitated discussion about “What the culture of the agency looks and feels like” and/or “What resources are available for this process? What capacity do we have?” • Group members are then given the opportunity to ask clarifying questions, raise concerns, etc. • The questions and concerns should be addressed by the public works director (or the senior manager present). • Participants are given two dots (different colors, e.g. red, blue) that will represent their opinions on where the agency is currently and at what level they are willing to commit to move the agency. For example, if a participant felt that currently the agency rates a 2 for readiness and a 3 on need, the red dot would be placed at the intersection of 2 on the x-axis and 3 on the y-axis. The blue dot is placed likewise for where the employee is willing to commit to help the agency move. • All participants come forward at the same time to post their votes. • The facilitator reviews the results with the group and determines where the center points are for each color – the current, and the future. • The group discusses what it will take to move from where they are now to where they want to be. The facilitator records the comments on a flipchart. • The group further discusses what the next steps need to be. Assignments are made – who does what by when? (if appropriate). • The facilitator captures the key points of the discussion, the action plan, etc., in a concise report. The report is distributed to all participants. Other facilitation guidelines are described in Chapter 2.

Now What? After going through these four steps, you should have a very good idea whether or not, and how, you should proceed. The readiness survey helps ensure that you go into the performance measurement process with your eyes wide open.

Check out these websites If you feel ready to see what other agencies are doing, you may want to check out these websites for starters:

Cities

• Charlotte, NC www.ci.charlotte.nc.us

• Multnomah Co., OR www.multnomah.lib.or.us

• Portland, OR www.ci.portland.or.us

• Prince William Co., VA www.co.prince-william.va.us

• Sunnyvale, CA www.ci.sunnyvale.ca.us

States

Counties

• Fairfax Co., VA www.co.fairfax.va.us

• Oregon www.state.or.us • Utah www.state.ut.us • Virginia www.state.va.us

How do we know if we are ready for performance measurement?

Federal

• FinanceNet www.financenet.gov • General Accounting Office www.gao.gov • National Performance Review www.npr.gov • Office of Management and Budget www.whitehouse.gov/omb • US State and Local Gateway www.statelocal.gov

Organizations

• Federal Laboratory Consortium • Infolink • Institute of Public Works Engineering Australia (IPWEA) • Institute of Transportation Engineers (ITE) • National Association of Transportation Technology Transfer Centers (NATTTC) • National Council for Public-Private Partnerships (NCPPP)

• Baldrige National Quality Program www.quality.nist.gov

• Professional Grounds Management Society (PGMS)

• Canada Awards for Excellence Program www.nqu.ca

• Solid Waste Association of North America (SWANA)

• GASB Statement No. 34 www.rutgers.edu/accounting/raw/gasb

• US Army Corps of Engineers (USACE)

• International City/County Management Association www.icma.org

Partners with APWA

• All of the following partners are linked at the APWA website – www.apwa.net • 3M, Traffic Control Materials Division • American Consulting Engineers Council (ACEC) • American Water Works Association (AWWA) • Asociacion De Municipios De Mexico, A.C. (AMMAC) • Associated General Contractors (AGC) • Civil Engineering Research Foundation (CERF) • Equipment Maintenance Council (EMC)

• Federal Highway Administration (FHWA)

• Urban and Regional Information Systems Association (URISA) • Water Environment Federation (WEF)

WHERE ARE WE HEADED?

“You’ve got to think about “big things” while doing small things, so that all the small things go in the right direction.” - Alvin Toffler

Checking in with your mission, vision and values Does your organization have clear direction, plans that align, and employees who understand how they fit into the big picture? This is a challenge that faces every organization of every type in all industries. As government agencies, we have particular challenges and opportunities in this regard. The expectations of our citizens are ever changing, increasing and the available resources must be stretched further and further. Is it possible to have clarity on direction? Should we expect our employees to align with the course that is set? Just imagine … • working in an agency where every member, from top management to the newly hired employee, shares an understanding of the agency’s goals and purpose. • participating in a work group where everyone knows how he or she contributes to the agency’s success. • being on a team whose every member can clearly state the needs of the agency’s customers and how the team contributes to satisfying those needs.1 How would you characterize your agency from the “knowing where it’s headed” perspective? Do any of these statements capture the level of interest and action? • We know where we are headed, have aligned ourselves with our community’s vision, and report our progress internally and externally quarterly. • We have everything aligned and linked together – everyone is on the same page, and they hold themselves accountable. • We have a mission, vision and values statement that was developed by top level management, but they are just words on the wall. • There are a few people at the top who know where we are headed, but it’s a secret to everyone else. • We don’t know where we are headed, and everyone seems to feel it would be very difficult to describe in just a few words. • No one really cares about this stuff inside our organization except the elected officials. There are many approaches to defining your direction and aligning your resources. There is a right answer for your agency. Your challenge is to select the approach that best fits your situation and circumstances. It is becoming harder and harder to not be proactive in this area. We must communicate effectively with our citizens and employees or face the consequences – citizen revolt, low morale, etc.

WHERE ARE WE HEADED?

Guidance from the Public Works Management Practices Manual For the public works industry the Public Works Management Practices Manual establishes a set of benchmarks and defines some important terms. This manual was first developed in 1991 through the APWA Research Foundation with the assistance of APWA Institutes and Councils for Professional Development and the Management Practices Advisory Committee. It includes more than 450 public works management practices that detail uniform criteria and procedures to perform all public works services. The manual has been updated twice since 1991 – in October 1995 and again in August 1998. In the first chapter of the manual, entitled Organization, the first management practice pertains to mission, vision, value statements as follows: 1.1 Mission, Vision, Value Statements Statements are developed which define the agency’s mission, vision and values. These statements are approved by the legislative or administrative body overseeing the agency and reviewed periodically. The agency’s mission statement is a concise description of the fundamental purpose for which the agency exists. This statement answers the questions of why the agency exists and who the agency is serving. Mission statements for fundamental areas (streets, water supply, solid waste management, etc.) also may be developed. The agency’s vision statement describes the vision of the agency’s leadership. The role of a leader is to create a vision and set a course for moving toward that dream. Leaders convert dreams into reality. This statement answers the questions of what the leadership of the agency wants to create and where the agency is going. The agency’s values statement establishes the core values which will assist in fulfilling the mission of the agency. This statement answers the questions of what culture the leadership of the agency wants to create and how all agency employees are to act. The values are tangible behaviors that define how each employee is expected to act.

Why is strategic direction important? It is true that if you don’t know where you are headed, you won’t know when you get there. If I don’t know why my organization exists (its mission), how can I know if I am contributing? It is the responsibility of leaders in all organizations to set the direction (the vision) by which business plans and work plans are developed, resources are allocated, and performance is judged. And if there are no clear expectations of how our employees are to treat customers and other employees, then their own value systems prevail. When management fails to provide clear direction, employees invent their own. Mission, vision, value statements are a foundation, an anchor, a guidepost for all to rally around. For example, Abraham Lincoln “preached a vision” of America that continues to serve as a powerful foundation. He provided a clear, succinct, and timely message that united people behind a common cause. Lincoln created opportunities to convey his message in simple, reverent, and patriotic terms. His vision communicated integrity, values, and high ideals that appealed to all people.

WHERE ARE WE HEADED?

“One country, one constitution, one destiny.” (Speech, March 15, 1837) “Liberty and union, now and forever, one and inseparable.” (Speech, January 26, 1830) “That this nation, under God, shall have a new birth of freedom, and that government of the people, by the people, for the people, shall not perish from the earth.” (Speech at Gettysburg, November 19, 1863)

Lincoln’s messages renewed the spirit of Americans. By clearly sharing his vision and then gaining acceptance and commitment, he created a battalion of energy within each person.

Such is our challenge. True leaders at all levels have vision. • • • •

Vision Vision Vision Vision

to to to to

see what’s possible invent a future clarify and communicate a direction set a course and lead people there

Real mission, vision, values definitions and examples Defining the terms (mission, vision, values) is crucial when attempting to create or update these statements. The following definitions and examples are provided to assist you. There are seven types or levels to be considered: Type / Level Institution Organization Department/Agency Division Program Family Personal

Example Constitution of the United States City or County of X Public Works Road Maintenance Street Sweeping the Smith Family Jim Smith

Mission Statements Mission statements answer these two questions: Why do we exist? Who do we serve? A good mission statement has the following qualities: • • • • • •

Is timeless - written as if it will never change Is short enough to remember Reflects the needs and goals of the people who create it Does not include goals - goals change Becomes the creed or constitution Is more than value statements

WHERE ARE WE HEADED?

Mission Statement Examples • • • • • •

Government Agency - Enhancing the quality of life for present and future generations. Nonprofit Agency - Uniting people and resources to build healthy communities. Public Works Department – Provide, control, and maintain public works facilities. Road Maintenance Division - To maintain all city roadways. Street Sweeping Program – To keep our streets clean. Family - To love each other. To help each other. To believe in each other. To have fun together. To wisely use our time, talents, and resources. • Personal - My mission is to give, for giving is what I do best. I will seek to learn, for learning is the key to living. I want to teach my children and others to love and laugh, to learn and grow beyond their current bounds. I will build personal, business, and civic relationships by giving frequently in little ways.

Vision Statements Vision is your picture of the possible future you want to create for your organization, department, program, family, or yourself. If you are a person of vision, you need to do more than stay busy, you need to be busy with actions that move you toward your life and work purposes. Vision statements answer these two questions: What do we want to create? Where are we going?

Vision Statement Examples • Government Agency - We desire to create a safe, secure and healthy community, a sustainable community, and a model public organization. • Nonprofit Agency – Each person’s wellness, health and safety needs are met through caring, giving communities. • Public Works Department - We will create transportation, water, and waste systems that will meet the needs of our children. • Road Maintenance Division – We will maintain our roadways at the quality level desired by our citizens. • Street Sweeping Program – We will meet or exceed the street cleanliness of our citizens. • Family – Each member of our family will achieve their greatest potential and be happy. • Personal - My personal vision or dream is to live happily with my family, to be respected in my field of work, and to live a principle-centered life.

Value Statements Values are our beliefs that underlie our thoughts, words, feelings, and actions. Our values define the way we act. They reflect our cultural background, personal discoveries, and family scripts. And they are the glasses through which we look at the world. We evaluate, assign priorities, judge, and behave based on how we see life through these glasses. Values answer these two questions: What culture do we want? How do we want everyone to act? There are many values from which to choose. A partial list includes:

WHERE ARE WE HEADED?

Accountability Alignment Assertiveness Attitude Caring Commitment Compassion Confidence Cooperation Courage Courtesy Creativity Dependability Determination Discipline Energetic Enthusiasm Equity Esteem Excellence Fairness Flexibility Friendliness

Generosity Helpfulness Honesty Honor Humility Humor Imagination Individuality Innovative Integrity Justice Kindness Knowledge Leadership Love Loyalty Moderation Optimism Participation Partnership Passion Patience Perseverance

Persistence Punctuality Purpose Reliability Respect Responsibility Results-Oriented Risk-Taking Safety Self-Discipline Self-Reliance Service Synergy Tact Teachable Tolerance Trust Trustworthy Understanding Unity Vision Wisdom Worth

It is important to define each of the terms selected so everyone gains a better understanding of what is expected.

Value Statement Examples • Government Agency - In fulfilling our mission, we commit to the following core values: partnership, accountability, respect, integrity, service. • Non-profit Agency – We commit to create a healing environment that role models our values in everything we do: respect, integrity, service, excellence. • Public Works Department – We commit to the following values: respect – everyone’s input will be acknowledged, respected and valued; accountability – everyone has a clear understanding of the direction and goals to be accomplished and is personally and collectively responsible for the outcome; integrity – foster an environment which inspires fair and honest treatment for all; creativity – establish an environment that promotes and inspires creative and innovative ideas and actions. • Road Maintenance Division – same as public works department. • Street Sweeping Program – same as public works department.

WHERE ARE WE HEADED?

• Family – We commit to treat each other with respect, to value individual worth, to have unity in our home, and an atmosphere of love. • Personal - My values (how I want to act): Integrity - being true to my values and remembering what is really important Trust - being trustworthy and openly trusting others Respect - respecting all people in every way and striving to be respected by others Service - providing quality services to my clients, establishing positive working relationships, and serving those in need

Two basic approaches There are many possible approaches to defining your mission, vision, values statement. Two approaches will be presented that represent the extremes regarding involvement – a whole system approach or a group approach (something less than the whole system, e.g., the management team or several representatives of the agency). The group approach may be necessary if there’s not time to get everyone involved or if the situation requires leadership (action from the top is needed) to move forward.

The whole system approach When almost everyone in the whole system is involved, employees of the public works agency in this case, the general term used is a large group intervention. The large group or whole system methods deliberately involve a critical mass of the people affected by change, both inside the agency (employees and management) and outside it (citizens, customers, suppliers). This type of process is based on the principle that involvement leads to commitment. When you involve a critical mass from the whole system, there is a lot of involvement, and when people contribute to a process they tend to be committed to the outcome – they have ownership. There are many different names for these methods including whole-system change, large-scale organizational change, the Conference Model, Future Search, Simu-Real, and Real Time Strategic Change. New names and approaches are being introduced constantly. This whole-system change process allows a critical mass of people to participate in: • • • •

Understanding the need for change Analyzing the current reality and deciding what needs to change Generating ideas about how to change existing processes Implementing and supporting change and making it work

People will support what they help to create. When everyone is involved in the decision process, carrying it out happens faster and with less resistance. And with the whole system involved there is often a more creative solution than a small group can produce. The major downside to this approach is that so many people are involved. There is a major commitment of time involved. The typical format is one to three days in a workshop or retreat setting with follow-up sessions as appropriate. It takes skilled facilitators to design and run a large group process which may be expensive if you do not have that expertise within your agency.11

WHERE ARE WE HEADED?

The City of Sacramento Public Works Department used the whole system approach to develop its mission, vision, values and goals statement and to identify and make desired improvements. Their process is a good example of how to use this approach successfully. The department serves a population of 400,000 in an area totaling 98 square miles with 800 employees. The services provided include maintenance services, technical services, solid waste, on-street parking, and animal care. Public Works Director Mike Kashiwagi saw the need to make some changes and improvements but wanted to tap into the knowledge and experience of his employees. That was the starting point. The primary drivers for change included: improving competitiveness and cost effectiveness – the need to demonstrate value to customers, to retain the privilege to serve; changing customer’s perception that we were doing something to them versus for them; and delivering services based on the needs of the customer versus convenience. To achieve these outcomes required fundamental changes in work systems and work behaviors. The compelling reasons to change included: the need to create an organization which understands, values, and appreciates the needs of our customers and ourselves; the need to create an organization which believes we are not just another governmental entity, but a business which must continually improve in order to stay in business; and the need to create an organization which continually identifies system and organizational improvements which supports quality service and customer satisfaction at a competitive cost versus trying to hold onto and preserve the status quo. The management team selected an approach called “real time strategic change” because it is based on values and represents a process that was compatible with two principles they wanted to drive their change effort – shared leadership and meaningfully involving people. This approach utilizes and leverages the knowledge, talents, creativity, and experiences of everyone in the agency. The outcomes result in improvements because the initiatives are introduced and driven by people most knowledgeable of operations. It supports the message that success cannot be the responsibility of a select few – everyone is responsible and accountable for the success of the agency. This approach is designed to achieve rapid and sustainable changes in the agency by involving all employees of the agency at one time in a largescale meeting. Prior to involving everyone in the agency they undertook some leadership development to build a stronger, more cohesive leadership team. In this process they built a shared understanding of: what business they’re in (mission); where they wanted to be (vision); key focus areas (goals); and for what they stood (values). Then to get a better sense of the current situation and to determine what their customer’s perceptions were the leadership team began a strategic planning process by gathering data from key customers. They interviewed 75 key customers – the business community, neighborhood groups, city officials, and other city departments – and asked: What is your first impression when you think of public works? What are the current or future issues facing the city? What are we doing right/wrong? What suggestions do you have for improvement? Based on the customer feedback and their own knowledge, the leadership team gained a shared understanding of why they needed to change and what they needed to do differently. The leadership team next explored ways to bring employees into the planning process. Many options were considered – top down, bottom up, representative cross section, etc.

WHERE ARE WE HEADED?

The criteria for determining how to proceed with an agency-wide effort included: • Choosing options that achieve their purpose. The focus was on the purpose and reasons why a change was needed. The purpose was the primary guide as they planned all aspects of how to involve everyone in the agency. • Walking the talk. They wanted to begin “walking the talk” of the culture they wanted to create. That meant creating a more inclusive culture – one that was participatory and involved everyone at all levels. • Building support capacity. The structures and systems needed to support the improvement effort was determined. For example, they looked for ways to support a collaborative environment through team activities. Did they have adequate training? Were there appropriate meeting facilities and staff support for these types of activities? • Logistical constraints. To bring together all 800 employees at one time would mean shutting down the entire operation, which of course was not feasible. Creative thinking and a real look at all of the constraints was necessary. • Everything you do sends a message. They realized that during times of change people are hypersensitive and look for underlying meaning in all decisions. A consistent message had to be sent to all employees about the improvement effort. A design team was organized that had representation from the whole agency. The purpose of the design team was to design a process so that everyone’s voice would be heard in creating the future of public works. Face-to-face meetings with employees to explain the need for change, the process for engaging everyone, and soliciting nominations for employees to serve on the design team were held throughout the agency. The 23 member design team was selected from 150 nominations and included a cross section of the agency in terms of ethnic and gender diversity, age, job classification and level, work site location, a balance between field operations and technical staff, all divisions represented, length of service, and an individual’s willingness to serve. After the design team selected and chartered, the team made a presentation to the city’s leadership requesting support and, in some cases, participation. The design team also took on the responsibility of communicating with all employees as the process progressed. Then additional training was conducted for all managers and supervisors. The training included communication, facilitation and coaching skills so they would be prepared to help employees as the agency transitioned to a more customer-focused and business-oriented organization. The design teams work culminated in a series of back-to-back, two-day meetings where all employees came to develop a shared vision and ways to improve the agency (four meetings with 200 employees each). The meetings were called “Workout ‘96” with the theme of “strengthening our future / building our future.” From each of the four groups 25% were selected to attend a final two-day convergence meeting where all suggestions were forged into a written plan of action. The top initiatives were identified that needed to occur to achieve their preferred future. A group of 20 employees volunteered to work with the leadership team to determine how to prioritize and move forward on the initiatives. The top five initiatives were identified along with recommendations for improvement: navigating the flight plan (a team responsible for coordinating the remaining four initiatives); training design, external communications, employee performance evaluation, challenge privatization (staying competitive). The agency’s mission, vision, values and goals were another result of this effort. In May 1998, follow-up sessions were held called “Update ’98 – Imagine… Building Our Future.” At these sessions (again four meetings with 200 employees each) there was an update of what had happened since Workout ’96, an opportunity to generate solutions on how to make things better, and a chance to

WHERE ARE WE HEADED?

reconnect with others in the divisions and departments to discuss common issues. The agency’s “hot issues” were prioritized and a plan developed to address them and reach the preferred future. Appendix C contains the details of their efforts including mission, vision, values, goals, process, and agendas.

Developing or updating your mission, vision, values statement in one fine day If you haven’t developed a mission, vision, value statement and desire to or if your current statement needs to be updated, it is possible to do so without great expense. Some organizations have taken years and gone to great expense to develop this statement. It is true that the process in developing the statement is very important. However, it does not need to take years, months, or even weeks. This group approach is intended to be accomplished in a short time frame. The following format and agenda have been used (with variations) to develop many mission, vision, value statements. The intent is to complete the first draft in a one-day workshop. The one-day format has several facilitation models, concepts, and tools incorporated including: design teams, environmental scanning, appreciative inquiry, brainstorming, problem solving, consensus building, and full participation. This structure is designed to honor everyone’s thoughts, encourage participation, enable hands-on experience, create sustainable agreements, and build teamwork. The workshop is designed to revisit, refresh or create mission, vision, and value statements as defined by public works management practice 1.1 in the Public Works Management Practices Manual.

Prior to the Workshop

A design team meets to detail the workshop including identifying the desired outcomes, location, attendees, etc. The agenda and any “pre-think” pieces are sent to participants one-week prior to the workshop. The workshop is designed for up to 40 participants.

After the Workshop

Within a few days of completion of the workshop, someone needs to prepare a written summary of the results (some facilitators provide this service). This report highlights the key points of the discussion and details the results and action plan. The completed report should be distributed to all participants. You may desire all public works employees to have an opportunity to review and comment on the proposed statement. This can occur by distributing the report and providing a means for feedback or holding discussion sessions. It is important that all employees have an opportunity to provide input before it is finalized. Once final, the statement can be shared with elected officials and administrators, and incorporated into letterhead, posters, signs, advertising, etc., as appropriate. Appendix C contains a sample one-day workshop agenda for developing a mission, vision, values statement.

Do we need a facilitator? To facilitate is “to make easy or more convenient.” The facilitator guides a group toward specific outcomes. Have you ever led a meeting and found it difficult or even impossible to both participate and lead the discussion effectively? You could have benefited from involving a facilitator. There are a number of situations when you might need a facilitator. Here are a few more to consider.

WHERE ARE WE HEADED?

• There’s a lot to be accomplished in a short period of time, and you need someone to keep the discussion on track and moving. • You need an objective, outside resource to provide feedback on the process and dynamics. • You want to use a structured process for exploring new options, creating new direction, or making decisions. Can you lead and facilitate at the same time? Sure you can! We do it every day. And it’s a skill that we develop through experience. At times it’s also very difficult to do both. Sometimes you can anticipate a need to designate someone else to facilitate. There are other times in the midst of a meeting when you need to be clear about which hat you are wearing. Knowing how to facilitate and developing the needed skills will benefit both you and those you are leading. Appendix C contains an outline of facilitation skills and details regarding problem solving skills, recording skills, and achieving consensus.

The principle of alignment In The Power of Alignment, George Labovitz and Victor Rosansky do a wonderful job of defining and describing the principle of alignment. Alignment can be thought of as both a noun and a verb – a state of being and a set of actions. The real power of alignment is when we view it as a set of actions. These actions enable us to: • • • •

Connect our employees’ behavior to our mission, turning intentions into actions Link teams and processes to the changing needs of customers Shape business strategy with real-time information from customers Create a culture in which these elements all work together seamlessly

Alignment is a force. It focuses an organization and moves it forward. Jim Barksdale, the CEO of Netscape, captured the greatest challenge that managers face today–keeping their people and organizations centered in the midst of change–when he said, “The main thing is to keep the main thing the main thing!” There are five basic steps in aligning an organization. They are deceptively simple, hard to implement, and even harder to sustain:

1 carefully crafting and articulating the essence of the organization and determining the Main Thing; 2 defining a few critical strategic goals and imperatives and deploying them throughout the organization;

3 tying performance measures and metrics to those goals; 4 linking these measures to a system of rewards and recognition; and 5 personally reviewing the performance of their people by top level managers to ensure the goals are met. The goal is to create alignment between people, customers, strategy, and process. This is accomplished by staying balanced and focused as illustrated by Labovitz and Rosansky in the following analogy. “If you’ve ever sat in the cockpit of a small plane as it makes its landing approach, you can appreciate the process of alignment. The pilot must sense and respond to a set of interactive variables that change as the plane makes its approach – with many things happening at once. Crosswinds affect the plane’s

WHERE ARE WE HEADED?

orientation to the runway, which must be adjusted. Airspeed must be controlled with the flaps and throttle. The rate of descent, and pitch and yaw of the plane, too, must be adjusted as the plane moves down the glide slope. And there is substantial interaction among these many elements on landing. If the pilot manages them properly, the plane stays aligned with the runway and the glide slope, and it makes a safe landing. Like landing an airplane, aligning a department or an entire organization is an ongoing balancing act that involves setting direction, linking processes and systems, and making constant adjustments. Fail to adjust and you’ll drift. Over adjust and you’ll lurch from one side of your intended course to another. The need for companies to reengineer is in many respects a consequence of past failures to make small, manageable adjustments on an ongoing basis. Alignment relies on two essential dimensions: vertical and horizontal. The vertical dimension is concerned with organizational strategy and the people we rely on daily to transform strategy into meaningful work. The horizontal dimension involves the business processes that create what the customer most values. Both of these dimensions must be in sync – independently and with each other. Once alignment is achieved, performance measures and proper management are needed to keep it that way.”2

WHAT APPROACH SHOULD WE USE?

“Genius is seeing what everyone has seen, and thinking what no one has thought.” - Albert Szent-Gyorgi

In what context do we use performance measurement? When it comes to selecting an approach or model to measure “big picture” results, there are many potential answers. Odds are that some aspects of the approach will be dictated to you by your administration or by external agencies. There are pros and cons to each approach, and performance measurement fits into or can be a part of each approach. Performance measurement is one element, one tool, one part of a performance management system. Performance management: The use of performance measurement information to set performance goals, allocate and prioritize resources, review results, communicate with all stakeholders, and reward performance. The definition of the ideal results oriented system is yet to be written. The very concept is still evolving, and many different approaches have a reasonable chance of success. Whatever approach is taken, however, there’s a real hazard in the assumption that time-consuming measurement systems automatically lead to tangible benefits. Simply measuring outcomes is worthless if nobody uses the outcomes to manage. The best strategic plan has roughly as much value as a map in the glove compartment: Unless you actually use it for guidance, you are going to wind up lost out in the middle of nowhere. Before selecting a performance management approach, you need to be clear about how performance measurement will align with other current initiatives. If you can’t easily explain how it all links together or draw the relationships on a single sheet of paper, it’s time to revisit where you are headed and how you are going to get there. The following performance management approaches are summarized for your review. There are many resources available in the form of books, seminars, and the world wide web on each topic should you need to do an in-depth evaluation. • • • • • • • • • • • • •

WHAT APPROACH SHOULD WE USE?

Accountability There are many levels of accountability – from governments answering to their citizenry to personal accountability of each employee. From a functional perspective, accountability has been defined by the American Accounting Association in the form of a ladder comprising five distinct levels3:

Level 1: Policy accountability – selection of policies pursued and rejected (value). Level 2: Program accountability – establishment and achievement of goals (outcomes and effectiveness).

Level 3: Performance accountability – efficient operation (efficiency and economy). Level 4: Process accountability – using adequate processes, procedures, or measures in performing the actions called for (planning, allocating, and managing).

Level 5: Probity and legality accountability – spending funds in accordance with the approved budget or being in compliance with laws and regulations (compliance).

There are both financial and non-financial accountability issues, and levels of measurement that range from objective to subjective. What is needed for accountability may be different for each level, each program or service, and the situation within the agency and community. If accountability is one of your agency’s values (from your mission, vision, values), then you need to define clearly what accountability means within your agency. The process for doing this can involve all levels of employees, managers and elected officials through focus groups, workshops, surveys, etc. Landing on a single definition that reflects the many perspectives of your employees can be a real challenge. Once this definition is developed, it should be communicated to all employees of the agency. Some agencies have developed workshops to discuss the concepts and engage employees in a dialogue. The accountability program for one agency included several components:

Leadership accountability retreat

Products of the retreat included a definition of accountability, a catalog of reasons to be accountable, definition of accountability measures for leadership, and development of an accountability action plan.

Expectations workshop

As a result of the leadership accountability retreat, a workshop was held with a group of employees to define “expectations,” define the roles for which they needed to develop expectations, and create a list of expectations for each role.

Accountability video

A video was developed featuring the top administrative officer (a county executive in this case). The purpose of the video was to present what accountability means to the organization. A companion brochure was developed highlighting the key points of the video and describing how to “walk the talk” in the organization.

WHAT APPROACH SHOULD WE USE?

Accountability focus groups

To validate the content of the video, the brochure, and a proposed agenda for an accountability workshop for all employees, an employee focus group was conducted. Changes were made based on the feedback.

Accountability training program All employees of the organization were invited to participate in an accountability workshop. The purpose of the workshop was to engage employees in a dialogue about accountability and what it means in the organization. This one-hour workshop was facilitated by internal facilitators.

Quarterly performance report

A new reporting process and format was developed to gather and report progress on organization-wide performance indicators and all department performance measures.

Management performance assessment system

A clearly defined list of expectations was developed for all directors, managers and supervisors in four areas – management of people, management of operations, management of issues, and customer/client relations. A performance contract was designed that defined the commitments being made for each year. At the conclusion of each year, an assessment of commitments versus accomplishments was made during each employee’s performance appraisal. The results influence the receipt or non-receipt of merit pay for the year.

Balanced Scorecard In The Balanced Scorecard Robert Kaplan and David Norton defined this term as: “The balanced scorecard translates an organization’s mission and strategy into a comprehensive set of performance measures that provides the framework for a strategic measurement and management system. The balanced scorecard retains an emphasis on achieving financial objectives, but also includes the performance drivers of these financial objectives. The scorecard measures organizational performance across four balanced perspectives: financial, customers, internal business processes, and learning and growth.” The measures are balanced between the outcome measures – the results from the past efforts – and the measures that drive future performance. They are also balanced between objective, easily quantified outcome measures and subjective, somewhat judgmental, performance drivers of the outcome measures. Many organizations are using this approach as a strategic management system to manage their strategy over the long run in these processes: • • • •

clarify and translate vision and strategy; communicate and link strategic objectives and measures; plan, set targets, and align strategic initiatives; and enhance strategic feedback and learning.

The process for building a balanced scorecard includes:

Define the measurement architecture

Select the appropriate organizational unit and identify the linkages.

WHAT APPROACH SHOULD WE USE?

Build consensus around strategic objectives

Conduct first round interviews, have a synthesis session, and an executive workshop – round one.

Select and design measures

Have subgroup meetings, and an executive workshop – round two.

Build the implementation plan

Develop the implementation plan, have executive workshop - round three, and finalize the implementation plan.

Baldrige National Quality Program The criteria for performance excellence contained in the Baldrige National Quality Program (as defined in the Malcolm Baldrige National Quality Improvement Act of 1987 – Public Law 100-107) help organizations enhance their performance through focus on dual, results-oriented goals: • Delivery of ever-improving value to customers, resulting in marketplace success; and • Improvement of overall organizational effectiveness and capabilities. Although local, state, and federal government agencies are not eligible to receive the national quality award, elements of the criteria are insightful as follows:

Strategy Development (2.1)

Describe your organization’s strategy development process to strengthen organizational performance and competitive position. Summarize your key strategic objectives. • What is your strategic planning process? • How do you consider the following key factors in your process? Include how relevant data and information are gathered and analyzed.

The factors: • • • • • •

Customer and market needs/expectations, including new product/service opportunities Your competitive environment and capabilities, including use of new technology Financial, societal, and other potential risks Your human resource capabilities and needs Your operational capabilities and needs, including resource availability Your supplier and/or partner capabilities and needs

Strategic Objectives

What are your key strategic objectives and your timetable for accomplishing them? In setting objectives, how do you evaluate your options to assess how well they respond to the factors most important to your performance?

Strategy Deployment (2.2)

Describe your organization’s strategy deployment process. Summarize your organization’s action plans and related performance measures. Project the performance of these key measures into the future.

WHAT APPROACH SHOULD WE USE?

Action Plan Development and Deployment •

• • • •

How do you develop action plans that address your key strategic objectives? What are your key short and longer-term action plans? What are your key human resource requirements and plans, based on your strategic objectives and action plans? How do you allocate resources to ensure accomplishment of your overall action plan? What are your key performance measures and/or indicators for tracking progress relative to your action plans? How do you communicate and deploy your strategic objectives, action plans, and performance measures/indicators to achieve overall organizational alignment?

Performance Projection

• What are your two to five-year projections for key performance measures and/or indicators? Include key performance targets and/or goals, as appropriate. • How does your projected performance compare with competitors, key benchmarks, and past performance, as appropriate? What is the basis for these comparisons?

Customer Satisfaction and Relationships (3.2)

Describe how your organization determines the satisfaction of customers and builds relationships to retain current business and to develop new opportunities.

Customer Satisfaction Determination

• What processes, measurement methods, and data do you use to determine customer satisfaction and dissatisfaction? Include how your measurements capture actionable information that reflects customers’ future business and/or potential for positive referral. Also include any significant differences in processes or methods for different customer groups and/or market segments. • How do you follow up with customers on products/services and recent transactions to receive prompt and actionable feedback? • How do you obtain and use information on customer satisfaction relative to competitors and/or benchmarks, as appropriate? • How do you keep your approaches to satisfaction determination current with business needs and directions?

Measurement of Organizational Performance (4.1)

Describe how your organization provides effective performance measurement systems for understanding, aligning, and improving performance at all levels and in all parts of your organization. • How do you address the major components of an effective performance measurement system, including the following key factors? • Selection of measures/indicators, and extent and effectiveness of their use in daily operations • Selection and integration of measures/indicators and completeness of data to track our overall organizational performance • Selection, extent, and effectiveness of use of key comparative data and information • Data and information reliability • A cost/financial understanding of improvement options • Correlation/projections of data to support planning • How do you keep your performance measurement system current with business needs and directions?

WHAT APPROACH SHOULD WE USE?

Analysis of Organizational Performance (4.2)

Describe how your organization analyzes performance data and information to assess and understand overall organizational performance. • How do you perform analysis to support your senior executives’ organizational performance review and your organizational planning? How do you ensure that the analysis address the overall health of your organization, including your key business results and strategic objectives? • How do you ensure that the results of organization-level analysis are linked to work group and/or functional-level operations to enable effective support for decision making? • How does analysis support daily operations throughout your organization? Include how this analysis ensures that measures align with action plans.

Organizational Effectiveness Results (7.5)

Summarize your organization’s key operational performance results that contribute to the achievement of organizational effectiveness. Include appropriate comparative data. • What are your current levels and trends in key measures and/or indicators of key design, production, delivery, and support process performance? Include productivity, cycle time, and other appropriate measures of effectiveness and efficiency. • What are your results for key measures and/or initiatives of regulatory/legal compliance and citizenship? What are your results for key measures and/or indicators of accomplishment of organizational strategy?

Benchmarking Benchmarking is a process for identifying and importing best practices to improve performance. It is not just a simple comparative study or simply copying from other agencies. A good benchmarking analysis produces two types of information: quantitative data that are used to measure performance by and to set future targets; and qualitative information on the key success factors that explain how the benchmarked agency became best-in-class in that function. The following steps will help ensure that a best practice is successfully imported: • • • • • • • • • • •

Determine the purpose and scope of the project Understand your own process Research potential benchmarking partners Choose performance measures Collect internal data on performance measurements Collect data from the partner agency Conduct a gap analysis Import practices to close performance gaps Monitor results Recalibrate based on findings Start the search anew

Best Practices Review A best practices review is a systematic study of variations in service level and design, work processes, and services among similar agencies to identify practices that are cost effective and that might be adopted by

WHAT APPROACH SHOULD WE USE?

other agencies. Practices refer to the processes, practices, and systems identified that when performed exceptionally well are widely recognized as improving an agency’s performance and efficiency in specific areas. At their core, best practices reviews involve examining the performance of a program or function in an agency, identifying practices associated with higher levels of performance, and sharing those practices with potential adopters.4 A best practices review includes five major activities: • • • • •

Systematic analysis of service delivery Improving the understanding of measuring performance Creating a database of effective services Enhancing communication Linking up with other management tools

Canada Awards for Excellence Program The National Quality Institute (NQI) promotes awareness and education of quality principles and practices, and recognizes excellence through the Canada Awards for Excellence program. The NQI developed the Canadian Quality Criteria for the Public Sector as a framework for effective public service organizations and agencies at all levels. The emphasis is on achieving citizen-focused service delivery. The public sector criteria related to performance measurement include:

Leadership

• A mission and mandate statement is in place and has been communicated to all levels in the organization. • Key success factors and priorities have been determined and are linked to strategic direction: for example, the accountability for the organization. • Strategic planning incorporates ambitious objectives necessary to achieve the mission and mandate, and is communicated to all levels in the organization. • Implementation of strategic planning is monitored and reviewed. • Responsibility, accountability and leadership for improvement are shared throughout the organization. • Indicators of effectiveness of leadership in setting strategic direction and demonstrating leadership in the quality principles. • Indicators of the level of understanding in the organization, of the mission, mandate and strategic direction. • The organization evaluates and works at improving its approach to leadership.

Planning

• Key improvement issues have been identified, prioritized, measured and improvement goals set, including any actions regarding external partnering arrangements for the delivery of client services. • Formal assessments, using criteria that reflect quality principles, are conducted to determine the organization’s strengths and opportunities for improvement. • The organization analyzes assessment findings to help determine priorities for improvement. • Indicators of the degree of understanding, throughout the organization, of the priorities and goals established in the improvement plan.

WHAT APPROACH SHOULD WE USE?

• Indicators of effective implementation of the improvement plan throughout the organization. • Levels and trends in quality assessment findings (for example, ratings and/or scores).

Citizen/client focus

• Information is gathered, analyzed and evaluated to determine client/stakeholder needs, including evaluation of potential partnering and/or third party service delivery arrangements. • There are methods and processes in place that make it easy for clients/stakeholders to provide input on their needs, seek assistance and make complaints. • The organization measures client/stakeholder satisfaction to gain information for improvement. • Levels and trends of performance in dealing with client/stakeholder inquiries and complaints are compared to established service delivery standards. • Levels and trends are examined in regard to client/stakeholder appeals, and, when applicable, in regard to product-related areas such as refunds, repairs, and replacement.

People focus

• The organization determines training and development needs to meet goals in the improvement plan, and responds to these needs. • The organization evaluates the effectiveness of training and development programs. • The organization measures people satisfaction at all levels, and links the feedback to future improvement opportunities. • Indicators of the effectiveness of training and education, particularly of quality improvement principles and methods are examined. • Indicators of involvement levels in improvement activities that link directly to the goals and objectives of the organization are determined. • Indicators of awareness and involvement in addressing issues related to well being; for example, health, safety and environmental concerns are identified. • Levels and trends of employee suggestions and ideas are submitted and implemented. • Levels and trends in employee turnover rates, absenteeism, and grievances are evaluated.

Process management

• Key processes are analyzed to determine opportunities for continuous improvement, through incremental refinement and/or fundamental redesign, including potential for reallocation of service delivery. • External information is gathered and used to compare performance and to identify opportunities/ ideas for improvement. • Indicators of the effectiveness of the design process for new services and/or products, such as cycle times and frequency of process design changes are determined. • Levels and trends in process capability and cycle time for key service delivery and/or production processes are examined.

Supplier/partner focus

• Levels and trends of suppliers/service providers in their process capabilities and cycle times are studied. • Levels and trends in the quality and value of provided services and/or products are identified.

Comparative Performance Measurement The ICMA Center for Performance Measurement provides a means for local governments to share data on a range of programs, benchmark their performance to comparable jurisdictions, and improve service

WHAT APPROACH SHOULD WE USE?

delivery through the application of best management practices and efficient use of resources. A major purpose of the effort is to provide a common platform for benchmarking of services and to identify governmental practices that may contribute to high performance that might be adapted by other jurisdictions. A consortium of cities and counties asked ICMA to coordinate this work and to assist in selecting services to be measured, identifying desired outcomes of service delivery, defining indicators, and collecting data. More than 120 agencies participate in the consortium. The service areas that apply to public works include: • Refuse collection – collection, recycling • Road maintenance – road condition, operating and maintenance expenditures, street sweeping • Fleet management – maintenance and replacement by vehicle type Three types of indicators are used: outcome indicators, efficiency indicators, and input indicators.

Governmental Accounting Standards Board (GASB) Statement No. 34 The Governmental Accounting Standards Board issued Statement No. 34, Basic Financial Statements and Management’s Discussion and Analysis for State and Local Governments in June 1999. This statement established new financial reporting requirements for state and local governments throughout the United States. When implemented (June 2001, 2002 or 2003, depending upon annual revenues), it will create new information and restructure much of the information that governments have presented in the past. For the first time, financial managers will be asked to share their insights in a required management’s discussion and analysis (MD&A). This will give an objective and easily readable analysis of the government’s financial performance for the year. This analysis should provide users with the information they need to help them assess whether the government’s financial position has improved or deteriorated as a result of the year’s operations. The MD&A will include: • Comparisons of the current year to the prior year using the new government-wide financial statements (described below) • An analysis of the government’s overall position and results of operations • An analysis of significant changes that occur in funds and significant budget variances • A description of capital asset and long-term debt activity during the year • A description of currently known facts, decisions, or conditions that are expected to have a significant effect on financial position or results of operations. New government-wide financial statements will be required, using accrual accounting and the economic resources measurement focus, for all of the government’s activities. It will report all revenues and all costs of providing services each year. These annual reports are expected to provide a new and more comprehensive way to demonstrate stewardship in the long term, the short term, and through the budgetary process. Use of this new information from a performance measurement perspective needs to be assessed by each agency. It will potentially be very useful if the performance information is aligned with this new data.

WHAT APPROACH SHOULD WE USE?

Managing for Results The managing for results process is a comprehensive approach to focusing an agency on its mission, goals, and objectives. It establishes the accomplishment of those goals and objectives as the primary endeavor for the organization, and provides a systematic method for carrying out the endeavor. It requires the establishment of performance measures and the use and reporting of those measures so that management, elected officials, and the public can assess the degree of success the organization is having in accomplishing its mission, goals, and objectives. The stages of the managing for results process are:

Strategic planning

Strategic planning includes defining the program, identifying needs to be addressed, establishing the program purpose and mission, establishing the managing for results process, and assigning accountability.

Program planning

Program planning includes identifying and setting goals and objectives, identifying outcomes, assessing the ability to address needs, prioritizing goals and objectives, evaluating feasibility of making a difference, deciding on a preliminary strategy, establishing management systems, and identifying outputs, benchmarks and baseline measures.

Setting priorities and allocating resources

This includes developing a budget, prioritizing requests, identifying sources of revenue, feedback on priorities, support for program requests, submission of requests, analysis of requests, and decisions on requests.

Activity planning and organization

Activity planning and organization includes assessing allocated resources, establishing or modifying strategies, outputs, action plans, processes and activities, delegation of duties and authority, establishing interim goals and objectives, establishing diagnostic measures, linking resources and outputs and outcomes, and activity-based costing.

Management of operations

Management of operations includes the management system, management philosophy, communications links, regular feedback on results, contingency planning, controlling costs and quality, and producing goods and services.

Monitoring operations and measuring results

Monitoring includes obtaining information on results, understanding factors affecting results, services provided by agency, services provided by others, explanatory factors, secondary effects, using diagnostic measures, and monitoring costs and revenues.

Analysis of, reporting, and obtaining feedback on results

Analysis includes long and short-term results, explanatory factors, strategies and outputs, verification of performance information, budget versus actual results, performance evaluations and audits, and communication of results to parties to whom the program is accountable.

WHAT APPROACH SHOULD WE USE?

Obtaining feedback on results

Feedback includes citizen and service recipient surveys, and feedback on results.

Management Practices Self Assessment and Accreditation The American Public Works Association (APWA) has developed a self assessment program and an accreditation program to enhance the effectiveness of agencies and their competencies in the public works field. The Public Works Management Practices Manual contains over 450 recommended practice statements that describe the critical elements necessary for a full-service public works agency to accomplish its mission. These statements provide general guidance on what a public works agency should be doing – not how it should be done. This non-prescriptive approach allows each agency to tailor its practices to meet local conditions, be they organizational, geographic, climatic, political, or community-related issues. The statements are not designed to be standards. Self assessment using the Public Works Management Practices Manual is a tool for determining how an agency’s policies, procedures, and practices compare to recommended practices identified by nationally recognized experts in public works. Agencies can use the recommended practices as a model for developing or improving existing practices, to enhance performance, increase productivity, and strengthen employee morale. Accreditation is the final step in a process to recognize public works agencies that have conducted strategic planning using the recommended management practices established by APWA. Accreditation is strictly voluntary. Some agencies complete the self assessment and decide not to proceed with accreditation. The self assessment process includes the following major steps: • ensuring the four essential ingredients are present: committed leadership all the way to the top, a high degree of employee involvement, an organized system of documentation, and commitment to improvement; • becoming familiar with the Public Works Management Practices Manual; • preparing a proposal; • selecting a manager; • preparing an operational plan; • developing a project budget; • setting up a filing system for documentation; • training all staff that will be involved; • orienting all staff regarding the process; • gathering information on each management practice; • determining compliance with each management practice; • keeping track of progress; • reviewing documentation; • inviting peer review if desired; • presenting results to the elected governing body; and • receiving recognition from APWA.

WHAT APPROACH SHOULD WE USE?

The accreditation process includes five phases:

1 Application. 2 Self Assessment – as described above. 3 Improvement – improving in all areas as identified in the self assessment. 4 Evaluation – on-site evaluation team of peers. 5 Accreditation – recognition, continuous improvement, reaccreditation. Performance Budgeting

A performance budget is a budget format that includes performance goals and objectives, and demand, workload, efficiency, and effectiveness (outcome or impact) measures for each program. Program budgeting is an integral part of performance budgeting. Instead of dividing expenditures along departmental lines, a program budget classifies expenditures into groups of activities that are designed to achieve a common purpose. A performance budget provides information on how well these activities are carried out. So performance budgets address costs per unit (e.g. cost per pothole filled) and also ask: • Whether services are being provided at an agreed-upon level of quality? • Whether programs are achieving their goals? The benefits of program and performance budgets include: • providing the ability to monitor and improve productivity; • making linking of performance to budget allocations possible; • holding managers accountable for a given level of outputs and outcomes (as opposed to being held accountable for the consumption of inputs); and • allowing for more decentralized decision making and more creative management. There are four stages to implementing performance budgeting:

1 Creating performance measures. 2 Linking performance measures to budget allocations. 3 Reporting performance accomplishments. 4 Institutionalizing the process. Service Efforts and Accomplishment Reporting (SEA)

In April 1994, the Governmental Accounting Standards Board (GASB) issued Concepts Statement No. 2, Service Efforts and Accomplishments Reporting. This statement focused on one of the objectives from GASB Concepts Statement No. 1, Objectives of Financial Reporting. Concepts Statement No. 1 said that “financial reporting should assist in fulfilling government’s duty to be publicly accountable and should enable users to assess that accountability.” In order to accomplish that objective, financial reporting needs to provide information that will assist users (a) to determine

WHAT APPROACH SHOULD WE USE?

whether current-year revenues were sufficient to pay for current-year services, (b) to determine whether resources were obtained and used in accordance with the entity’s legally adopted budget, and (c) to assess the service efforts, costs, and accomplishments of the governmental entity. Concepts Statement No. 2 further develops the last of these three objectives of financial reporting. It begins with a discussion of the governmental environment and the need for performance measurement reporting, explores the dimensions of governmental accountability, sets forth the elements of performance measurement reporting, elaborates the objectives and characteristics of performance information, and discusses the limitations of performance information and how to enhance its usefulness. Finally, it calls for experimentation with performance measurement and reporting and states that performance measurement reporting is considered an essential part of comprehensive financial reporting for state and local governments. Concepts Statement No. 2 asserts that information provided by governments should be intended to assist in decision making during the public administration system and budgetary cycle. Therefore, ideally a governmental entity should: • Establish and communicate clear, relevant goals and objectives • Set measurable targets for accomplishment, and • Develop and report indicators that measure its progress in achieving those goals and objectives (measures of performance). For example, for governmental entities to have appropriate information for making decisions and assessing accountability, information must be provided about results achieved (service accomplishments or performance) through the use of the resources provided (service efforts) and how those results compare with what was planned. The terms economy, efficiency, and effectiveness often are used to describe the categories of performance information needed.

Strategic Planning APWA chose the term “Strategic Planning for Public Works Agencies” to describe a process that includes developing an agency’s mission, vision and values statements, conducting a needs assessment using the Self Assessment process, establishing goals for improvement, developing an implementation plan and successfully attaining the agency’s goals. An agency that completes the entire strategic planning process is eligible to apply to APWA for agency accreditation. Voluntary agency accreditation is not the end of the process. Accredited agencies should use the strategic planning process to provide a framework for continuous improvement of their policies, practices and procedures. In Strategic Planning for Local Government published by ICMA, strategic planning is defined as: Strategic planning in local government is a systematic process by which a community anticipates and plans for its future. The result is a written document that guides the community toward its future goals. A formal strategic planning process has the following results: • a mission statement for the organization;

WHAT APPROACH SHOULD WE USE?

• • • •

an environmental scan and conclusions about future scenarios in a three- to five-year period; basic goals for the time period in the scan, and goals for the coming one-year period; strategies and action steps that will move the organization toward the goals; and implementation plans that assign responsibilities for action steps.

Total Quality Management Total quality management (TQM) is a system and set of procedures for performing operations analysis on a continuous basis. TQM relies on quantitative methods and the measurement of organizational activities to seek out ways to improve services and products continuously. The philosophy of total quality management is that providing quality services results in increased customer demand, and that continuous monitoring of quality is less costly than having to repair defects and incurring customer ill will. The hallmark of total quality management is a focus on the customer and on labor: asking the customer to define quality and viewing labor as an investment. The primary tool of total quality management is “quality circles,” composed of members of labor and management that analyze a problem or function and recommend improvements. Quality circles include staff with the responsibility or expertise necessary to contribute to solving the problem at hand. Participants base their decisions on carefully gathered statistics. TQM uses performance measurement data as baseline data for conducting a more detailed analysis of a particular program or process.1

And there’s more These thirteen approaches are not meant to be all inclusive or suggested as “the answer.” Each approach has received or is receiving attention in the performance measurement airwaves. There are other approaches currently available and there are certainly more to come.

ORGANIZING THE PERFORMANCE MEASUREMENT EFFORT

“Never tell people how to do things. Tell them what you want them to achieve, and they will surprise you with their ingenuity.” - General George S. Patton

Appoint a performance measurement program manager Every program needs a program manager – even performance measurement programs. This individual needs to possess (or have the capability to obtain) the following qualifications and characteristics to be effective:

Qualifications

• Knowledge of performance management, performance measurement and other improvement approaches. • Project management skills. • Belief that performance measurement is both good for the individual and the agency. • Understanding of the many levels and decision making dimensions of the agency. • Knowledge of systems analysis and flow charting. • Facilitation skills. • Knowledge of communications planning, report writing and graphical presentations. • Respect from within the agency and organization.

Characteristics

• Asks tough questions by: focusing on what needs to be asked, not what they want to say; asking questions as a tool to help gain understanding. • Stays connected with those involved by: remaining accessible; being receptive to bad news; and passing good news up the line. • Listens and communicates effectively by: communicating more precisely – gets to the point; and gets back to them quickly. • Encourages a climate of innovation and creativity; rewarding behavior that leads to improvements; encouraging people to ask “why” they do everything. • Works on controlling things (processes, schedules, etc.), not on controlling people. The responsibilities of the performance measurement program manager include: • • • •

Lead the project team – set agendas, handle logistics, notify attendees, prepare meeting results, etc. Develop a work plan – tasks, timelines, accountability. Develop a communications plan. Facilitate discussions as needed (or seek an outside facilitator if appropriate).

Start with a performance measurement team and work plan Depending upon on the size of the agency, each program may need a work group or performance measurement team to oversee development and implementation of the performance measures. Members could include:

ORGANIZING THE PERFORMANCE MEASUREMENT EFFORT

• • • • • •

Program Manager Members of the program staff Representatives of related programs within the agency A representative of the agency’s central office A representative of the budget office A person knowledgeable about information processing, flow charting, and the agency’s information systems

Working groups of 8 to 12 are usually an effective size, but fewer will work if appropriate. For particularly complex programs, groups may need to be larger. The frequency of meetings is dependent upon the timeline. Initially, regular meetings are important. And this is not a short-term project – it takes three to five years to fully implement a performance measurement system. The work plan for the performance measurement team should include6:

1 Establishing the purpose and scope of the team. 2 Identifying the mission, goals, objectives, and clients of the program (see Chapter 2). 3 Developing a communications plan to announce the program and to detail ongoing communication needs.

4 Training the work group on performance measurement tools and techniques. 5 Identifying the results (outcomes) that the program seeks. 6 Mapping out the current processes. 7 Holding meetings with interest groups, including customers, in order to identify outcomes desired from a variety of viewpoints.

8 Selecting specific indicators for measuring each outcome and other appropriate measures. 9 Setting baselines and performance targets. 10 Identifying appropriate data sources for each indicator and the specific data collection procedures needed to obtain the data. Developing data collection instruments.

11 Identifying

the specific breakouts needed for each indicator, such as breakouts by customer demographic characteristics, organizational unit, geographical location, type of approach used, etc.

12 Identifying appropriate benchmarks against which to compare program results. 13 Developing an analysis plan – ways that the performance data will be examined

to make the

findings useful.

14 Selecting formats for presenting the performance information that are informative and user-friendly. 15 Determining the roles that any program partners (such as project grantees and contractors) with substantial responsibility for service delivery should play in developing and implementing the performance measurement process.

16 Establishing

a schedule for undertaking the above steps, for pilot-testing the procedures, and for making subsequent modifications based on the pilot results.

17 Planning,

undertaking, and reviewing a pilot test of any new or substantially modified data collection procedures.

18 Preparing

a long-term schedule (typically about three years) for implementation, indicating the timing of data collection and analysis relevant to each year’s budgeting cycle, and the persons responsible for each step in the process.

ORGANIZING THE PERFORMANCE MEASUREMENT EFFORT

19 Identifying

the uses of the performance information by agency staff (such as in budgeting and helping improve programs).

Identifying programs to measure The various activities that your agency carries out should be clearly identified and divided into distinct programs. Programs are groupings of routine activities aimed at providing support for a certain service. Groupings of individual activities make up a program. For example, these four activities—street patching, crack sealing, seal coating, and street resurfacing—are activities that constitute the program that is usually called street maintenance. Usually, programs are listed on an organizational chart contained in the operating budget. Programs relate directly with the organizational structure and the managerial areas of responsibility. Programs also correspond with the expenditure classification structure; that is, with the formally recognized cost centers. Choosing which programs to measure is a matter of judgment. On the one hand, programs should not be too few, so that only a tiny portion of services is covered or the information collected is insufficient. On the other hand, too much reporting can be excessively costly, overwhelming and impractical because it is not followed through. Performance measurement systems work best when they concentrate on collecting limited but essential information about basic programs that need the most managerial oversight and where accountability reporting is most important.

Developing a program mission statement Developing a well-articulated mission statement for a program is a very important step. We can measure the performance of a program only if we know what the program should accomplish. A clear mission statement is the best beginning. See Chapter 2 for a process and guidelines for developing mission statements.

Identifying program performance measures The primary focus is on outcomes. However, other types of measures should be considered if they work well. Program inputs are expressed through expended money in an operating budget and the number of man-hours or full-time equivalent employees. Outputs usually represent workload measures, or the quantity of the delivered service to the users. The unit of output—the quantity of each service (output/ activity) being produced by the program – can be included. Efficiency measures the cost (whether in dollars or employee hours) per unit of output or outcome, while productivity indicators combine the dimensions of efficiency and effectiveness in a single measure. For example, in a sanitation department, input indicators will be the amount of labor-hours, the budget of the department, number of vehicles operated, etc. Output measures will include tons of refuse collected, miles of roads cleaned, number of customers served, etc. Efficiency indicators will include labor-hours per ton of refuse collected, dollars spent for one mile of snow removal; and productivity indicators will include measures such as cost per mile of a clean street (i.e., total cost of all road cleaning divided by the total miles of clean streets).

ORGANIZING THE PERFORMANCE MEASUREMENT EFFORT

Setting targets for accomplishment We need to specify the conditions that determine the goals of the program have been met. Identify service effectiveness and quality, and explicitly state how we are going to check whether our objectives have been met. Usually, this is done by expressing the goals of a program through date (or period of time) and quantities (including percentages). For example, for a street cleaning program of the sanitation department, this may mean achieving by the end of the fiscal year 75% cleanliness rating in citizen surveys, and 80% cleanliness rating by trained observers. Targets for accomplishment are not necessarily confined to effectiveness indicators. Agencies can similarly set efficiency and productivity targets, or input/output targets. For example, in the case of resource scarcity, input targets may be very important. In the same street cleaning program, the inputs might be the number of FTEs, the program budget, and the number of street sweepers available.

Training program leaders If the performance measurement program manager doesn’t coordinate all of the program teams, program leaders need to be trained to do so. The performance measurement program manager should lead this training.

Developing a communication plan The primary goal of your communication plan is to create an understanding of and support for the performance measurement program. Effective communication will enlist the involvement and support of employees, managers, elected officials, and citizens. The tools (surveys, meetings, etc.) used to get buy-in and to connect with your community will be defined and described in your communication plan. A good communication plan answers the following questions: • • • • • • • • • • •

What are our communication objectives? Who is our audience? What are our assumptions? How do we test our assumptions? What don’t we know? How will we measure success in accomplishing the objectives? What’s the message? How do we position the message? What tools will be used to deliver the message? What is our implementation strategy? How does this align with everything else that’s going on in our agency?

MAPPING OUT CURRENT REPORTING PROCESS

“The worst thing we can do is do the wrong thing well.” — Irwin Miller

Why is mapping important? In step one of the readiness survey (Chapter 1), the first question is, “Is our agency already engaged in some sort of performance measurement process? What other initiatives are underway or are on the drawing board?” Mapping out all of the initiatives is very important. By listing the initiative, timeline, and who’s involved you start with some clarity on what is already on the plate and the competition for your limited resources. With this as the beginning point, you are ready to analyze your processes. There are several questions that need to be answered for each process. • • • • •

What are the current processes? To whom is the information given? When is the information generated? What sources are used? What systems are used?

Using flowcharts Flowcharting is a tool that helps to graphically display each process, its relationships, boundaries, cycle time, and decision locations. When using flowcharting, it is important to: • • • • •

visualize the process–talk yourself through it; list the major steps and then arrange them in sequence; draw simple, easily understood symbols around or next to each step; connect the symbols with flow arrows; and select those areas that need further details.

You should be asking the following questions when you are flowcharting: • • • • •

Who is doing the process? What is being done? When is it being done? Where is it being done? How much time does it take?

MAPPING OUT CURRENT REPORTING PROCESS

There are many types of flowcharts. The most common include:

Basic flowchart The basic format shows different steps or actions represented by boxes or other symbols. This format can be used effectively to understand the major steps in a complex process. It is a good idea to begin with a basic flowchart.

Step 1

Top-down flowchart The top-down format provides an overview of a process and its major sub-steps or sub-processes. Beginning with the basic format, add the details for each major step. It is helpful to number the detailed steps in a sequential manner to make it easy to follow the relationship between the â&#x20AC;&#x153;parentâ&#x20AC;? process and the sub-processes.

Step 1

1.1 Sub-step 1 1.2 Sub-step 2

Step 1

2.1 Sub-step 1 2.2 Sub-step 2 2.3 Sub-step 3

Step 1

3.1 Sub-step 1 3.2 Sub-step 2

MAPPING OUT CURRENT REPORTING PROCESS

Detailed flowchart

Activity:

When a detailed understanding is needed for a process, additional symbols are used to understand the particular activity that is occurring in each step. The standard set of symbols follow the example below. 1.1

The activity symbol is a rectangle that designates an activity. Within the rectangle is a brief description of the activity.

Decision: The decision symbol is a

diamond that designates a decision point from which the process branches into two or more paths. The path taken depends upon the answer to the question that appears within the diamond.

Flow line: The flow line represents a process path that connects process elements, e.g., activities, decisions. The arrowhead on a flow line indicates direction of a process flow.

1.2

Connector or Link: The connector or

1.3

1.3.1

1.3.2 1.4

1.5

link is a small circle that is used to indicate a continuation of the flow chart on to another column or page. Insert an alpha (a, b, c) or a numeric (1,2,3) in the circle when you come to the end of the page or column. Use that same alpha or number in another circle on the next page or column to continue the process flow. The link can also be used to indicate a place where the process moves to another program or activity.

Terminal: The terminal symbol is a rounded rectangle that identifies the beginning or end of a process. Within the symbol is a brief description of the activity that begins or ends the process.

Document: The document symbol 1.6

represents the production of a paper document such as a form, report or letter that is pertinent to the process.

Electronic information: The electronic

information symbol indicates the use of automation or an electronic transmission. 1.7

MAPPING OUT CURRENT REPORTING PROCESS

Exploring possible improvements Once you have a good understanding of the process, the next step is to explore the possibilities for improvement. The following questions will help you frame the analysis: • • • • • • •

How is the process impacting the program’s performance? Are there changes that would have a positive impact on the program? Are there too many hand-offs to different people or other processes? Are there particular steps where errors are occurring? Are there too many or too few decision points? Can any steps be eliminated? Can any steps be automated to improve the process?

GETTING BUY-IN

“Few things are harder to put up with than a good example.” - Mark Twain

Recognizing the major obstacle One significant obstacle to getting buy-in is the idea that performance measurement information will somehow wind up driving government policy. This assumption is wrong. At best, good performance measurements, like solid strategic planning, merely provide guides for the political process. This may help or hinder your performance measurement process. However, it is important to be up-front about this so everyone understands.

Addressing all the key questions The first three questions your employees are going to think, but probably won’t verbalize, when you announce your performance measurement process (or any other major initiative for that matter) will be “me-based”: • What’s my job? • How am I doing? • Does anybody care? Until these questions are addressed, employees are not able to move on to the next levels. The fourth and fifth questions are more “we-focused”: • How are we doing? • Where do we (our team) fit in? The sixth question is where employees truly get involved: • How can I help? We face a real challenge when we take “macro” messages developed at the organization or agency level and try to translate them into how people do things differently – behaviorally, attitudinally, and functionally – at the work level. This is where most management improvement efforts break down. Not answering these questions, particularly when you are talking about accountability, performance, potential direct impacts on budgets and pay, will create significant obstacles - resistance and even organized efforts to get rid of the program.

Connecting with your employees, labor groups, managers and elected officials Involving front-line supervisors from the beginning is critical. Without the front-line supervisor’s input, understanding and buy-in for the effort will not succeed. Likewise, if you exclude union leaders until the end of the process, they will not support or buy into the program. If you spring requests for

GETTING BUY-IN

information or involvement onto other departments like Human Resources, Finance, and Information Systems when the system is “ready to go,” you should expect resistance and a refusal to dedicate limited resources toward your effort. And if you just start sending new reports to the Mayor, Manager, Executive and Council without laying the groundwork, you will end up with lots of questions, concerns, anger and embarrassment. Buy-in cannot be over-sold. You need to have a plan that ensures all of these (and maybe others) have an understanding of what you are trying to accomplish, what the benefits are, what resources this will require (that can’t be used for other things), and what improvements to services will result. Briefings, presentations, feedback mechanisms, and progress reports are valuable throughout the process.

Committing to full disclosure Some agencies find that a commitment to “full disclosure” is very useful – that means sharing the bad with the good, no sugar coating. Being forthright and direct is a must. It is a big mistake to withhold information. The rumor mill will beat you every time – and a lot of times the rumor mill is true.

Communicating directly with employees One of the ways to beat the rumor mill is through direct communication. Face-to-face communication builds trust and credibility while ensuring that employees are receiving consistent messages. The best solutions for employee communication will always involve interactions, involvement of the various groups, and engaging people in discussion. When face-to-face isn’t possible, all employees should receive the same information via the appropriate electronic or hard copy media. Communicating just once isn’t enough in today’s fast-paced, multi-channel communications environment. The broken-record approach needs to be utilized. The key is to use a combination of channels – face-to-face, print and electronic media. Each employee needs to be touched at least two or three times with the message, and in more than one way. Multiple tools, multiple channels, multiple ways – the more opportunities you can provide for employees to receive key messages, the more likely you are to connect with them.7

CONNECTING WITH THE COMMUNITY

“When a man say him do not mind, then him mind.” — Southern proverb

How do we find out what the needs of our community are? You are already getting feedback from your citizens about what they want and what they think. All public works agencies are very accustomed to the public meeting and public hearing process. In many cases we are asking our citizens to “react” to a proposed plan, project, proposal, or policy change. This is an important process and will likely continue to be the mainstay of public interaction in many communities. There are other dimensions to “connecting with our community” that are more proactive. The importance of citizen involvement cannot be understated. Involvement can help build a sense of responsibility for the community. It can encourage greater interest in the governance of community and in the results of that governance. Typically, citizens focus on results: Is government getting the job done? Is the job being done fairly and ethically? Does the end result provide value for the public money spent? We need to be able to answer these questions in a way that is unbiased and believable. On the other hand, what can we actually measure that provides tangible results to address these questions? We can focus on service quality as a performance measure. When we do this we must also turn our attention to measuring customer satisfaction. To measure customer satisfaction, we must ask our customers what they think. There are a number of ways to do this. The approach(es) you select will take into account the purpose of gathering the information, the timeline and resources available, and the effectiveness of the approach based on your experience with the community. Not all public works agencies have a staff of public involvement specialists. External consulting services may be an option; however, much of the work still needs to done by agency staff. Whenever engaging in a “connecting” effort, it is important to do it well. The image of the agency, its relationship with the citizens, and the impact on elected officials are all at stake at each one of these events. Doing them well or not at all is wise counsel. The following connecting approaches can be considered.

Customer Councils Feedback is given by a group considered representative of the customer of the agency or service. This type of feedback is useful when there are ongoing issues that customers may discuss among themselves, and feel that others can refine and reflect their views, and when customers may prefer to speak with other customers rather than approaching the agency directly.

Customer Interviews A selected group of customers is interviewed systematically. This is similar to a focus group, but individuals are interviewed separately using open-ended questions. It provides greater in-depth information than a survey, but lacks the opportunity to delve into sensitive areas or new issues as well as in a focus group. This approach is useful for explorative research.

CONNECTING WITH THE COMMUNITY

Focus Groups In-person interviews are used in a small group setting with a relatively homogeneous group of individuals selected for an in-depth discussion. This is useful for testing a survey for new questions, or testing a message designed to change opinion, or when addressing a new or possible controversial area or service where reliable measures have not been validated.

Telephone Hotlines A telephone number is provided to the public in order to provide comments, suggestions, etc. While not a statistically valid representation of the population, it does provide a forum to receive feedback on services.

Town Meetings Town meetings are typically one or more elected officials meeting with constituents to get input/ feedback on issues relevant to their district. The scope, format and intent of this approach can vary and change as desired.

Surveys From a performance measurement perspective, surveys can offer the results desired to assess customer satisfaction. There are many approaches to surveying and much to consider. Appendix D contains details regarding surveys in the following areas: • • • • • • • • •

Why survey? The American Customer Satisfaction Index What do you want to achieve with your survey? Types of surveys What resources are required? What needs to be considered when writing survey questions? Rules for constructing a survey Survey design Pretesting

Appendix E contains definitions for the following surveying terms: • • • • • • • • •

Bias Data Population Pretest Proxy or Surrogate Measures Respondent Service Quality Sample Survey

GETTING CLEAR ON PERFORMANCE MEASUREMENT DEFINITIONS

“The definition of insanity is doing the same thing the same way but expecting a different result.” - Michael Tamborrino

Many terms to consider Many terms need to be defined as you develop your performance measurement system. There are also many different definitions for each of these terms, although they are similar in nature. What is important is that you clearly define all of the terms that apply for your agency – not all need apply. To assist you in this effort, alternatives are provided below for each of the most common terms. Three broad categories of performance measures measure service efforts, measure service accomplishments, and those that relate efforts to accomplishments. Performance measures should be reported for services your agency is responsible for providing, whether your agency provides the service directly or contracts it out. Additional explanatory information should also be provided as appropriate. Use these definitions to create your own. The process you employ to create your definitions can range from you deciding what they are to a process that involves employees and citizens. Only you can decide what is most advantageous for your agency and its performance measurement process. After the definitions are developed, they should be reviewed and approved by the mayor, manager, administrator, or executive and the appropriate council or board. This process will ensure that everyone is on the same page from the beginning. Differences of opinion on the definitions can be a major stumbling block when you report results later. You can also advertise the fact that there is agreement on the terms (as appropriate).

Fully defining each performance measure A definition of each performance measure needs to be created so it can be understood by everyone who views it. A typical definition includes: • • • • • • • • • • • • • •

Program name Program manager name Program goal Type of measure Performance measure Specific base-line Performance goals/targets Data requirements Frequency and source of data Actual calculation method Definition of key terms Reports in which the data will appear Graphic presentation that will eventually be used to display the data Other rationale for the measure

GETTING CLEAR ON PERFORMANCE MEASUREMENT DEFINITIONS

Two examples of outcome measures using a target approach follows:

Performance Measure Definition Program: Street Sweeping Program Manager: Jim Smith Program Goal: Continually meet or exceed the desired cleanliness level expected by our residents. Type of Measure: Outcome (target approach) Performance Measure: End Outcome – clean streets. Intermediate Outcome - 85% “acceptable” or better ratings for citizen survey and trained observations. Baseline: 74% “acceptable” or better ratings from the first citizen survey. 80% “acceptable” or better ratings from observations of trained observers. Data Requirements: Survey Results Frequency and Source of Data: Annual citizen survey administered quarterly in specified council districts, annual trained observer survey administered quarterly in specific council districts. Calculation Method: Six levels of satisfaction – terrible, poor, fair, acceptable, good, great. Definition of Terms: Resident = all people who live or own businesses within our geographical boundaries. Satisfaction = the six levels of opinion. Cleanliness = free from dirt, litter, and debris. Streets = those roads within our geographical boundaries that are our responsibility. Reports in which the data will appear: monthly reports to the administration and council, quarterly public newsletter, annual report. Graphics: Five to ten year trend line Rationale: Our city prides itself as being a clean, comfortable, and safe place to live. Cleanliness of our streets is a tangible example of this in the community. The only way to really know how our residents feel in this area is to ask.

GETTING CLEAR ON PERFORMANCE MEASUREMENT DEFINITIONS

Another example for equipment maintenance follows:

Performance Measure Definition Program: Equipment Maintenance Program Manager: Jim Smith Program Goal: All rolling stock is available for use when needed. Type of Measure: Outcome (target approach) Performance Measure: % of downtime (equipment not available when needed). Baseline: 90% or better overall (10% or less downtime). Performance Goals/Targets: 99% of equipment is available when needed. The time period is generally 8:00 a.m. to 5:00 p.m., Monday â&#x20AC;&#x201C; Friday, but also includes emergency use, i.e. snow plowing or emergency sewer or street repair. Data Requirements: Current equipment records. Frequency and Source of Data: The equipment maintenance information system is available on-line and updated daily. Calculation Method: % of downtime. Definition of Terms: Downtime = equipment not available for work use due to need for maintenance or repair. Rolling Stock = refers to self-propelled motorized equipment such as trucks, front-end loaders, autos, etc. Reports in which the data will appear: monthly reports to the administration and council, quarterly public newsletter, annual report. Graphics: Five to ten year trend line. Rationale: If equipment isnâ&#x20AC;&#x2122;t available operators will be idle and critical public services may be postponed and service / efficiency will be decrease.

GETTING CLEAR ON PERFORMANCE MEASUREMENT DEFINITIONS

Performance measurement definition options Appendix E contains definition options for the following performance measurement teams. • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Accountability Aggregation / disaggregation Baseline Data Benchmark Benchmarking Best-in-Class Continuous Improvement Customer Effectiveness / Outcome Measures Efficiency Measures (cost effectiveness) Explanatory Information Indicators Input Measures Measures of Accomplishments Measures of Efforts Measures that Relate Efforts to Accomplishment Metrics Outcomes Output Measures (workload) Performance Auditing Performance Communication Performance Goal Performance Management Performance Measurement Performance Measures Process Owner Productivity Program Quality Strategic Direction Strategic Goal Strategic Objective Strategic Planning Target Timeliness Unit Cost World Class

FOCUSING ON OUTCOMES

“If I had eight hours to chop down a tree, I would spend six of them sharpening the axe.” — Abraham Lincoln

Don’t expect everyone to get it right away Sometimes people have trouble measuring, because they are unclear about what they want to accomplish by doing so. Some people simply don’t know what to measure and how to measure it. Nonetheless, managers keep asking for better measures so they can decide which programs to fund and support. All of us are being asked to prove that our programs are worth the investment. Don’t expect everyone to understand what outcomes are. “Not everyone will welcome outcome measures. People will have trouble developing them. Public employees generally don’t focus on the outcomes of their work. For one thing, they’ve been conditioned to think about process; for another, measures aren’t always easy to develop. Consequently, they tend to measure their work volume, not their results. If they are working hard, they believe they are doing all they can. Public organizations will need several years … to develop useful outcome measures and outcome reporting.”

— Report of the National Performance Review, 1993

Some things really can’t be measured! There are some who say that the really important things in life cannot be measured, made tangible, quantified, packaged, boxed, or tied down. Quantifiable data can be collected about the experience, but that is different from the experience itself. Calipers cannot be put on a dream, on happiness, on excitement, or on motivation. The essence is being in the experience, not standing outside measuring it. People tend to confuse the measurement with the experience. Measurements give useful, widely agreed-upon indicators of progress toward what is important. The measure is not what is important. The progress is. Measuring is not done for the sake of measurement, it is done because people want indicators of their progress toward something unmeasurable and important. For example, if the following results were reported, how would you respond? “We overlaid 25 miles last quarter, 25 percent more than last year, and we are really motivated and happy in our work.” As the leader, I might feel terrific about that. We all understand “25 miles” and “quarter” and “25 percent” in the same way. We know what they mean; we share the same understanding of these numbers. But “motivated” and “happy” and “terrific” are a different matter. Though we all value motivation and happiness and “terrificness,” what is meant individually by these words can be quite different. And if we are out promoting work motivation and happiness and terrificness across the agency, these feelings will be much harder to communicate than last quarter’s paving figures. Our structures and systems do much better with quantities than with qualities. We know how to keep track of numbers and are inclined to reinforce that ability rather than to figure out how to support what is immeasurable and important.

FOCUSING ON OUTCOMES

Another example is a set of values that are part of the mission, vision and values of the agency. Maybe one of your values is “respect.” The agency needs to show that it respects employees, customers, the council, etc., in a variety of ways. Saying it in a couple of speeches and hanging it on the walls of our buildings just won’t cut it. The agency can express this value with structures, systems, policies, practices, and mechanisms. It should demonstrate that kind of support. But success relies on each person internalizing that feeling of being respected. Everyone in the agency needs to have a common understanding of what it means, how to act and not act to honor it, and see others acting in ways that are respectful.

Categories of information There are three main categories of information that need to be measured for all public works agencies: • Financial considerations • Customer satisfaction • Results Depending upon which approach(es), model(s), or initiative(s) your agency has chosen, the structure for gathering performance measurement information will vary. All of the approaches begin with the “big picture” and eventually get down to specific performance measures for each “program or service.” • • • • • • • • • • • •

Big Picture (Agency mission, vision, values, goals; strategic direction) Agency Goals – overarching, ongoing goals Agency Objectives – what will be accomplished during the fiscal year Agency Performance Measures Division Mission Division Goals Division Objectives Division Performance Measures Program Mission Program Goals Program Objectives Program Performance Measures

Taking it to the next level – the “So what?” Measurement, when done well, helps people make better decisions about where to direct resources, what programs to fund, and if they should spend more (or less) in current programs. You spend all this time and energy to end up with data, actual results, trends, and indicators. Then it is time to take it to the next level, to answer the “so what?” question. Let’s return to our paving program example where we overlaid 25 miles last quarter, 25 percent more than last year. So what? Big deal! What difference does this information make? What needs to change to improve things? Were there factors or influences that had an impact on the results that we had no control over? Is 25 miles great or poor? Was the weather the reason so much was accomplished? How much overtime did we have to work? What’s not going to get done since we spent more than budgeted?

FOCUSING ON OUTCOMES

Are we raising expectations for next year? How does this impact the replacement cycle? The important questions need to get answered. When you answer them, the “So what?” is described and everyone can understand it. Using the overlay program described above as an example, the “So what?” answers might look like this.

The “So what?” for our Overlay Program Overlay Program Outcome: Overlay all street surfaces every 13 years. Planned overlay for 2000: 20 miles. Actual Results: 25 miles overlaid in 2000 a 25% increase over 1999. Trend: Overlay Program - Miles Completed 30 25 20 15 10 5 0

Satisfaction Level:

Annual citizen survey: Baseline is 70% acceptable or better. Target is 85%. Trained observer ratings: Baseline is 75% acceptable or better. Target is 90%.

Explanatory Factors: The weather was a major factor in the success of the 2000 overlay program. The normal average temperature (past 10 years) during the paving season is 72 degrees. The average during 2000 was 81 degrees. The normal average rainfall is 6 inches. In 2000 we had 1.5 inches. The conditions were prime for this program. The replacement cycle will be slightly advanced. This one time surge will allow other road sections needing overlay to move up the priority list. Increases in overlay also improve the use of our annual maintenance budget because we spend less time filling potholes and doing crack sealing. Additional revenues were available from the FEMA reimbursement for flooding in the fall of 1999. This one time revenue of $x,xxx,xxx was applied to materials, supplies, and overtime as approved via a supplemental appropriation by council in April, 2000. Our staff worked x,xxx hours of additional overtime to accomplish the higher level of work at cost of $xxx,xxx. The consequences of this action are neutral financially, and the people living in the Citrus Heights, Parkview, and Mountainview neighborhoods are very happy – they were scheduled for overlay two to three years out.

FOCUSING ON OUTCOMES

Types of outcomes There are three main types of outcomes: targets, benchmarks, and change statements. You need to identify the most appropriate outcome type.

Targets

Targets are specific levels of achievement. For example: a 90 percent rating on a satisfaction survey.

Benchmarks

Benchmarks are comparative targets. Comparisons generally relate to other time periods or other agencies. For example, a higher satisfaction rating than other public works agencies in the region, or a 50 percent increase over the 1995 satisfaction rating.

Change Statements

Change statements reflect an increase or decrease in behavior or attitude. For example, an increase in the satisfaction level of our citizens regarding street cleanliness. Consider the following questions when determining the type of outcome. • • • •

What kinds of data are available? What is the time frame for measurement (short term or long term)? Is baseline data available for making comparisons? Can the change actually be measured?

What are the consequences? Outcomes are the consequences of doing or not doing the job. The consequences can affect customers, employees, and the agency. Some general questions to consider include: • • • • • •

Satisfaction, confidence, feelings – How confident and satisfied are people with the results? Accomplishment – Was the goal achieved? Aftermath – Was there any unforeseen fallout? Compliance – What regulations were met or not met? Cost – Were direct and indirect costs incurred or avoided? Image – How did goal achievement affect the status of the job, agency, or person?

EVALUATING PERFORMANCE MEASURES

“One doesn’t discover new lands without consenting to lose sight of the shore for a very long time.” — Author Unknown

Criteria for a good set of performance measures Indicators in a properly developed set of performance measures should satisfy the following criteria established by David N. Ammons: • • • • • • • • •

Valid Reliable Understandable Timely Resistant to perverse behavior Comprehensive Non-redundant Sensitive to data collection cost Focused on controllable facets of performance

Characteristics performance measures should possess Performance measures should meet the characteristics of relevance, understandability, comparability, timeliness, consistency, and reliability.3

Relevance

Performance measures should include data essential to provide a basis for understanding the accomplishment of goals and objectives of the agency that have potentially significant decision-making or accountability implications. Because the purpose of government is to establish and enforce laws, regulate activities, and provide services economically, effectively, and efficiently – not to earn profits – no single measure of performance is readily available to assist citizens in assessing accountability and in making economic, political, and social decisions. A broad variety of performance measures are required to meet the diverse needs of different users of services, to report on the many goals and objectives of the programs and services provided, and address the issues being considered for different decisions and levels of accountability.

Understandability

Performance measures should be communicated in a readily understandable manner. They should communicate the performance of the agency, department, program, or service to any reasonably informed interested party. The use of tables, graphs and charts often help information to be understandable. Performance information should be concise yet fully disclose everything that needs to be shared. Both conciseness and comprehensiveness in reporting performance measures are important because of the number, diversity, and complexity of programs and services. A balance should be achieved among the number of services reported, the performance measures reported, and the capability of people to understand and act on information.

EVALUATING PERFORMANCE MEASURES

Performance information should include explanations about important underlying factors and existing conditions that may have affected performance. Explanatory information should be reported with the measures of performance both for factors over which the agency does and does not have control. Performance information may be accompanied by a description of the way in which the performance measures should be used. This could include comments on the need to consider performance measures in conjunction with explanatory information, the need to consider multiple aspects of performance when assessing results, instances where proxy or surrogate measures are being reported because of an inability to measure an outcome of a service, and the difficulty of using performance information to assess policy accountability.

Comparability

Performance information should provide a clear frame of reference for assessing the performance of the agency and its programs and services. Performance measures, when presented alone, do not provide a basis for assessing or understanding levels of performance. Therefore, performance information should include comparative information. This information may take various forms: for example; comparisons with several earlier years, targets established by the agency, externally established norms or standards of performance, or other comparable agencies.

Timeliness

Performance information should be reported in a timely manner so it is available before it loses its value in assessing accountability and decision-making.

Consistency

Performance information should be reported consistently to allow readers to compare performance over time and gain an understanding of the measures used and their meaning. However, performance measures also need to be reviewed regularly and modified or replaced as needed to reflect changing circumstances.

Reliability

For performance information to be of value, it is essential that it be reliable. To be reliable, information must be verifiable and free of bias, and should faithfully represent what it purports to represent. Therefore, performance information should be derived from systems that produce controlled, verifiable data. The value of a strong internal control structure has long been recognized when dealing with financial information. The same controls are needed for non-financial information.

The realities of performance measurement You can count on a number of realities as you design, develop, implement and improve your performance measurement system. Here are a few of which to be aware:

Acceptance is essential

Acceptance of the measurement process is essential to its success as a performance improvement tool. The process by which you determine what to measure, how to measure, and how to utilize the measures is more important than the actual results themselves.

Audience - for whom are you doing all this?

The audience/user and purpose must be clearly defined. Who are the customers and end-users for the performance measurement system? What are their requirements? What do they feel they need from measurement to help them do a better job managing, problem solving, and decision-making?

EVALUATING PERFORMANCE MEASURES

Participation - the more the better

The greater the participation in the process of creating a performance measurement system, the greater the resulting performance change, and the greater the ease of implementation of future changes based upon the results. This participation includes employees, management, administration, council and the citizens. Measures must be seen to have value well beyond the program performance level. Performance measurement and reporting becomes not only an accountability tool, but also an advocacy tool.

It’s difficult, hard and complex

Performance measurement is hard and complex. Once we accept this, measurement can become less difficult. A complete and effective system of performance measurement will require several years of consistent, incremental work to achieve. One of the reasons performance measurement is difficult is that these measures were not measured before; and the resulting uncertainty dampens enthusiasm substantially for some individuals and work groups. It is complicated by the fact that there is no generally accepted “bottom line” in civic government because there is no scientific or analytical measurement that indicates the relative benefit to society of, for example, less toxic waste or better public transit.

Behavior is impacted

Measurement of any kind will affect the behavior of individuals within your agency - for better or for worse. It has nearly universal capacity to focus attention. Management needs to recognize its obligation to monitor and direct the resulting changes in focus. Reporting performance measures will also affect the behavior of the administration, council and the citizens.

The right measures, the wrong target

It is NOT “the right measures” that you are shooting for. Instead, it is a process and culture for choosing, using and revising measures to assist employees in focusing on achieving improvement over the long run.

Tells about the past

Measurement at its best only tells you something about the history of your performance.

Employees’ attitudes are impacted

No matter how well an employee’s work is planned, managed and measured, the outcome will depend much more on how passionate the employee is concerning the work.

Poor management of the system

Problems related to an agency’s outcomes are much more often related to poor management of the systems than to poor performance.

Watch out for the trivial

It is easy to measure the trivial. It is much more difficult to measure what is truly important and do it in an objective way.

Strategy alternatives There are many different strategies that you can employ as you proceed with your performance measurement system, as follows:

EVALUATING PERFORMANCE MEASURES

Focus on service delivery improvement

Do not make a commitment to measurement, benchmarking or any other process or program, nor to the reporting of heroic results, nor to assigning the blame. Instead, make a commitment to service delivery improvement, adding new activities and deleting old activities.

Improve those things that make a difference

Aim to improve the things that make a difference (those with large costs, large customer value, substantial consequences, etc.).

Measure what employees can translate into direct action

Measuring global hunger is interesting and of monumental importance, but few can apply any direct action in their daily work. For performance measurement to be motivational, those they apply to must be able to see what to do. There must be a “line of sight” between the actions employees can take and what shows on the measure. Being held accountable for measures with no clear means to affect them is negative at any level.

Measure what’s important strategically

Measure what is of value to citizens and will move you toward your vision, not just what is easy to measure or already being measured.

Involve each work group

Have the work group or program team that produces the result develop the measures, perform the measurements, and report the results. Good performance measures need to be: • directional - to confirm that you are on track to reach the goals, • quantitative - to show what has been achieved and how much more is to be done, and • worthwhile - adding more value than they cost to collect and use.

Limitations of performance information Unfortunately, many public employees, elected officials, and media people believe that regularly collected outcome information tells whether the program and its staff were the primary causes of the outcomes. But outcome information provides only a score. The information tells whether one is “winning or losing” and to what extent, but it does not indicate why. What caused a change in the outcome? Many factors could have been the cause. Usually, only if the program has considerable evidence, such as that from an in-depth program evaluation, can clear linkages be obtained between the program and the extent of an outcome. Performance measurement helps focus ad hoc program evaluations on the issues or questions, if this type of in-depth analysis would be warranted and cost-justified. A clearer understanding of the limitations of outcome data can reduce the tendency to blame public employees immediately when performance measures show unfavorable outcomes. A reduction in blaming, in turn, makes the process less threatening to managers and staff and should encourage greater use of outcome information for improving services.

EVALUATING PERFORMANCE MEASURES

We need to explain the nature of outcome data internally, to elected officials, and particularly to the media, so that our agencies are not blamed prematurely and unfairly for negative outcomes. Readers of performance information need to be aware of the limitations so that the information can be used appropriately. The types of limitations include:

One measure wonâ&#x20AC;&#x2122;t work

Generally, a single composite measure cannot adequately communicate the results of providing a service or group of services. It is necessary for readers to use several measures to assess the performance of an agency or program.

Measures are not enough

Performance information does not, by itself, explain why performance is at the level reported, how to improve performance, or the degree to which a service (or other factors) contributed to the outcome reported. Readers may require additional information beyond what can be provided in the report to fully understand the relationship between an outcome and the many factors affecting that outcome. Performance measurement results only report that a condition exists, and does not explain the cause or causes of a trend. Further study may be necessary â&#x20AC;&#x201C; if the cost justifies it.

Relevancy is hard to peg

It may be difficult to determine whether the reported performance measures are the most relevant measures of the achievement of a goal or objective.

Proxies complicate matters

For some services, it may not be possible to measure the most important outcome, so the performance measure reported may be a proxy or surrogate measure that is in some way related to the desired outcome. These measures are more difficult to understand.

The right goals and objectives

Performance information provides data about the achievement of goals and objectives, but does not provide the information needed to assess whether the goals or objectives are the most appropriate ones, and the ones that most clearly reflect the values of the community.

Policy accountability is a whole different game

Performance information does not provide all the data needed to assist in assessing policy accountability. This assessment involves determining the relative value of a service to society or specific groups, or the comparative value of two or more distinct services.

Watch the cost

The cost of gathering data for some performance measures can exceed the value received. In some cases, an acceptable level of imprecision may be needed.

We have limits

Our agencies have limits on influencing community outcomes. Pointing out these situations can be important.

Overcoming the limitations of performance information Many of the limitations of performance measurement can be overcome, and the usefulness of the

EVALUATING PERFORMANCE MEASURES

information enhanced. When done well, performance measurement provides essential information to assist in assessing accountability and making decisions. Here are a few of the ways that these limitations can be mitigated:

Clear communication

The uses of performance information and the potential difficulties associated with its use should be clearly communicated. There is an important educational component in communicating this information, which can include a description of how performance information is a necessary part of the data needed to measure the performance of your agency and the programs and services you provide. Descriptions also could explain that this information is intended to assist the readers in determining whether goals and objectives are being achieved in terms of efforts, outputs, and outcomes and the efficiency of operations.

Explain the “why?”

The descriptive information also could be used to comment that performance measures should be considered in conjunction with explanatory information to understand why performance is at the reported level, and the degree to which the items reported in the explanatory information may have affected the reported results. When proxy or surrogate measures are used, an explanation of what would be considered an ideal measure and why it was not used, combined with an explanation of how to interpret the proxy or surrogate measure, is helpful.

Highlight “other” factors

Even with comparative information, there often is not a clear cause-and-effect relationship between the service provided and the resulting outcome. There may be numerous explanatory factors, completely or partially beyond the control of the agency, that have a significant effect on the results. These factors, when identified, should be reported as explanatory information with the performance measures, together with an explanation of their possible effect.

Going beyond the results

Performance information, even with comparisons and explanatory factors, may not provide sufficient information about why a program or service is performing at the reported level. Therefore, additional information gathered through program evaluations, performance audits, or other means may help readers understand the reasons for a given level of performance.

Be concise, yet complete

Performance information should include only those measures that are essential to provide a basis for assessing accomplishments or that have potentially significant decision-making or accountability implications. However, it is important to balance this need for conciseness with concern for completeness. If the reported performance information is less than comprehensive, the reader may be left with gaps in understanding about the results, or with an incomplete picture of the results. Comprehensive reporting of performance information also helps to prevent selective reporting of only those measures that provide positive results. At the same time, too many measures can confuse and overwhelm readers.

Willingness and openness to change performance measures

We shouldn’t expect our performance measures to be eternal. Times change, technology changes, and we just plain may have missed the mark with our first, second, or fiftieth attempt. Be willing to change or modify the measures.

GATHERING DATA

“Each problem that I solved became a rule which served afterwards to solve other problems.” — Rene Descartes

Keeping a handle on data collection Data collection is a key tool in the performance measurement process. Some of the benefits of data collection include: • • • •

Allowing everyone to work with the facts. Enabling the agency to establish baseline measurement criteria. Providing information to measure the success of implemented improvements. Helping identify or confirm a problem that exists.

Data are gathered and analyzed for each performance measure to determine if – and how well – goals and objectives are being met. It is easy for the data collection and analysis process to get out of hand. It is tempting to take advantage of the many data resources available via the Internet, Intranet, and financial system which, while valuable, should not be undertaken solely for the sake of research. Keep in mind that data are collected and analyzed to get answers. The following guidelines will assist in the data gathering process:

Keep it focused

Ensure that the right data, and only the right data, are collected. Avoid repetitious or unnecessary compilations. Make sure that the questions originally posed by the performance measures are answered.

Keep it flexible

Data should be collected from a variety of sources and through a variety of media. Any one system isn’t necessarily right or wrong. Although using automation is preferable, sometimes manual systems are necessary and even cost-effective.

Keep it meaningful

A few basic, well-aligned measures are better than a number of complex ones.

Keep it consistent

Data collection should be based upon a set of agreed-upon definitions. These definitions need to be universally understood by employees, supervisors, managers, administrators, elected officials, and citizens. Data collected within a framework of understanding can be easily compared and analyzed, allowing subsequent evaluations to be “apples to apples.”

Different levels of information for different needs Each agency, department, division, and program have different needs for the data gathered. These differences need to be reflected in the data collection process.

Operational differences

The data focus for line supervisors and employees relate to daily operations and customer service.

GATHERING DATA

Operational performance data is useful to these groups. These data are often best gathered as part of the employee’s interaction with the customer, e.g., number served, time to process requests, etc.

Tactical differences

The data focus for managers and program managers relate to customer satisfaction (or dissatisfaction). These data are usually collected through customer surveys. Another kind of data that managers and program managers are interested in is program cost. These data come from financial systems. This data is used to react to conditions, and also to implement proactive measures to reduce unnecessary costs. Additional performance information that is useful relates to employee morale, monitoring safety, and identifying skill deficiencies which can be obtained through surveys.

Strategic differences

Senior managers need to determine whether their departments or divisions are meeting or exceeding the expectations defined in their strategic plans. Generally, they target a few vital measures as critical to their responsibilities. Rather than immersing themselves in day-to-day program details, senior managers look for trends. These different focuses result in different uses of the data, different time frames for using it, and differing opportunities for improvement. The frequency that performance measures are collected and analyzed influences your ability to make needed improvements.

Transforming data into information Data analysis in performance measurement is the process of converting raw data into performance information and knowledge. The data collected are processed and synthesized so that informed assumptions and generalizations can be made about what happened. This is then compared to what was expected to happen; if there is a difference, why; and what corrective action might be needed. To ensure that everyone can use and understand data and its analysis, staff must understand some statistical methods such as: • Trend analysis: comparing data from one period to another and noting the percentage change. • Plotting data: upper and lower limits are set to determine how many events fall outside of the norm. • Aggregating and disaggregating data: showing many layers of data combined into one (aggregated) or broken out into meaningful parts (disaggregated). The disaggregation of data is one area that can really help turn data into information. It helps in communicating by providing information at a more meaningful level and by separating variations in performance that may be hidden by aggregated information. For example, a man has drowned in a lake whose overall average depth is 12 inches. The overall average depth of the lake, while accurate, hides the fact that sections of the lake are much deeper. The lack of such breakouts has been a major defect in most performance measurement systems at all levels of government throughout the country. A service may be working well for some types of clients and not for others. It may be working well in some field offices, districts, or facilities, but not well in others. It may be working well for a less difficult workload, but not for a more difficult workload.

GATHERING DATA

Breaking out the aggregate data for outcome and service quality indicators will be much more useful to program staff in assessing where the service has been successful and where it has failed. It will help staff direct their attention to where improvements are most needed. New York Cityâ&#x20AC;&#x2122;s department of sanitation has for many years allocated its sanitation crew effort to those locations that had poor cleanliness ratings, based on the cityâ&#x20AC;&#x2122;s regular trained-observer rating procedure for street cleanliness. Inspectors rate the city streets from 1 to 5, according to a photographic rating scale. As you can see from this example, one of the most important uses for performance information is to help programs allocate their scarce resources to those areas that most need attention. You should try to disaggregate whenever you can. However, there is no standard way to disaggregate. You need to decide what best communicates the results. Most commonly, performance measures are broken down by geographical areas (e.g., streets, neighborhoods, etc.); organizational units (e.g., different facilities within a sanitation department, etc.); and by degree of difficulty of the incoming workload (e.g., mortality rates for different age and risk groups). Other levels of disaggregation include by the size of the jurisdiction, or by type of a service. There are also many automated tools available to perform statistical analysis of the data including charting, graphing, trending, forecasting, etc. Advanced statistical programs are available and staff with spreadsheet knowledge can develop multiple types of analytical models.

Sources of data Data can come from many different sources; therefore, it is important to limit yourself to the data that already exist in your file cabinets, bookshelves, and computer systems, if at all possible. A great deal of time and expense can go into developing new sources of data. If really necessary and justified, it may be appropriate to develop a new source of data. Generally, you want to work with the data you already have â&#x20AC;&#x201C; with the exception of survey information.

Factors to consider when collecting data In order to ensure the collection of accurate, reliable, and timely data, the agency should develop a data collection strategy. Failure to do this may result in vulnerability to questionable data and constant rework of the data as you react to criticisms or requests for credible and complete information. Consider these factors in your strategy:

Define the data

The data definitions must be developed, identifying the attributes to be included as well as those to be excluded from the reporting system. Document the process At a minimum, an outline should be developed that identifies how the data will be collected. This should be developed for each performance measure given that the process may be different for each.

Document data sources

Whether data is obtained from manual logs, check sheets, computer databases, surveys, or focus groups, it is critical to maintain a record of the source. This step is critical because the person collecting the data this year may not be the same person next year. In addition, it is dangerous to rely on memory for tasks

GATHERING DATA

not done daily, weekly or even monthly. Not documenting data sources makes the process vulnerable to inconsistency and inaccuracy in reporting data.

Data manipulation

The design should explain how data has been manipulated to create the reported results. The term “manipulation” used here does not imply the data are being doctored to present a skewed picture. Rather, the term refers to the calculation to determine a numerical relationship.

Explanatory factors

Any contextual or procedural information should be recorded as an explanatory factor. Where assumptions are made, there is always the possibility of misinterpretation of the data. Assumptions should also be included in the explanatory factors. And any factors beyond the agency’s control that influence program outcomes also should be identified and explained.

The data collection process The process for collecting the data needs to include the following considerations:

Define the data to be collected

Brainstorm the types of data to collect for each program. Be sure to involve all those who will be required to provide data or otherwise assist in the effort.

Consider issues related to sample size and frequency of data collection

Determine whether or not sampling is appropriate. Sometimes it is impractical from both a time and expense perspective to collect data on every process or from all members of a population. It often is necessary to rely on sampling to infer characteristics of the whole by examining selected portions.

Design a data collection sheet

The data collection sheet should capture who will collect the data, how often the data will be collected, how the data will be recorded, and the process that will be used (manual or automated).

Test the collection method and refine the steps

Always do this step, no matter how simple you think the data collection is. It will save you rework in the long run.

Work on summarization during data collection

Summarizing data as you go enables you to spot trends early, which is better than waiting for the deadline to begin tallying. Trying to analyze data, especially significant amounts, under the pressure of a deadline, often leads to careless mistakes. Summarizing data as you go along also will avoid unpleasant surprises at the end.

Generate and collect only the data to be analyzed

Do not waste time collecting data that you do not plan to use. A caution: you don’t always know if the data will be fully used. It is better to err, initially, on having too much. Going back to gather data can be painful and time-consuming.

GATHERING DATA

Monitoring targets Each target should be monitored on a continuous basis. Monitoring provides you with the results needed to decide whether or not the target is accomplished. Monitoring gives the program manager an opportunity to keep tabs on the operation of the program, and take corrective action if requested during the program. For instance, if the target calls for 95 percent of citizens to be satisfied with the state of cleanliness and general maintenance of roads by the end of the year, it does not mean that results are going to be checked once every year. Lesser-scale customer surveys can be completed on a quarterly or monthly basis. They will help to see whether everything is going as planned, and check whether there is any seasonal (cyclical) pattern in customer satisfaction. Monitoring will vary depending on the service and target of accomplishment. For most important services (programs), monthly data collection and reporting system should reflect all identified program measures.

Using trained observers Another way of collecting data is through the process of observation. For example, trained observers, using a set of pictures as a guide, can rate the cleanliness of streets. The concept of trained observers involves training people to rate conditions of tangible structures â&#x20AC;&#x201C; streets, sidewalks, signs, buildings etc. This is done by using scales that have been carefully chosen as meaningful to the public as well as to you and by training those observers in such a way that their results could be duplicated. Rater reliability is important in that you want consistency. The training of observers and the selection of scale are key. To help the training process and to make the scale understandable, most of the scales hinge on photographs that illustrate the different levels of the scale in addition to written descriptions. The trained observer ratings are used as input to performance measures. If you are measuring street cleanliness, trained observer ratings on litter is one way to get reliable information. Trained observers also are very useful in dealing with citizen equity issues. Citizens often are concerned that the streets and sidewalks in their area are poor in comparison to those in the rest of the city. A systematic set of measures of the condition of the streets and sidewalks can show whether there is equity across areas or whether the citizen is right and their area is worse than others.

DEVELOPING PERFORMANCE INFORMATION THAT HAS REAL VALUE

“Never mistake motion for action.” — Ernest Hemingway

Reporting and using performance information Performance information needs to be disseminated quickly. The value of the information can decline and even be useless if not made available when decisions are being made. The methods used to disseminate the performance information include hard copy reports, newsletters, scorecards, e-mail, meetings, videoconferencing, use of local area networks, etc.

Who should receive the performance information and when? It is important to develop a schedule for disseminating performance information. following guidelines will work:

Generally, the

• Performance information that is for the whole agency should be disseminated to everyone within the agency at the same time. • Performance information that applies to a work group or individual should be disseminated to the manager, program manager, work group or individual simultaneously. • Performance information that is being shared with administrators, elected officials and/or the public should be reviewed for accuracy before being disseminated.

How can the performance information be used? Performance information can be used for many purposes including:

Making resource allocation decisions

There are important linkages among resource allocation (budgeting), strategic planning, and performance measurement. If there is alignment among the strategic plan, goals, objectives, and performance measures, performance information can be used as one of the factors when allocating resources.

Conducting employee performance appraisals

“Pay for performance” systems are being implemented in some public works agencies. When performance information contributes to these pay systems, it is imperative that clear, consistent performance measures be developed, that these measures be communicated to employees, and that they receive regular feedback on their performance. See Chapter 13 for more information.

Improving processes

In many cases, if performance targets are not met, corrective action or process improvements are undertaken. On the other hand, if performance targets are exceeded they may be reset to establish ‘stretch’ goals. Caution should be taken to not continually increase the target without the involvement of those employees impacted. There needs to be a balance between continuing to improve versus not allowing employees to exceed the expectations.

DEVELOPING PERFORMANCE INFORMATION THAT HAS REAL VALUE

Documenting accomplishments

Actual results and accomplishments can be shared with many audiences, both internal and external to the agency. This is an opportunity to share this information, to educate, and to engage in a dialogue about how to improve.

Performance information presentation and reporting Presentation of the data should be concise, easy to understand, and tailored to the needs of the audience. Citizens, and other external audiences, require summary presentations of the data. An operational manager needs more details and supporting contextual information. A senior manager requires data presentations and displays that focus on bottom-line performance results, allowing him or her to quickly digest information, to focus on problem areas, and be more efficient in making program decisions. The following general guidelines apply to all presentations: • Text: This is generally the simplest way to present data. The data is listed in interpretable order. However, text does not lend itself to visualization of trends. • Tabular: Tables prepared to present information should be clear, simple, and easy to read. Tables can record figures or percentages, and are a good tool for presentation of a range of information in a compact and interpretable form. Percentage tables should claim no more precision than your data warrants. In most situations you don’t need more than one or two decimal places (if any). • Graphical: There are many types of graphical presentations including bar graphs, line graphs, pie charts, pareto charts, run charts, control charts, etc. These visual displays can be very effective. Each type of graph can be more or less effective depending upon the situation. Experimentation graphical display software or advice from a graphic artist may need to be employed. Both management and policy participants in the performance improvement process can utilize service and financial measures more powerfully if they are displayed clearly, i.e. graphically. Tabular compilations of data are useful, primarily for professionals who use that data daily. Other participants in the public policy process—elected executives, legislators, board members, business advisory groups, policy-level administrators, union members, interest groups, the general public and the media—are not always as adept at drawing conclusions from detailed tables as are the professionals who construct them. Most people, however, can draw conclusions when the data is displayed visually, e.g., by the use of bar graphs and trend charts. By relating information about service to information about demography and geography, graphics facilitate the identification and analysis of present and projected problems/services. Performance reporting summarizes all the indicators and compares actual results to the targets. Besides comparisons with targets, reports may include comparisons with: • • • • •

the previous period (and include a year-to-date roll-up); similar jurisdictions; technically developed standards or norms; geographical areas of client groups within the same jurisdiction; and public-sector/private-sector costs and results with similar organizations.

Information should be organized in such a way as to facilitate comparisons. Reporting and monitoring formats can coincide quite frequently. Analysis and action are the logical conclusion of performance measurement. A well-developed system allows you to spot weaknesses and threats, as well as strengths and opportunities. Better knowledge of strengths and weaknesses will give the program manager (and others) an opportunity to diagnose program growth capabilities and take relevant actions.

LINKING YOUR PERFORMANCE MEASURES TO BUDGET PROCESS

PERFORMANCE APPRAISALS AND PAY

“I find that the harder I work, the more luck I seem to have.” — Thomas Jefferson

The challenges and difficulties of linking performance measures to the budget process, performance appraisals, and pay causes many agencies to struggle with these concepts. There is not agreement that any of the three linkages is a good idea or that it is appropriate. On the other hand, how do you really establish accountability if there is no direct impact on employees, supervisors and managers? You are now part of the debate. You need to decide how to address these challenges in your performance measurement program.

Linking to the budget Linking your performance measurement system to the budgeting process is always a good idea. However, it is easier said than done. Most budget systems are used to accomplish a number of purposes: • • • •

allocate resources, control spending, establish service levels (based on resource allocation), and communicate what is to be accomplished with the resources.

There are also several potential budget formats including: • • • • • •

incremental budgeting, line item budgeting, performance budgeting, program budgeting, target-based budgeting, and zero-based budgeting.

Many organizations are moving toward a budget system that monitors performance targets, measures productivity, and establishes accountability. The desire is to know whether each program is achieving the intended results at a reasonable cost. If you have implemented program budgeting and/or performance budgeting, the odds of success are much higher. Before performance measures can be integrated into budgeting in a meaningful way, agreement must be reached on program mission, goals and objectives. With these established, appropriate measures can be selected that will adequately reflect whether progress is being made. Some agencies are linking performance to budget allocations by awarding larger allocations to managers who have achieved or surpassed performance targets. The theory behind this approach is that other departments will try to emulate the budgetary success of the better managers – and that overall productivity will improve. For example, the City of Sunnyvale measures performance to reward successful managers. If a program exceeds its objectives for quality and productivity, its manager can

LINKING YOUR PERFORMANCE MEASURES TO BUDGET PROCESS PERFORMANCE APPRAISALS AND PAY

receive a bonus of up to 10 percent. Another approach might be to increase the manager’s budget so he/she can do “research” or “experiment” with innovative improvements. On the flip side, in some agencies, elected officials are also trying to reduce funding for those who fail to meet targets. Although such links are a potential incentive for improvements, they raise complex management issues and should be handled with caution. When funding reductions are used to punish poor performance, the result may be a reduction in productivity. It may also encourage managers to lower their targets so they can meet them. Not achieving a performance target may signal the need for reduced funding, but it may also signal the need for more funds. There is an added challenge. The relation of cost to the amount of expected workload is relatively clear for many services. However, the relation between dollars (or number of employees) and service outcomes often is unknown. For example, how many additional employees and dollars would enable your agency to increase the customer satisfaction level by five percentage points? In most cases, no one really knows. However, program managers should articulate strategies to bridge the gap between desired and baseline performance, and should use measurement to help test the effectiveness of these strategies. Some other measures of service quality, such as response times to calls for service, are more closely related to amount of staff available. When service information is obtained for a number of years, more of these causal relationships will become better known. Despite this limitation, it is appropriate to discuss, formulate, and justify budgets at least partly on the basis of service outcomes and quality. In preparing and justifying their budgets, program managers should discuss at least qualitatively what they believe the outcome and quality implications will be. A link between budgeting and performance that is both respectful of each program and responsive to the needs requires: • Frequent (monthly or quarterly) performance reports • A careful assessment of barriers to meeting the targets • Thoughtful analysis of the implications of the report for budget allocations A performance budget creates a contract between elected officials and the program managers and their teams (so much service of such a quality for so many dollars).

What affects people’s performance? The activities and duties that make up a job also shape performance. People’s performance is affected by: • • • • • •

how how how how how how

well the agency has defined their roles, responsibilities, and relationships; well the agency has designed the rules, procedures, and processes associated with their job; efficient and appropriate the technology used in the job is; mature and functional the business relationships they must deal with are; clear, accurate, and timely the information they must work with is; and similar and reasonable their customers’ expectations and needs are.

These elements make up the characteristics of the job. Some jobs have well developed, thoroughly documented procedures; others do not. Some jobs put people in long established, mature relationships; others do not. Roles and relationships are well defined for some jobs but less so for others. A change in procedures, relationships, or any of the other elements disrupts the balance. Whether the change is

LINKING YOUR PERFORMANCE MEASURES TO BUDGET PROCESS

PERFORMANCE APPRAISALS AND PAY

looked on positively or negatively, the agency and the individual have to adapt to restore the balance. If they do not adapt, performance suffers.8

Linking to performance appraisals and pay Some agencies are trying to link pay to their performance measurement system. This is difficult to do well. The most common practice is for managers to ensure that performance goals are met by rating individual contributions to performance goals in individual performance appraisals. Others are using incentives by giving employees credits toward training courses. Exceptional performance can also be acknowledged in newsletters and other publications, or with annual awards, plaques, and dinners. A powerful message is sent to employees by judging individual performance by the achievement of the performance goals. It also establishes dynamics that may or may not be wanted. In some cases, increased focus on individual employee performance produces decreased focus on responsibility to the work group and the agency. Intense focus on individual performance encourages competition at the expense of cooperation. A popular multi-source performance appraisal, commonly known as 360-degree feedback, has a real stronghold in the private sector and is growing in the public sector. It is estimated that 90 percent of Fortune 1000 firms use some form of this type of system. In the majority of organizations, this feedback is used developmentally: ratings are collected anonymously and fed back to managers in the aggregate. Usually, only the manager being rated sees the feedback. The ratings are not included in the managers’ formal performance appraisal. Increasingly, however, management is asking, “How do we

hold individuals accountable for making improvements if they are the only ones who see the data? If the individual needs development and chooses to ignore the feedback, we can’t remedy the situation.” This is a reasonable question, particularly when organizations are spending a good deal of time and money on the 360-degree feedback process.

There are valid reasons for limiting the use of 360-degree feedback to developmental purposes and separating it from the formal appraisal process. When individuals believe the ratings will be used for performance appraisals, they may alter their ratings. Game playing may occur – supervisors may try to get higher ratings by catering to subordinates, etc. In some cases, the idea of subordinate or peer ratings as part of one’s appraisal is so taboo that many individuals boycott the process and refuse to participate. Many individuals involved in the 360-degree feedback implementation process strongly discourage its use for evaluation. This also holds true for all types of performance appraisal systems.

USING PERFORMANCE MEASUREMENT INFORMATION TO MANAGE

14 “Dissatisfaction is the basis of progress. When we become satisfied, we become obsolete.” - J. Willard Marriott

To make the cost of implementing performance measurement worthwhile, public agencies need to be sure that the information obtained can help improve public services.

Can performance measurement really help me manage? This is a good question. If you feel the answer is “no,” then performance measurement in your agency is probably a waste of time and money. The answer can be “yes” for the following reason. To run their agencies successfully, public managers must have certain key items of information available. This applies to all levels of management and to all kinds of agencies to be effective. It is important that performance measurement be considered an inherent and indispensable part of the management process. Performance measurement contributes to the following: • Better decision-making: it provides managers with information to perform their management control functions • Accountability: it fosters responsibility on the part of managers • Service delivery: improvements in public service performance • Public participation: clear reporting of performance measures can stimulate the public to take a greater interest in and provide more encouragement for government employees to provide quality services • Improvement of civic dialogue: it helps to make public deliberations about service delivery more factual and specific

Asking the right questions and doing something with the results Many supervisors, managers, directors and elected officials intuitively understand how to ask the right questions about the results produced by the agency’s programs. But as the performance measurement answers arrive in the form of hard numbers and softer survey responses, another set of questions arise: • • • •

What should elected officials and public works leaders do with the information? What changes should be made in programs, policy or management practices in response to the data? How should budgets be modified? Does measuring and reporting performance have any consequences at all?

If the end result is that we do nothing with the information, make no changes, keep the budget the same and there are no consequences, then why are we spending all this time and effort? An excellent manager’s superior interpersonal and leadership skills have much greater potential to foster improvement than does performance measurement. However, the combination of excellent skills and good performance information can be extremely powerful and have a positive impact on the programs and services.

SUGGESTIONS FOR IMPROVING PERFORMANCE INFORMATION USELFULNESS

15 “The best way to get a good idea is to get a lot of ideas.” - Linus Pauling

Enhancing the usefulness of performance data is a true challenge and should be a continuing goal of the performance measurement program. These suggestions should help you focus on this aspect of the work.

Address service quality and outcomes explicitly when reviewing services and programs These discussions among elected officials and the public works director should occur both during the annual budget process and during the year as program and policy issues arise. When elected and executive branch officials begin to discuss specific service outcomes and quality, then management throughout the agency will take performance measurement seriously.

Ask program managers to set a target for each performance measure and assess progress These targets should be set annually based on past experience and based on the budgeted resources anticipated for the upcoming year. Targets also should be established for each reporting period during the year, e.g. quarterly. Actual performance against the targets should be reported, and managers should review with their staff the program’s performance after each report has been issued. Upper-level managers and executives should also review and discuss the actual performance results. Their focus should be on needed improvements and be constructive.

Include indicators of both “intermediate” outcomes and “end” outcomes Few outcomes that are truly end-oriented are fully under the control of the public manager. And many months, if not years, may elapse before the results show up. Intermediate outcomes, the ones most commonly reported by programs, directly relate to customers but do not indicate end results. For example, environmental programs encouraging households or businesses to reduce hazardous wastes and to dispose of them properly should track such intermediate outcomes as the number of households and businesses that report changing their behavior in favorable ways. The water quality and the condition of living resources in the water should be measured as end outcomes, but these program effects likely will take longer to show up (after household and business behavioral changes.)

SUGGESTIONS FOR IMPROVING PERFORMANCE INFORMATION USELFULNESS

Include performance measurement in your training programs Agency training programs should include information on the performance measurement process and on ways to use the information, such as for allocating resources, performance contracting, formulating and justifying budgets, helping with budget justifications, and making performance appraisals. In the early stages of performance measurement one-on-one consultation helps managers develop useful measures that are feasible to collect.

Incorporate outcome performance requirements into contracts wherever feasible In writing contracts, pay more attention to what is to be accomplished and less to how the contractors are to accomplish it. Requests for proposals and even, in some cases, requests for bids should specify the results expected in quantitative terms to the extent possible. To make such contract stipulations meaningful, the agency needs to monitor performance carefully against these requirements. Local governments have included in contracts such measures as the number and timeliness of correction of complaints in solid waste collection contracts; number of road breakdowns and percent of scheduled times met in transit contracts; and percent of vehicles returned in automotive repair contracts. Some of the above contracts included specific dollar rewards or penalties for not meeting or not exceeding targeted values. The existence of an ongoing performance measurement system should allow agency managers to monitor more easily and enforce the provisions of these contracts.9

IS THERE A BOTTOM LINE?

“Imagination is more important than knowledge.” - Albert Einstein

What is measured and reported gets attention! The statement in this section’s heading (what is measured and reported gets attention) is true, especially if the information is used to formulate and justify budgets, for performance appraisals, and gets reported to the elected officials and media. Misuse and counterproductive use can occur if the performance measures are not chosen and implemented with care. For example, solely focusing on the time it takes to provide a service is likely to cause employees to push for speed of service delivery at the expense of the quality of the service. To reduce such problems, take these two steps: First, when selecting performance measures, explicitly consider potential negative side effects and include measures that track them. Response time measurements should be balanced by measures of the quality of the service, such as by surveying all, or a random sample, of the service’s customers. Second, to protect against counterproductive behavior, periodically query staff as to whether they feel they are being pushed by the measures to undesirable behavior, such as rushing through a service at the expense of service quality. If so, revise the set of performance measures to provide a better balance between desirable and undesirable effects.

Measuring program outcomes and quality is not easy At best, the development of a performance measurement system is a multi-year effort. For many of the services we provide, new data gathering techniques will be required, such as customer surveys and trained observers. All of these techniques require staff time, resources, and technology to make them happen. To make the cost of implementing performance measurement on a continuing basis worthwhile, we must ensure that the information can help improve public services. Our managers are key users of this information and, perhaps, play the most crucial role in making performance measurement successful. When managers reap real benefits, their commitment to the sometimes formidable task of capturing and reporting performance data increases. 6

Is there an answer? It is dangerous to offer generalized solutions for a wide variety of agencies, situations, and needs. But performance measurement requires a certain level of boldness. There is one strategy that will surely work no matter which “big picture” approach you are using.

IS THERE A BOTTOM LINE?

Suggested Strategy Develop outcome measures for each of your programs. The results will fit into any of the higher level approaches. The ultimate performance measurement system is illusive. Many public works professionals, including you, are in search of some answers. You are in good company. Few, if any, public works agencies have fully developed outcomes for each program and have linked them to their business plans, work plans, and budgets. A method by which work groups can create and continually modify performance measurement systems suited to their own inevitably special needs and circumstances needs to be created. No matter which approach you take, what won’t work is a standard set of measurements created by experts or obtained from a “shopping list” that are imposed on the agency. The shopping list approach may help you get started down the path, but it won’t get you to your final destination. Performance measurement has no purpose if the data are not used to improve performance. The bottom line is that sooner or later you are going to be required to measure performance. Are you going to be proactive and design a useful, good system or be reactive and play along with whatever is dictated? The choice is yours.

APPENDIX - SOURCES

Joni Leithe, Implementing Performance Measurement in Government, Government Finance Officer Association, 1997. 1

George Labovitz and Victor Rosansky, The Power of Alignment, 1997.

Governmental Accounting Standards Board, Concepts Statement No. 2, Service Efforts and

Accomplishments Reporting. Wray and Hauer, “Best Practices Reviews for Local Government: Identifying and Sharing Better Ideas on Public Services”, Public Management, January 1996. 4

Bland and Rubin, “Budgeting: A Guide for Local Governments,” ICMA, 1997.

Harry P. Hatry, Performance Measurement: Getting Results, 1999.

Lin Grensing-Pophal, “Follow Me,” HR Magazine, February 2000.

Judith Hale, The Performance Consultants Fieldbook: Tools and Techniques for Improving Organizations and People, 1998. 8

Hatry, Gerhart, Marshall, Eleven Ways to Make Performance Measurement More Useful to Public Managers, September 1994. 9

Karen Carney, Successful Performance Measurement: A Checklist, 1999.

Bunker, Alban, Large Group Interventions: Energizing the Whole System for Rapid Change, 1997.

BEST PRACTICES IN PERFORMANCE MEASUREMENT MODEL

- APPENDIX

PERFORMANCE MEASUREMENT PROCESS MODEL Customers and Stakeholders Input

Performance Reporting to Customers and Stakeholders

Management Priorities and Decisions Congressional Priorities and Decisions

Customer Driven Strategic Planning Multi-year Goal Setting and Resource Planning Annual Performance Planning Resource Allocation • Mission is clear and energizes employees • Strategic goals and objectives have focus and are stretching • Owners are identified for goals and objectives • Strategies are developed and resources allocated • Customer needs are addressed • Outputs and outcomes are defined (logic models or other tools are used) • Decision issues and decision processes are used

Establishing and Updating Performance Measures and Goals

Establishing Accountability for Performance

Measuring Performance (Data Collection and Reporting)

• Management culture is supportive • Measures flow from goals and objectives and are developed by managers working with - multi-disciplined teams - focus groups and - stakeholders • Inventory of common measures is explored • Balanced scorecard or similar tools are used • Measures cascade and align through the organization • Performance levels are reflective of resources

• Ownership of each measure is formalized and resources provided • Responsibilities for data collection, reporting, analysis and posting are identified • Managers use measures to evaluate performance • Reward systems are clear and consistent, and are reflective of level of success

• Data sources are identified • Information systems are designed to support data collection and reporting • Pilot tests are conducted • Automated or Manual requests are used for periodic updates • Data entry, tabulation, summarization methods are documented for each measure • Data definition for common measures are followed • Reliability, timeliness, accuracy, rapid access and confidentiality are addressed

Analyzing and Reviewing Performance Data

Evaluating and Utilizing Performance Information

• Data are integrated • Analytical capabilities are developed • Results are analyzed and validated • Management reviews results vs. expectations and makes mid course corrections • Feedback is provided to activity/process owners for continuous improvement

• Activity/process owners use performance information for continuous improvement • Results are displayed and shared with customers and stakeholders • Rewards and recognition are based on results • Benchmarking and comparative analysis with best in class are done • Management feedback is provided for updating goals and measures • Performance information is used to identify opportunities for re-engineering and allocation of resources

APPENDIX - MISSION, VISION, VALUES, GOALS

Mission, Vision, Values One-Day Workshop Agenda 8:00

Welcome Why are we here?

8:10

Background & Beginnings Agenda review, what’s happened so far, expectations, ground rules, warm-up

8:30

Environmental scan What’s going on in the community and agency that impacts the work and atmosphere of agency staff? How much change is going on?

Large group brainstorm Outcome: Description of the current situation

Changes, Transitions, Beginnings and Endings Presentation and discussion of the change process. Foundation materials from: Managing Transitions – Making the Most of Change, William Bridges