Outputs and Outcomes

Outputs and Outcomes JULY 2016 A research note prepared by Charles Sedgwick for the Tertiary Education Union Te Hautū Kahurangi o Aotearoa

1

TERTIARY EDUCATION UNION Te Hautū Kahurangi o Aotearoa

The drive to ensure accountability for taxpayer dollars now means that identifiable outcomes feature in most government policy, including for the tertiary education sector. For some, the outcomes of tertiary education are the major aims set out in the Education Act (a view held by the Tertiary Education Union). However, within much government policy and documentation the term ‘outcomes’ is frequently conflated with outputs. These tend to be narrower in nature, more easily measurable and often more economically focused than outcomes.

This paper seeks to explore the concepts of outcomes and outputs; and to consider how narrowing the focus of tertiary education to easily identifiable and measurable outputs has undermined the much broader, collective goals of tertiary education.

the health system. Thus a quantifiable output such as surgical operations becomes the desired outcome of the system, rather than the actual desired outcome of having a health system that contributes to better health across society. It is easy to see this situation transposed into educational institutions.

Outputs targets are easily set – literacy and numeracy attainments; course completions; the quantity of peer reviewed research – and are mistakenly seen as the system’s desired outcomes, but they are a far cry from the broad educational, social, and humanistic goals of education.

In New Zealand the Education Act (s 159AAA) makes clear what the ‘results for citizens’ should be from tertiary education. This includes contributing to “The development of cultural and intellectual life in New Zealand … the sustainable economic and social development of the nation” and the attainment of “social and environmental goals”. Furthermore, universities should develop intellectual independence and be the critic and conscience of society, polytechnics should provide a diverse range of vocational learning, and wānanga should advance and disseminate knowledge with regard to āhuatanga Māori according to tikanga Māori.

Focusing on outcomes is about better understanding what the expenditure of taxpayer dollars has achieved. According to Cook (2004: 2) the aims of the public management system are that they must be accountable to the people; that services delivered “should be in line with achieving the objectives of the Government of the day’; and in addition to accountability, the system should be sufficiently flexible to allow innovation in the way agencies seek to deliver specified services.” But Cook does warn (2004: 17) that this focus on accountability can go astray: “In looking towards a ‘managing for outcomes environment’ it is clear that the public service will need to have a culture that is also concerned about achieving results for citizens rather than simply achieving specified output targets.”

The Productivity Commission acknowledges a number of these elements in its work, however the sector’s responsibility to support broader collective outcomes appear to be lost as the focus turns to narrow outputs. This is not surprising given the trajectory of tertiary education in New Zealand, with central government having a much tighter control of the sector.

Similarly Easton (2016) observes the tendency to redefine outputs as outcomes in the public sector. Drawing on the work of Gorman and Horn (2015) he notes how this has occurred in

How did this situation come to pass in the New Zealand tertiary system? What are the outcomes that are important for both the immediate future and for lifelong learning?

2

NARROWING THE OUTCOMES OF TERTIARY EDUCATION In terms of outcomes for the sector, the Productivity Commission gives primacy to “impart[ing] knowledge and skills which are valued by employers, society at large and by students.” This is followed by other views of the system’s purpose which include: “the production of basic research, enhancement of cultural institutions and being critic and conscience of society” (Sherwin, 2016).

what is not contested in their opinion is the need to evaluate whether you are ‘receiving good value’ as an output, or whether the focus is on achieving broader and longerterm outcomes that may benefit society collectively.

When we talk of outcomes we need to specify who we are referring to as the recipient. If we refer to the Act then we are looking at very general recipients: the “cultural and intellectual life of New Zealand ... a skilled and knowledgeable population … economic and social development … innovation … international competitiveness … attainment of social and environmental goals.”

This is a far cry from the way outcomes are set out in the Education Act. The Act clearly presents the recipients of the outcomes of tertiary education as the whole nation, not some section or particular interest group, and while the term ‘wellbeing’ was not part of the rhetoric in 1989 (it appears around 2000) one gets the sense that this is a reasonable synonym for what was proposed.

Are we then to conclude from the above that the ‘sharply contrasting consequences’ of two views of the university in New Zealand that Sir Kenneth Keith (2000: 255) drew attention to has been resolved?

Secondly the outcomes in the Education Act are all of equal value in the Act, not presented as a hierarchy of possibilities with the diminutive add-on of ‘other views’. The Act says nothing of the need to have trade-offs (that is ‘the pursuit of some goals is likely to come at the expense of others’). This becomes unnecessary when you have a multiplicity of possible outcomes. Nor does the Act qualify any outcome with ‘should’ or ‘can’ or ‘might’.

The top-down [view] may, as events elsewhere in the world show, lead to the professoriate being seen as public servants, with resulting disabling restraints on the critical, independent, spirit and role of the members of the university. By contrast the bottom up view might lead to a refusal by the institution to acknowledge any public responsibility, including its responsibility to potential and present members of the institution.

From the Productivity Commission’s perspective the Act quite accurately presents a system with multiple purposes; however the context in which tertiary education operates today means this is often interpreted as having to make trade-offs. The Commission does point to the possibility that some measurement means are problematic but

The resolution has become a causal process; the former has created the latter.

WHAT HAPPENED TO THE ORIGINAL OUTCOMES? Where did it go awry? According to Elizabeth Eppel (2008: 95), we get a glimpse of a shift to more centralised government control under National in the 1990s. The Government achieved this by more highly

regulating the funding system. According to one funding and monitoring adviser (Eppel 2008: 93) consulted: “the funding system was perverting the academic process by incentivising people the wrong way, to dream

3

contributing to achieving the Māori development aspirations; raising foundation skills to allow participation in the knowledge society; developing the skills needed for a knowledge society; educating for Pacific people’s development and success; and strengthen the research knowledge creation and uptake function.

up these new post-grad qualifications ... the management of the university were telling their departments to come up with new ideas to bring in students at that level.”

Another complicating factor in this process emanated from the Public Finance Act 2004 concerning service performance and other non-financial reporting. This new approach meant that government departments had to report on the delivery of outputs (service performance) and provide information on the relationship between these outputs and the government’s desired outcomes. The latter are what the minister designates; we could assume that those outcomes would be as near as possible to those set by statute, but if they represent prioritised outcomes of government then they might be quite different (for example ideologically rather than evidentially driven).

If we contrast this with the most recent Tertiary Education Strategy (2014-19) we note there are six sub-categories: Delivering skills for industry; getting at risk young people into a career; boosting achievement of Māori and Pasifika; improving adult literacy and numeracy; strengthening research-based institutions; growing international linkages (Tertiary Education Strategy 2014-19: 1).

The tenor and direction of the former sounds much more like outcomes that would fit within the parameters of the Act, while the more recent categories are far more instrumental outputs. In a recent press release (2016) the Minister, responding to statistics that showed increasing number of graduates in STEM subjects, said:

Under Maharey in the Labour Government a group of eight advisors described as ‘strategic thinkers selected for their vision, expertise and credibility’ (TEC 2000: 34) were brought together to work on the future of tertiary education. One advisor was quoted as saying: “I still think ... that Nation Building is still one of the best kind of markers of what it was all about, and what is it still about - that harnessing tertiary education to what we are now calling economic transformation” [Ministerial adviser] (Eppel 2009: 102). While reflecting a change in government, this also has continuity with Bradford’s claims in 1999.

It’s important that we grow the number of graduates in areas where there is real industry demand ….

Producing more engineering, ICT and science graduates will help alleviate actual and potential skill shortages in these areas.

The breadth of this whole period of transition was captured in the 2008 OECD report which noted that tertiary education review and policy in New Zealand had closely followed economic policy as it moved out of regulation and protection to a liberalised free market state (OECD 2008: 16). However if one looks at the tertiary strategy document noted above, we find according to the OECD report (Goedegebuure, et al 2008: 17):

The Minister therefore specifies certain limited outputs and the beneficiaries of these.

The ‘outcomes’ are now produced for the ‘most powerful customer’ – the Government (Sherwin 2016), which is also the purchaser of services, contributes the bulk of funds to the system (tax payers money) and by virtue of its power has the capacity to use both funding and rules to ensure the required outcomes.

Six “sub-strategies” comprise the current TES: strengthening of the system capability and quality;

4

INTERNAL CYCLES OF OUTCOME CHANGES Performance management is both part of the quality assurance regime for the consumer and a way the institution proves its compliance to the funder/consumer.

Performance management can be measured by outputs and these outputs now become the new outcomes. The Productivity Commission quite accurately describes this cycle in operation and concludes two things:

Tertiary education institutions receive directives about outcomes/outputs required by the prime consumer – the Minister.

Assuming they are not going to exit the education market, they have two choices: voice or loyalty. Voice, which in this sense means opposition, is defined by the ‘customer’ as ‘intransigence’, ‘inertia’ or ‘sheer perversity’ (Sherwin, 2016), and the consumer reacts by imposing regulation, funding controls or contractual conditions. Then institutions must react to the new ‘inputs’ by managing their system – usually by performance management directed towards outputs via the variables under their control.

a. that this system produces a risk averse consumer who is constantly overwhelmed by wanting value for ‘their’ investment and a risk averse university sector confined by contract, funding and regulations lest thing get worse.

Thus institutions react by wanting to reduce the risk of further regulatory, funding or contractual pressure and their managers are instructed to act accordingly. Post 2004 in New Zealand this has become general government policy – managing for outcomes.

b. that we must look elsewhere for ways to improve the outcomes – the national productivity as they have defined it

WHAT HAS BEEN FORGOTTEN? catering for over half a million predominantly domestic students” (OECD 2008: 15).

We have lost track of outcomes in the multiple sense the Education Act presents them. This has been replaced with the production of an employable person with skills and knowledge for the immediate-term. The benefit of the outcomes outlined in the Act is that they give consideration to the future of society, tertiary education institutions, staff, students, and the community. As Hawke (2008) points out, skill requirements are constantly changing. This will be always the case, so you are not just learning skills for life but rather for lifelong learning – we must be capable of developing skills and enhancing them as necessary.

To talk of outcomes in this context seems problematic and prone to driving any definition of broad collective outcomes to find an all-inclusive model which allows easy measurement of outputs. The obvious result is the narrowing of the sector’s focus and turning of multiple outcomes into measureable outputs – the GDP of a tertiary education sector.

The other implication of this shift is to use these re-interpreted outputs as institutional management guidelines which are then required outcomes to be implemented by management in each institution. Outcomes become a means of measuring both managerial competence and institutional outputs, and if managers and institutions want to survive, they comply. The public-good

A further difficulty is the very broad church of tertiary education institutions: “The present tertiary education sector with well over 900 institutions is a diverse amalgam of small and large, comprehensive and specialised, and private and public providers

5

clear, however, that replacing outcomes with outputs will not suffice as a response.

outcome is now converted to a private output, steering behaviour and institutions. This is not dissimilar to the means of convincing students that education as a public good is really a private good and you do this by charging them for the experience. This occurs much in the same way that PBRF, as a quality assurance mechanism for the system, is transformed into a private assurance/ success measure for each participating academic. How does one destroy collegiality, cooperative research, and publication or peer group collective learning for students?

Unfortunately very easily.

One is drawn back to Sir Kenneth Keith’s (2000, p.256) assertion that an education system has a public responsibility, which he describes as “the basic New Zealand objective, stated in 1938”:

[T]hat every person has a right as a citizen to a free education of the kind for which they are best fitted and to the fullest extent of their powers. While that objective was stated for primary and secondary schooling, it is capable of application in principle to tertiary education as well. My impression is that the top-down model too often, too heavily, influences public debate.

The Productivity Commission’s aim, as Sherwin (2016) states it, is: ‘to concentrate on educational outcomes generated by the system. This will require us to consider what a good tertiary education system looks like, and how that quality might be assessed’. It is

6

References Cook, A.L. 2004 “Managing for Outcomes”.

In The New Zealand Public Management System Working Paper 04/15, Wellington, New Zealand Treasury

Grey, S, and Scott, J. 2012. “When the government steers the market: implications for the New Zealand’s tertiary education system”. A working paper for NTEU’s Future of Higher Education Conference, 2012 University of Sydney.

Earle, D.2008. “University objectives; an analysis of university annual reports 20022006”. Wellington, Ministry of Education

Keith, Sir Kenneth. 2000. “Why is Academic Freedom Important for New Zealand?” in Crozier, R. (Ed) Troubled Times: Academic Freedom in New Zealand, Palmerston North, Dunmore Press

Easton, B. 2016. “Outputs or Outcomes; the Difference Matters”. In Briefing Papers, 22 February http://briefingpapers.co.nz/2016/02/ outputs-or-outcomes-the-difference-matters/

Sherwin, M. 2016. “What can a five-year old Productivity Commission add to a thousand year-old institution?” UC Connect Lecture, 6 April 2016,Canterbury University http://productivity.govt.nz/news/tertiaryeducation-speech-canterbury

Eppel, E. A. 2009, “The contribution of complexity theory to understanding and explaining policy processes. A study of tertiary education policy processes in New Zealand”. PhD thesis, Victoria University of Wellington.

MBIE and MoE. 2014. “Tertiary Education Strategy, 2014-2019” MoE and MBIE Wellington.

Goedegebuure, L, Santiago, P, Fitznor, L, Stensaker, B, and van der Steen, M. 2008.

“OECD Reviews of Tertiary Education: New Zealand”. http://www.oecd.org/ newzealand/38012419.pdf Gorman, D. and M. Horn, 2015 “Purchasing better, innovative and integrated health services.” Internal Medicine Journal 45 (2015).

7