Epnosl Evaluation Report

European Policy Network On School Leadership (EAC/42/2010) Grant Agreement EAC-2010-1388 Specific Agreement number: EAC-2013-0536

EPNoSL Final Evaluation Report

External Evaluation Effectiveness and Efficiency of Work Processes in relation to the Call’s objectives Del 1.2 Date: 30-06-2015

With the support of the Lifelong Learning Programme of the European Union

1

Work Package:

No. 1

Authors:

Jacky Lumby and Gerry Macruairc

Status, Version No.

Final

Submission date:

11/07/2015

Start Date of the Agreement:

11.2. 2014

Duration of the Specific Agreement

18 months

Dissemination Level:

Public

Project Coordinator:

Kathy Kikis-Papadakis, FORTH/IACM katerina@iacm.forth.gr With the support of the Lifelong Learning Programme of the European Union

Financing:

This project has been funded with support from the European Commission.

This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use that may be made of the information contained therein.

2

Table of Contents 1. INTRODUCTION ......................................................................................................................... 6 1.1 GENERAL CONTEXT ......................................................................................................... 6 1.2 SPECIFIC CONTEXT FOR THE EXTERNAL EVALUATION REPORT ..................................................... 6 1.3 APPROACH TO EVALUATION .............................................................................................. 6 2. SECTION ONE: EFFECTIVENESS AND EFFICIENCY OF WORK PROCESSES........................................ 9 2.1 APPROACH TO EVALUATION .............................................................................................. 9 3. QUESTIONNAIRE RESULTS: PARTICPANTS' EXPERIENCE OF THE PROGRAMME .............. 10 3.1 PROFILE OF RESPONDENTS .............................................................................................. 10 3.2 ENGAGEMENT WITH EPNOSL ACTIVITIES ........................................................................... 10 3.3. WORKING AS A NETWORK ............................................................................................. 12 3.3.1 COMMUNICATION ...................................................................................................... 12 3.3.2 PROBLEMS OR OBSTACLES ENCOUNTERED ........................................................................ 14 3.3.3 RESPONSES TO PROBLEMS OR OBSTACLES......................................................................... 15 3.3.4 OVERALL SATISFACTION WITH PROJECT MANAGEMENT ........................................................ 16 4. QUESTIONNAIRE RESULTS: SKILLS, KNOWLEDGE OR ATTITUDES ACQUIRED .................. 18 4.1 OUTCOMES OF THEMATIC AREAS EXPLORED DURING EPNOSL EVENTS ..................................... 18 4.2 IMPACT OF NATIONAL EVENTS ......................................................................................... 19 4.3 IMPACT OF EPNOSL ONLINE ........................................................................................... 21 4.4 EFFICACY OF LEARNING ACTIVITIES AND DIVERSE PROFESSIONAL BACKGROUNDS OF PARTICIPANTS ON LEARNING OUTCOMES......................................................................................................... 22 5. QUESTIONNAIRE RESULTS: CHANGES IN JOB BEHAVIOUR............................................. 24 5.1 CHANGES IN IMPORTANCE ACCORDED TO SCHOOL LEADERSHIP DEVELOPMENT............................ 24 6. QUESTIONNAIRE RESULTS: IMPACT OF CHANGED BEHAVIOURS ................................... 27 6.1 PRIORITY CHANGES IN ORGANISATIONS ............................................................................. 27 6.2 AREAS OF IMPACT AND INFLUENCE.................................................................................... 28 6.4 PLANNED ACTIVITIES ..................................................................................................... 30 7. CONCLUDING COMMENTS: SECTION ONE .................................................................... 33 8. SECTION TWO: DOCUMENT REVIEWS............................................................................35 8.1 APPROACH TO EVALUATION ............................................................................................ 35 8.1.1 SAMPLING................................................................................................................ 35 8.1.2 ANALYSIS OF INDIVIDUAL OUTPUTS ................................................................................ 38 8.1.3 ANALYSIS OF OVERALL COHESION AND CUMULATIVE EFFECT .................................................. 38 9. DOCUMENT ANALYSIS RESULTS .................................................................................. 39 10. IMPACT OF OUTPUTS ON POLICY ............................................................................... 66 10.1 OVERALL IMPACT OF OUTPUTS ....................................................................................... 66 10.2 OVERALL SEQUENCE OF OUTPUTS .................................................................................... 66 10.2.1 STAGE 1 ................................................................................................................ 67 10.2.2 STAGE II ................................................................................................................. 67 10.2.3 STAGE III ................................................................................................................ 68 11. CONCLUDING COMMENTS: SECTION TWO ................................................................. 70 3

12. SECTION THREE: THE IMPACT OF EPNOSL'S WORK ON POLICY RELATING TO SCHOOL LEADERSHIP ................................................................................................................................ 71 12.1 APPROACH TO EVALUATION........................................................................................... 71 13. EPNOSL CONCEPT OF POLICY ..................................................................................... 72 14. THE IMPORTANCE OF CONTEXT ................................................................................. 74 15. LEVELS OF IMPACT .................................................................................................... 77 16. FORMS OF IMPACT ................................................................................................... 79 16.1 VARIETY OF IMPACT..................................................................................................... 79 16.2 UNDERPINNING VALUES ............................................................................................... 79 16.3 THE EDUCATION/TRAINING OF SCHOOL LEADERS ................................................................. 80 16.4 APPROACHES TO LEADERSHIP ......................................................................................... 81 16.5 COLLABORATIVE APPROACHES TO POLICY DEVELOPMENT AT A NUMBER OF LEVELS ...................... 81 16.6 EMPOWERING THE POLICY DEVELOPMENT COMMUNITY........................................................ 81 16.7 USING THE EPNOSL TOOLKITS ....................................................................................... 82 17. CASE EXAMPLES........................................................................................................ 84 17. 1 CASE 1: THE IMPACT OF EPNOSL ON POLICY: ESTONIA AS A CASE EXAMPLE.............................. 84 17.2 CASE 2: THE IMPACT OF EPNOSL ON POLICY: FINLAND AS A CASE EXAMPLE .............................. 86 17.3 CASE 3: THE IMPACT OF EPNOSL ON POLICY: GERMANY AS A CASE EXAMPLE ............................ 88 17.4 CASE 4: THE IMPACT OF EPNOSL ON POLICY: ITALY AS A CASE EXAMPLE ................................... 89 17.5 CASE 5: THE IMPACT OF EPNOSL ON POLICY: PORTUGAL AS A CASE EXAMPLE ........................... 91 17.6 CASE 6: THE IMPACT OF EPNOSL ON POLICY: SCOTLAND AS A CASE EXAMPLE ............................ 95 18. CONCLUDING COMMMENTS: SECTION THREE............................................................ 97 19. EPNOSL EVALUATION: OVERALL CONCLUSION ....................................................................... 99 APPENDIX 1: LISTING OF EPNOSL DOCUMENTS .......................................................................... 103 APPENDIX 2: SAMPLE OF EVALUATED DOCUMENTS ................................................................... 107

4

Tables and Figures Table 3.1 Engagement of respondents with EPNoSL activities ............................................... 11 Table 3.2 Experience of communication ................................................................................. 13 Table 3.3 Impact of problems obstacles on achievement of planned objectives ................... 14 Table 4.1 Degree of development in knowledge and understanding of themes: PLA participation ............................................................................................................................ 19 Table 4.2 Degree of development in knowledge and understanding: National events.......... 20 Table 5.1 Respondents’ views of the development of their approach to training ................. 24 Table 6.1 EPNoSL: Areas of impact .......................................................................................... 29 Table 6.2 Timescale for proposed activities ............................................................................ 31 Table 8.1 Document sample by category and country of focus……………………………………………36 Table 8.2 Document analysis framework…………………………………………………………………………….37

Figure 3.1 Profile of survey questionnaire respondents ......................................................... 10 Figure 3.2 Adjustments made to original plans to achieve planned objectives...................... 16 Figure 4.1 Development in knowledge and understanding in key EPNoSL thematic areas.... 19 Figure 4.2 Number in attendance at national events ............................................................. 20 Figure 4.3: Outcome of online platform on knowledge and or understanding ...................... 21 Figure 4.4 Impact of different event activities on learning outcomes .................................... 22 Figure 4.5 Impact of different partner professional roles on learning outcomes................... 23 Figure 5.1 Changes in practice................................................................................................. 25 Figure 6.1 Intended outcomes or impact ................................................................................ 31

5

1. INTRODUCTION 1.1 General Context The European Commission call EAC/42/2010 was intended to establish a European Network of relevant individuals and organisations to implement priorities for school leadership development. These had been identified by the European Council in the context of the Lifelong Learning Programme. Specified objectives were to be achieved over a period of four years from 2011 to 2015. During this period, three successive Work Plans (known as Specific Agreements) focused on cumulatively building work to achieve the specified objectives. This work was undertaken by the European Policy Network on School Leadership (EPNoSL).

1.2 Specific Context for the External Evaluation Report Monitoring, reflection, internal and external evaluation are intrinsic elements of all three Work Plans. As the project has now concluded, a final evaluation is able to build on previous evaluative reports to consider more comprehensively the work of the project and its impact.

During the Third Period the contract required evaluation by external evaluators of: a. Impact on stakeholder groups (focusing on policy) b. Effectiveness and efficiency of work processes (in relation to the stated milestone MS3 – Quality Assurance of EPNoSL processes and outputs in relation to the Call’s objectives) c. Relevance of EPNoSL contents to support policy development.

We have interpreted these requirements in the Work Plan as evaluation of: 1. The effectiveness and efficiency of the way partners have worked together 2. The relevance and value of the intellectual activities, the issues explored and products of EPNoSL's work for shaping policy and practice 3. The impact of EPNoSL's work on policy relating to school leadership.

A report on each of these areas was completed over the course of 2014/2015. This final report provides a fourth and final comprehensive external evaluation of the work of the European Policy Network on School Leadership by providing a composite report of all three interim reports.

1.3 Approach to Evaluation Evaluation might be viewed as an account of whether the measurable outputs for which EPNoSL is contracted were achieved; that is, the efficiency of the project. While outputs or efficiency can be verified, there are considerable challenges in evaluating achievement of the central aim of the call, the impact of outputs; that is, the outcomes. In other words, the effectiveness of EPNoSL is more difficult to assess than its efficiency. Impact on both policy and practice is a long-term and often indirect effect that may not be easily discernible, or discernible at all in the short timescale available in the final phase of the project.

6

Bearing these challenges in mind, in order to provide a framework for approaching the collection and analysis of relevant data Nutley et al.'s (2007)1 model of the impact of social science research was adopted as a guiding framework, scrutinising evidence relating to effects that is:   

Instrumental (for example, influencing the development of policy, practice or service provision, shaping legislation, altering behaviour) Conceptual (for example, contributing to the understanding of these and related issues, reframing debates) Capacity-building (for example, through technical/personal skill development) (ESRC, 2009: 4).2

The approach is designed to illuminate the impact of EPNoSL's work from a number of perspectives. Changes in policy and practice that have resulted to date will be assessed although, in the light of the limitations given above, changes may proceed on a longer timescale than the project’s lifetime. Consequently, evaluation will also consider those foundational changes in understanding and capacity that may ultimately lead to further change in policy and practice.

1

Nutley, S.M., Walter, I. & Davies, H. (2007) Using Evidence: How research can inform public services.

Bristol: Policy Press.

2 ESRC (2009) Taking Stock: A summary of ESRC’s work to evaluate the impact of research on policy & practice. Accessed online 18.3.11 at http://www.esrc.ac.uk/_images/Taking%20Stock_tcm8-4545.pdf)

7

2. SECTION ONE: EFFECTIVENESS AND EFFICIENCY OF WORK PROCESSES 2.1 Approach to Evaluation The first evaluation report assessing the effectiveness and efficiency of work processes used a questionnaire survey to collect responses from Network members. Its design drew on Kirkpatrick's (1987)3 framework to shape feedback on how people have worked together and to what end. Kirkpatrick offers a generic framework for assessing the impact of development activities on individuals and organisations. It suggests four levels of impact:

Level I

Reaction

How well did the participants like the programme?

Level 2

Learning

What were the skills, knowledge or attitudes that were learned?

Level 3

Behaviour

What were the changes in job behaviour resulting from the training programme?

Level 4

Results

What is the impact of the changed behaviours on organisational effectiveness?

The survey structure embedded the four levels: requesting perceptions on general satisfaction with how the Network has been managed and developed; the skills, knowledge and understanding acquired; new thinking or actions resulting from these; and, finally, the impact of these, if any, on policy and practice.

Using this framework, the first section of the questionnaire collected profile information about respondents and asked about how they experienced the project. The second section focused on key learning outcomes, that is, the extent to which Network participants' knowledge and/or understanding was challenged, developed or changed. The third section asked for indications of any changes in job behaviour resulting from involvement in EPNoSL. The fourth and final section asked respondents to indicate the impact of EPNoSL on the policy and/or practice of school leadership in a professional work context, that is, the effect of the changed job behaviour. The survey was consequently shaped to enable assessment of capacity building and the results of such capacity building to date in terms of impact on policy and organisations, as perceived by Network members.

Seventy-five Network members were invited to participate in the survey, including representatives of 42 partners from 21 countries. Thirty-three responded, a response rate of 44%. Consequently, results must be seen as indicative rather than definitive in analysing the experience of just under half of the Network’s members.

3

Kirkpatrick, D.L. (1987) Evaluation of training, in R.L. Craig (ed.), Training and Development Handbook: A guide to human resource development. New York: McGraw-Hill.

9

3. QUESTIONNAIRE RESULTS: PARTICPANTS' EXPERIENCE OF THE PROGRAMME 3.1 Profile of Respondents The first section of the questionnaire gathered brief information about each respondent's organisation. Twenty-six respondents indicated the nature of the organisation that they are employed by or represent. The profile of respondents reflected that of the Network as a whole: a high proportion worked in higher education and also included practitioners, education ministry staff, employees' representatives and those engaged with developing school leaders, as shown in Figure 3.1.

Category of respondent's institution

Education Ministry Regional administration of education Employers' representative Employees' representative Higher education Leadership development organisation, not in higher education School leader

Figure 3.1 Profile of survey questionnaire respondents Five respondents indicated that they did not belong to any of the categories: two were from capacity building/research institutes, one from an administrative agency that implemented policy, one from a national statutory body relating to teaching, and one was a representative of stakeholders.

3.2 Engagement with EPNoSL Activities Asked about the nature of their engagement with EPNoSL, 25 members responded that they had participated in and so were familiar with a wide range of the Network's activities.

10

Table 3.1 Engagement of respondents with EPNoSL activities Nature of activity

% of respondents

Attended PLAs

96

Brought EPNoSL materials to the attention of my organisation

76

Attended management meetings

72

Managed an element of EPNoSL activity

64

Undertook research on leader development

64

Brought EPNoSL materials to the attention of the Education Ministry

64

Authored academic papers or research reports

56

Delivered keynotes/presentations

52

Brought EPNoSL materials to the attention of regional administration

44

Held EPNoSL national Networking event

44

Held national EPNoSL workshop

44

Produced learning materials

36

Held EPNoSL regional Networking event

28

Supported technical aspects of activity such as online activity

24

Organised a PLA

24

Produced standards or indicators

24

Held regional EPNoSL workshop

20

Nearly all respondents had attended peer learning activities, and high percentages also engaged with a range of other activities. Notably, just over three-quarters had brought EPNoSL materials to the attention of their organisation, two-thirds to the attention of the relevant Education Ministry and just under half to the attention of regional administration.

Nearly half had held national workshops. Overall, Table 2.1 suggests that responding Network members not only engaged in producing outputs such as research, papers and learning materials but were active in translating these into outcomes, that is, connecting policymakers and practitioners to the outputs through peer learning activities, regional and national events.

Comments indicated further planned activity, including activity to promote impact which fell outside the categories offered by the question. For example: While, to date, separate regional/national events have not been organised, we have fed EPNoSL information into existing national events.

In another case: Work sessions under the format of workshops with school practitioners at their demand in their own contexts, responding to their questions and concerns on SL issues where there were 451 participants involved along seven field interventions.

11

National Networking events were planned to take place in the future in Berlin as a joint initiative of NLQ Hildesheim and LISUM, aiming for over a hundred participants. Nordic countries were developing collaborations to produce impact tools. One respondent noted the low number of practitioners involved in discrete EPNoSL events, such as peer learning activities, but other respondents’ comments suggested that Network members carried knowledge, skills and understanding and their relevance to policy outwards from such events to regional and national levels, rather as concentric circles generate ripples.

Comments suggest that there was potential to reach large numbers of practitioners in other ways in each country or region and that in some cases this has been implemented.

In summary, respondents to the questionnaire represent a range of individuals and organisations that have sufficient experience of both EPNoSL activities and their spin-offs to provide valid information about the working of the Network and its outputs and outcomes.

3.3 Working as a Network The questionnaire explored respondents' experience of working as a Network, what had proved helpful or the contrary, and how members had made adjustments where necessary to implement the Work Plans. The focus was on the efficiency of the process.

3.3.1 Communication The questionnaire asked about the timeliness of communication from the project management during the project on a six-point scale ranging from ‘never timely’ to ‘always timely’. The average rating was 3.92, reflecting that 14 of 25 responses, or 56%, rated communication at 4 or higher. Clarity of communication from the project management was assessed using a range from ‘never clear’ to ‘always clear’ and received an average rating of 3.48, 10 of 24 responses rating this at 4 or higher. Given that 14 respondents or 58% rated clarity at 3 or below, this appears to be more of an issue than timeliness. It may be that the language of the European Commission is not always clear to practitioners, policy makers and representatives in the Network, or the translation of such language into terms comprehensible to Network members may not have been sufficiently clear.

Communication among Network members was rated at 3.56, with 14 of 25 responses rating it at 4 or higher, towards the upper end of the scale, indicating that communication was ‘easy’. Nearly as many responses, 11 out of 25, rated ease of communication at 3 or below.

Clearly, the ease of communicating amongst Network members varied considerably.

12

Table 3.2 Experience of communication

Communication from the project management was:

Communication amongst Network members was:

Never clear

2

3

4

5

Always clear

8%

16%

36%

12%

16%

12%

2

4

9

3

4

3

25

Difficult to achieve

2

3

4

5

Easy to achieve

Total

8%

24%

12%

28%

16%

12%

2

6

3

7

4

3

Total

Average rating 3.48

Average rating 3.56

25

Communicating internationally did not seem to raise many issues, with 17 or 71% of 24 responses rating this 4 or higher towards ‘raising no problems’, with an average of 4.17.

Communicating internationally raised:

Significant problems

2

3

4

5

No problems

4%

17%

8%

25%

21%

25%

1

4

2

6

5

6

Total

Average rating 4.17

24

Positives were stressed in the accompanying comments, noting the skills of those working in a second language: As an English speaker working in a project using English as its primary language, communication for me has been relatively straightforward. My wholehearted admiration goes out to other participants for their skills in working in English. At times the complexity of some of the requirements and the systems being used has been slightly problematic but over the piece I think communication has been remarkably successful.

Generally technology meant communication was unproblematic. Partners were also generally extremely responsive and helpful.

Not all comments were positive. One noted that too much text was sometimes used and another that objectives were not always clear. The use of English as the medium of communication was questioned: Most of our members do not speak English, so it would be more effective if EPNoSL would operate in various major European languages such as French, Spanish, German and Russian.

13

However, the overall impression in terms of international Networking was that this was a positive experience for the majority of people.

3.3.2 Problems or obstacles encountered Respondents were asked if Network members had encountered any problems or obstacles and, if so, the degree to which these had impacted on their ability to achieve planned objectives. The scale for response was from 1, ‘problem had not impacted at all’, through to 6, ‘had impacted a lot’. Twenty-five responses generated an average rating of between 2.5 and 3.5 for the majority of factors, indicating that the problems identified in the question were commonly experienced and that they had a moderate impact on ability to achieve planned objectives, as in Table 3.3 below.

Table 3.3 Impact of problems obstacles on achievement of planned objectives Problem/obstacle encountered

Average rating of impact on achieving planned objectives

Engaging national policy makers

3.61

Engaging regional policymakers

3.50

Differing working methods amongst partners

3.38

Access to regional policy makers

3.35

Too few opportunities for discussion among partners

3.30

Access to national policy makers

3.29

Lack of clarity in project timelines

3.29

Differing understandings of key concepts among partners

3.29

Lack of clarity in project objectives

3.21

Differing objectives amongst partners

3.13

Time available for the work

3.00

Reaching practitioners

2.57

Motivating others to undertake tasks

2.52

Resources other than time available for the work

2.43

Working in English

2.22

Engaging policymakers at regional and national level was most problematic. The high average rating suggests that this issue impacted considerably on achieving planned objectives and proved a significant obstacle. One comment implied that the Commission might be more proactive in facilitating contact: The Commission has not as of yet supplied us with supporting materials to reach high level policy officials.

Overall, none of the scores were below 2: all of the identified problems/obstacles had a noticeable impact on achievement of objectives.

14

Average ratings can conceal where problems or obstacles were considerably more significant for individuals or organisations within the Network. Three comments indicated the difficulty of working in English and engaging others in the Network, given the English medium. While overall, this does not seem to have been as significant a problem as engaging policy makers, it may nevertheless have been a critical issue for some Network members. Three comments suggested that time was a significant issue, not only for Network members but for policy makers and practitioners who already manage heavy agendas. The time taken up by project administration was seen to be detrimental to core activities: The process of attending to administrative and financial issues is OVERLY time consuming and thus it/these take away time from content type of work.

A further comment raised the issue of lack of diversity amongst the theory adopted to underpin school leadership, questioning the ways of looking at equity and distributed leadership promoted through the Network: There was a lot of unclarity as to the content of the concepts used. It was more of a one paradigmatic way of looking with equity and distributed leadership as the concepts to worship.

Other comments raised the issues of depth of discussion and insufficient attention to context: The need for more depth in the discussions, thanks to strong and shared frameworks. The approach of 'good practice' is quite superficial and does not always take into account the contextual factors.

An over-emphasis on general education, to the detriment of a focus on vocational education, was also raised as an issue by another respondent.

Overall, the responses to this question suggest that Network members faced a range of significant problems/obstacles in achieving the objectives embedded in the Work Plans. The degree to which each of these impacted on the achievement of objectives varied between individuals and organisations, but none of the issues identified by the question appear to be irrelevant or unimportant in constraining the efforts to achieve objectives. Equally, the ratings suggest that none of the problems identified was so problematic as to be insuperable.

3.3.3 Responses to problems or obstacles Respondents were asked how they had reacted to the experience of problems or obstacles and, specifically, in what ways they had made adjustments in response in order to achieve planned objectives. Twenty-one respondents indicated changes that they had made, as in Figure 3.2 below:

15

Adjustments to initial plans to achieve EPNoSL objectives

Adapted the objectives Used additional resource from myself or my organisation to achieve a specific objective Changed the personnel involved to achieve a specific objective Increased the personnel involved to achieve a specific objective Brought forward timeline to achieve a specific objective Extended timeline to achieve a specific objective 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0%

Figure 3.2 Adjustments made to original plans to achieve planned objectives The adequacy of resources, in terms of the ambition of the original plan in relation to the resource available to achieve it, comes under question. More resources were added either by individuals or their organisation in a number of cases. For example, time or personnel were allocated above the levels originally anticipated. In a minority of cases timelines had to be brought forward rather than extended. One specific comment was that the allocated resource was insufficient to cover the costs of planning and running a national event.

Additional comments suggest that there was a range of issues shaping adaptations in response to problems encountered. In three cases, engaging practitioners or ministries was highly problematic as they were engaged in their own activities and strategies, and not open to others. In one case there was a strong sense of being contracted and thus obliged to achieve all the specified objectives, and responses indicated tension between the wish to achieve the planned objectives and the difficulties of doing so within constrained resources.

3.3.4 Overall satisfaction with project management The questionnaire asked about overall satisfaction with management throughout the life of the project. Twenty of 25 responses, or 80%, indicated that, overall, the project managed well or very well to achieve its objectives.

16

Table 3.4 Experience of project management overall Answer choices

Responses

Overall, the project was very well managed to achieve its objectives

28%

7

Overall, the project was well managed to achieve its objectives

52%

13

Overall the project management was not well managed to achieve its objectives

20%

5

Total

25

Some of the issues in managing the Network were raised in the comments. For example, one respondent wished for greater opportunity for in-depth discussion and less emphasis on meeting planned objectives. The appropriateness of activities is, of course, a matter of judgement, although meeting objectives for which the Network is contracted is axiomatic.

The comment suggests issues of competing agendas for members. A further comment questioned the ability of the management to translate some of the work into 'practical relevance', an outcome that the responses to previous questions presented in this report suggest was indeed problematic for many Network members.

Three additional comments spoke positively about the project management, for example: Remarkably well managed and having to meet the requirements of funders and balance the competing agendas of such a high number of partners.

Given the complexity of coordinating a geographically dispersed and highly diverse membership, addressing multiple complex and challenging objectives, the result is an endorsement of the skills of the project managers.

17

4. QUESTIONNAIRE RESULTS: SKILLS, KNOWLEDGE OR ATTITUDES ACQUIRED 4.1 Outcomes of Thematic Areas Explored During EPNoSL Events The third level of Kirkpatrick’s model relates to changes in participants’ skills, knowledge, and attitudes. For the purpose of this evaluation, it was more appropriate to focus on examining changes in knowledge and understanding, including shifts in attitude arising from respondents’ participation in the Network. Trying to capture and measure developments in knowledge and shifts in attitudes using a self-reflective questionnaire is of limited value, posing challenges for assessing the accuracy and reliability of the outcomes claimed. The diversity in the professional backgrounds of the participants also will have had some impact on the degree to which changes and shifts are discernible. Certain themes will have had a differential impact, depending on a number of factors, including past knowledge and experience, and where individuals are at in their thinking. It is also difficult to ascertain fully the causal effect of participation in the EPNoSL Network; other concurrent factors could have impacted the different participants’ perspectives. What can be established by this survey is a broad sense of the outcome of the work of EPNoSL and, with this in mind, the outcomes and impact of the main thematic areas explored during the EPNoSL events and related activities were examined in the survey.

Notwithstanding the measurement difficulties outlined, respondents were asked to reflect on their experiences in the Network and to indicate the degree to which they felt their knowledge and understanding of the different themes developed as a result of EPNoSL activity. The scale for response was from 1, where the work had not impacted at all, through to 6, where the thematic areas covered had impacted greatly. Twenty-five responses generated a positive response with an average rating between 3.64 to and 4.36. Some thematic areas elicited more positive outcomes, such as the work on Equity and Inclusion and Distributed leadership at the higher end of preferences, with Accountability at the lower end. Further evidence that the exploration of distributed leadership and equity/inclusion had the most impact on participants’ knowledge and understanding was that over half the respondents ranked these at either 5 or 6 on the scale. Figure 4.1 provides a summary of the overall ratings for each of the six themes.

18

Figure 4.3 Development in knowledge and understanding in key EPNoSL thematic areas There were a small number of additional comments on this item that referred to a lack of clarity in relation to the concepts that were examined, resulting in a less than favourable outcome in terms of development in knowledge and understanding.

The degree to which the PLA events, as a distinct component of the work of EPNoSL, had a specific impact on the participants’ knowledge and understanding of the issues raised can be seen in Table 4.1. Participants were asked to rank the extent to which their knowledge and understanding of school leadership improved through participation in the PLAs. A six-point scale was used, from 1, ‘not at all’, through to 6, signifying ‘a lot’. Responses here were broadly positive and the average rating was 4.29. Half the responses to this item were at either 5 or 6 on the scale, with only 8% claiming that the PLAs had no impact. While these claims are also subject to the issues explored in the introduction to this section, there is a sense that the perspectives on school leadership among participants were impacted by participation in the PLA activities.

Table 4.5 Degree of development in knowledge and understanding of themes: PLA participation

To what extent has your knowledge and or understanding of school leadership improved from participation in the PLAs?

Not at all

2

3

4

5

A lot

8%

8%

13%

21%

17%

33%

2

2

3

5

4

8

Total

Average rating 4.29

24

4.2 Impact of National Events A total of five national events were organised by Network members in their own countries prior to this evaluation. This type of activity was an overall aim of EPNoSL and the events were intended to be a key component of the dissemination of the outcomes of the Network 19

as a whole. Twenty-five of the respondents to the questionnaire indicated that they attended at least one. This corresponds to 75% of the total number of respondents (n=33), indicating a high level of commitment to the national events among those who completed the questionnaire. Figure 4.2 provides details on the level of attendance at each. The most recent national event in Vilnius was the most attended, with 22 of the 25 respondents present.

Figure 4.4 Number in attendance at national events The degree to which national events impacted on participants’ knowledge and understanding of the issues raised and discussed may be seen in Table 4.2.

Table 4.6 Degree of development in knowledge and understanding: National events

The overall impact of national event

Not at all

2

3

4

5

A lot

Total

Average rating

9%

9%

18%

32%

18%

14%

22

3.82

Respondents’ sense of the impact of these events, while broadly positive with an average rating of 3.82, was not reviewed as positively as the PLA events and the thematic work covered on these occasions. A number of additional comments were included for this questionnaire item. What emerged was additional information on other national events beyond those listed on the questionnaire. These included regional, sub-regional and local workshops that were open to a broad range of stakeholders. Other members organised regional conferences specifically for teachers and head teachers. It was not always possible to establish where these events took place, due to the anonymity of information on the questionnaires. It is evident that events were organised in Austria, Malta, Spain, Holland and

20

Estonia. It will be necessary to follow up this activity in future components of evaluation because of its potential to impact directly on practice in schools.

4.3 Impact of EPNoSL Online Over the course of the project, online activities attracted 10,639 unique visitors to the EPNoSL portal and 185 views of online discussion activities. Detailed information on the numbers participating is available in the EPNoSL evaluation report Del 1.1, dated 9.1.14. The main types of online activities were webinars and keynote addresses, followed by online discussions and the provision of keynote articles to serve as a catalyst for discussion in different ways. The online activity focused on nine different themes, with some corresponding development of the themes covered in the PLA/national events. Participants were asked to comment on the efficacy of these activities in terms of their impact on the overall development of knowledge/understanding. A scale of 1–6 was used here, with a score of 1 indicating no impact and 6 indicating a considerable impact. Again, 25 people responded to this item and the answers on this occasion were generally mixed. It is clear that there was a range of views on the experience of the online environment – the average rating was 3.47, slightly over the mid-point. It is worth noting that there was a slight propensity among respondents to be more negatively disposed to the online platform – an average of four respondents indicated that the online activities had not impacted at all, while an average of three viewed the online work as having considerable impact. A review of the overall preferences reveals that there was little difference among those expressed for any of the themes covered in sessions. This suggests that, in this sample, the online activity was experienced in a similar way, irrespective of the topic or type of activity. Figure 4.3 provides an overview of the expressed preferences of the groups on the different themes.

Figure 4.5: Outcome of online platform on knowledge and or understanding A number of comments were added to this questionnaire item. All indicated a positive experience, with a lack of exposure to or engagement with the online activities being cited as the reason for their lack of impact rather than anything to do with the quality of the

21

material or the presentations themselves: ‘I did not engage actively with the webinars other than school leadership – this is the reason for their lack of impact on me, not their quality.’

Positive comments also related to the opportunity provided by the online platform to explore how issues are mediated by different national contexts: Being able to have insight into how school leadership is developed in different national systems has been a real boon and has provided significant opportunities to reflect on how such developments are carried out within my own national context.

4.4 Efficacy of Learning Activities and Diverse Professional Backgrounds of Participants on Learning Outcomes Two particular features of EPNoSL were evaluated in order to establish if these impacted on the overall quality of the outcome of the Network activities. The first relates to the type of learning activities and learning protocols used during the Peer Learning Activities. Five of the key types of activity were included in the questionnaire item and participants were asked to rate each activity on a scale of 1–6 (where 1 indicated no impact and 6 indicated a lot) in terms of the degree to which they felt that the activity had facilitated a development in knowledge and understanding of the different issues addressed at the PLAs. Figure 4.4 provides a summary of the overall rating and it is clear from the findings that all of the activities were positively rated, with a clear preference for informal discussions and keynote inputs, with an average rating of 4.64 and 4.40 respectively. The activity that was least preferred was the whole group plenary, where the expressed preferences were just over the mid-point on the scale at 3.56. Having said this, it is worth noting that a high number of respondents (40%) to this item opted for number 3 or 4, at the mid-range of the scale, indicating a general lack of certainty in relation to the differential impact of the different types of activities.

Figure 4.6 Impact of different event activities on learning outcomes A key feature of the composition of the EPNoSL Network is how it brought together members with varied professional roles from a range of stakeholder groups in the field of education (see Figure 2.1). This provides for a variety of perspectives on the different issues and has the potential to contribute to high quality developmental discussions that could 22

have significant impact on policy and practice. With this in mind, respondents were asked to rate the degree to which the composition of the Network itself and the range of perspectives and positions therein influenced the overall quality of the learning and effectiveness of the PLA events. The findings here are mixed, the impact of some partners being viewed as less than others. All were broadly positive, yet some were unsure that this feature had a significant impact. This may be explained by the spaces that were created within the PLAs for the variety of voices to be heard and views to be voiced. Maybe there was a lack of awareness in the group at large that the Network was quite so heterogeneous in terms of its membership. For one respondent this certainly was the case, as evidenced by a written comment to this effect. An examination of the findings on this item, summarised in Figure 4.5, indicates that the overall view was broadly positive. However, with an average rating of 3.6, there was no strong endorsement of the impact of the variety of perspectives on the learning outcomes. The involvement of school leaders themselves was viewed most positively with an average rating of 4.28, followed by higher education personnel (comprising the majority of the sample in this survey) with an average rating of 3.92. The impacts of parents groups (3.29), regional administrators (3.29) and employer representatives (3.36) were rated lowest. It is difficult to be definite about claims made here regarding this particular item. The evaluation may relate to the relatively low number of participants in these categories compared to those of others, as much as to the nature of impact, and in all cases remains positive.

Figure 4.7 Impact of different partner professional roles on learning outcomes

23

5. QUESTIONNAIRE RESULTS: CHANGES IN JOB BEHAVIOUR The third section of the questionnaire reflects Kirkpatrick's third level of impact, that is, the degree to which job behaviour changed as a result of development activities.

5.1 Changes in Importance Accorded to School Leadership Development Twenty respondents indicated that they were engaged with leader development and that, as a result of involvement in EPNoSL, half now gave greater importance to leader development: six people or 25% had allocated more time to leader development and four people had changed their priorities. Three explanatory comments suggested that equity and social justice issues had become more significant in respondents' development activities: Issues of social justice and equity have risen up the priority list here in Scotland, not just because of EPNoSL but it has had an influence.

I give greater attention to equity issues.

One respondent noted that seeing things in a European context had increased motivation to develop school leader training activities.

Respondents also noted a range of ways in which their approach to training had developed, as indicated in Table 5.1. Over half of the 24 respondents had changed some aspect of their pedagogy. The responses also indicate that EPNoSL materials are used not only by Network members in development/training activities but are also diffused out to others as they are brought to their attention by Network members.

Table 5.7 Respondents’ views of the development of their approach to training Answer choices

Responses

If you do not develop leaders, please check this box

46%

11

I have changed my pedagogy in terms of aims

13%

3

I have changed my pedagogy in terms of content

17%

4

I have changed my pedagogy in terms of methods

21%

5

I use EPNoSL materials in my teaching

38%

9

I have directed others to EPNoSL materials

46%

11

I have changed my evaluation of learning

8%

Total

24

2 24

Additional comments noted that evaluation is more focused on outcomes relating to the work of school leaders, and that EPNoSL had been influential in the general discussion and debate about leader development in Scotland.

Responses also indicated that the practice of leadership had changed as a result of EPNoSL activities, as seen in Figure 5.1.

I give more time to influencing relevant national policy

Nature of change

I make reference to standards or indicators generated

by EPNoSL in deciding how to act I have changed the aims of educational leadership that I communicate to others I give more time to influencing relevant regional policy I have changed my priorities in what I give time to as a leader I give more time to influencing relevant local policy

I have changed my relationship with those I lead 0

5

10

15

20

25

30

35

% Responding

Figure 5.8 Changes in practice Additional comments in one case stressed that the importance of equity had grown. Another indicated the influence of EPNoSL in the development of a national centre for leader training, and another that more time is given to European policy.

If this group is at all representative of the Network as a whole, it might be inferred that attention to influencing national policy has increased. However, this should be treated with some caution in that the response rate was relatively small; also, in answer to each of the questions in this section, one respondent stressed that nothing had changed. Consequently, there are undoubtedly Network members who did not change their job behaviour as a result of EPNoSL activities and it is not possible to assess the relative size of the group that did and did not. All that can be stated with certainty is that, of the 24 responses – approximately a third of the full Network – most had instituted one or more changes in their engagement with policy or their own practice as a result of their involvement in the Network.

Given the primary aim of the Network to influence and support policy development, responses in relation to this area are of particular significance. One question focused specifically on changes in job behaviour in relation to influencing and supporting policy. Of 22 responses, 13 or 59% indicated that they gave greater importance to leader development policy and 9 or 41% that they had increased the amount of time spent on leader development policy. One person who had not previously been focused on leader development policy had become so. As discussed in the previous paragraph, it is not possible 25

to extrapolate from this change to the behaviour of the whole Network. It is likely that respondents to the survey are those who are most active within the Network. However, it is possible to conclude that a range of changes in behaviour in developing leaders, in the practice of leadership and in influencing policy is perceived to have resulted from the activities of EPNoSL. The next section considers the evidence for Kirkpatrick Level 4, the impact of the changed behaviours.

26

6. QUESTIONNAIRE RESULTS: IMPACT OF CHANGED BEHAVIOURS This final level in Kirkpatrick’s model takes the perspective of examining the extent to which there have been identifiable changes in policy and practice as a result of an initiative or catalyst for change. Because of the complexity involved in change within the field of education, examining the issue of impact or even tangible influence at this juncture and looking for evidence of activity is problematic. Many of these issues were pointed out in comments by respondents to open-ended questions in the survey. We recognised these in our design and analysis of the responses. The timescale within which to recognise and measure changes and impacts reliably was too short, and the issue of causality too difficult, to trace the origin or catalyst for developments specifically back to EPNoSL. Notwithstanding these and other limitations, the Network was already aware that some projects, policies and initiatives currently up and running were directly related to the work carried out in EPNoSL. There was also a sense that other activities were planned. Consequently, two dimensions of impact/influence were explored in this survey. The first relate to achievements already evident and already impacting policy and/or practice. The second relate to initiatives that were in train or in a development stage but likely to have some results in the future. The latter focus provided a way of capturing some aspects of future work that would continue after EPNoSL had come to an end. In this way it was possible to get a better sense of what the project could achieve.

6.1 Priority Changes in Organisations Respondents were asked to indicate if their participation in the EPNoSL project had led to policy changes relating to school leadership in their professional work contexts. It is worth noting here that there may have been professional contexts where this issue was neither possible nor applicable. There were 25 responses to this item, with 60% (n=15) indicating that changes had taken place and 40% (n=10) stating that there were no changes in priorities. Eight respondents skipped this question.

A number of comments were added to this item. In some cases these provided evidence of a direct, discernible impact: ‘Greater focus on international aspects of leadership development in strategic planning’, ‘We changed leaders development program’ and, in higher education, some module content changes were made. Other comments raised issues relating to establishing causality and also a view that the Network is part of a broader sphere of influence that needs to be seen positively if it complements and develops work in other areas: ‘Leadership is now in policy contributions of my organization. This is however more due to changes at the EU level than through the project.’

This is a difficult question to answer as much of what was/is going on... would be broadly in line with EPNoSL in any case. I think EPNoSL has also made Scotland think about how school leaders spread their influence beyond their school – locally and nationally.

There was a view expressed that the Network led to ‘a stronger perspective on leadership’ and ‘gave more attention to leadership development’. The work has a role in raising awareness about leadership issues and its impact will take time to filter through. The fact 27

that it has started a process of awareness-building, in itself, is evidence of a positive outcome to the Network: more colleagues became involved with our work in this topic. All seminars and workshops organized were attended and appreciated. Ideas for strengthening collaboration sprang out.

6.2 Areas of Impact and Influence Respondents were asked to indicate specific areas where the work in EPNoSL has had some impact and to develop and provide evidence of this activity. A high number of respondents (almost 50%) skipped this question, so it is important to bear this in mind when reviewing the findings. Table 5.1 provides an overview of the different areas of impact/influence. The highest levels of impact were in school leadership development programmes (88%) and in teacher education (59%). The level of influence on national policy was also high at 53%. It can be argued that the low level of impact on classroom practice and student outcomes, at 18% and 12% respectively, concurs with other work in the area of implementation science (Joyce and Showers, 20024) where higher level approaches to change that are not accompanied by direct support and intervention at classroom level often do not bring about much change in practice. However, this may be the result of too narrow a focus on tangible outcomes within what is a widely acknowledged as a contested field of scholarship, or a lack of recognition of smaller, less discernible shifts in thinking and attitudes that, in turn, shapes and frames how people approach particular issues in a broad, longer-term context.

There were a number of comments on this item that qualify the more quantitative dimension in Table 6.1. There is some clear evidence in these comments of activity that is up and running and in areas of influence. Three areas where impact was most discernible were ministry/policy level, leadership development programmes, higher education course development – and sometimes all three, or combinations of the three: Increased awareness on the issue of SL in Europe, of EPNoSL materials as we communicated with and involved representatives of national and local policy makers, representatives of implementers of school leadership training and development programmes and teacher educators, communicated/and educated teachers and other school representatives directly.

Representatives of LT Ministry, Local administration, University, Schools took part in Vilnius PLA activities, on-line activities, increased interest in the field of SL.

4

Joyce, B. R., & Showers, B. (2002). Student achievement through staff development. ASCD.

28

Table 6.8 EPNoSL: Areas of impact Response per cent 52.9%

Response count 9

Local policy (e.g. municipal)

23.5%

4

School leadership training and development programmes

88.2%

15

Teacher education

58.8%

10

Curriculum development

35.3%

6

Classroom practice

17.6%

3

Student outcomes

11.8%

2

Answer options National policy

Comments related to Ministry/Policy We are working in collaboration with the Ministry of Education, informing the authorities about the work developed in our institution and responding to the authorities requests, for example, participating in 2 workshops organized by the National Inspectorate My organization has come up with policy recommendations for governments Comments related to Professional Development of School Leaders/Teachers conceiving a programme for school leaders’ and teachers’ continuous professional development We are working in collaboration with the Schools' Association Training Centres responding to local practitioners’ demands on school leaders’ and teachers’ professional development related to the different issues addressed by EPNoSL. Curriculum development for school leadership training.

The use of resources produced as part of the project on leadership programmes Within the revised Teacher Education Standards, GTC Scotland has written in materials about leadership to all the Standards, from being a student teacher to being a Head Teacher. We have also included literally the same text on Values (including social justice) to all the Standards. Clearly, writing something into Standards does not guarantee change in teacher behaviour or performance but these changes have certainly already meant a higher profile for these aspects in discussion within education circles – discussions which will be even more focused once Professional Update comes fully into effect.

9 seminars and workshops involving 451 participants, coaching 4 leaders on developing their school improvement projects

29

Comments related to Higher Education Coursework Use of EPNoSL materials has changed the way student teachers and leaders see leadership to small degree. dozen Master dissertations produced Some student groups have voluntarily chosen to focus on equity and leadership as a major project working together.

There was also a sense that EPNoSL dovetails with other EU-funded projects working in the area of leadership. It has contributed to a EU project on leadership in VET and a broader sense that the context was created within the Network, where ‘Representatives of the University and schools are interested to research, discuss the issues, be involved and go ahead for a change’.

6.4 Planned Activities Respondents were asked to identify activities planned for the future in which they believed that the impact of EPNoSL would be evident. Three main areas of activity emerged as spaces where impact could be evidenced. Again, for this item there was a high number of respondents (n=15) who skipped the question. Of the 18 responses tallied, 61% had planned activity in the area of leadership training and development and 22% of respondents indicated that plans were in place in the area of policy, with only 11 planning changes in organisational practice. It is difficult to make definite claims for the findings of this item. The high number of skipped answers reduced what was already a small sample by almost 50%, while the varied composition of the Network and the number from the higher education sector in the sample limited the representativeness of the findings on this item. It is possible to claim that 18 respondents indicated that some future activity was planned that resulted from their participation in EPNoSL.

30

Figure 6.9 Intended outcomes or impact When asked to indicate the likely timescale for the proposed activities, over 50% indicated that they had activities planned for the subsequent year (see Table 6.2); 21% indicated plans that would take effect in a two to three year framework. These subgroups comprise 13 respondents. However, 31% (n=6) of the respondents were unsure when the plans would take effect, which may indicate early stages of development or a lack of certainty in relation to the likelihood of the plans coming to fruition.

Table 6.9 Timescale for proposed activities Answer options

Response

Response

per cent

count

Next year

47.4%

9

2-3 years

21.1%

4

4-5 years

0.0%

0

> 5years

0.0%

0

31.6%

6

Unsure

answered question

19

skipped question

14

Additional comments by respondents indicated a sense of the type of planned activity. The examples below indicate that initiatives are in train in a diverse range of areas, from activities to facilitate the dissemination of EPNoSL material, including translation and enhanced online access, to international visits of personnel, Networking and the organisation of conferences: We developed a separate division for EPNoSL in the LEU PKTI webpage in order people could very easy find the materials and etc.

We developed EPNoSL booklets in Lithuanian language and disseminating it in the LEU PKTI. We had partners’ visits from Ukraine and we have initiatives to work together in the new projects on the issue of SL. National Networking towards a new leadership culture.

We have an initiative to organise international conference on SL this year on the 1819th November, 2014 in Vilnius.

Leadership development activity is evident in higher education and among some professional organisations, including ministries with a training and development remit: As a result of my work with EPNoSL I have introduced a lecture and seminar on leadership in education in one of my BA Education Studies modules.

31

The programme for school leaders ‘and teachers’ continuous professional development requested by the Ministry of Education, which will be implemented during the next schooling year within a national scope.

Considering EPNoSL project ideas our institution proposes for the Ministry possible topics to be included in CPD activities for school leaders.

32

7. CONCLUDING COMMENTS: SECTION ONE This part of the evaluation set out to examine three main dimensions of the work of EPNoSL: the effectiveness of the Network’s ways of working; capacity building within Network participants and their communities; and, finally, the degree to which the activities of the Network have impacted and might further impact on policy development and leadership practice.

A number of limitations have been detailed in the report including sample size, the timing with respect to identifying evidence of impact and, arguably the more challenging issue of causality, the degree to which identified outcomes can be attributed specifically to EPNoSL activity. Notwithstanding these issues, the survey revealed a number of important factors relating to how Network participants worked together, developments in thinking across the Network as well as evidence of impact already in place or at different stages of planning.

Responses to questions relating to ways of working within the Network, including online, PLA and national events as well as the overall composition of the Network itself, generally indicated a positive experience yet did not point conclusively to any strong overall preferences on these dimensions. However, there are some aspects of the findings that are worth noting. The positive view of the participation of school leaders in the Network would suggest that this is an area where there may be scope for development. An increase in their proportion within Network members was anticipated to lead to greater effectiveness in facilitating the application of ideas and concepts to practical contexts. Network meetings, keynotes and informal discussions were identified as the most positive ways of working and could have been given a greater proportion of time.

There is considerable evidence in the survey of a high level of commitment to the Network among members, as indicated by the level of participation in PLAs and the efforts that have been identified in the report in relation to the dissemination of the outcomes of Network themes among different cohorts. This includes making materials developed for the Network available online and translating materials into different languages.

There are areas where the impact of the Network was already discernible. The most positive outcomes evidenced are in the area of leadership development programmes and in course/ module reviews in higher education institutions with a leadership development remit. It is also clear that, while there has been some success in disseminating ideas and materials to key stakeholders, engaging policymakers at regional and national level proved problematic and the consensus in the data suggests that this issue impacted considerably on achieving planned objectives and proved a significant obstacle.

Within the Network there is evidence that participants’ perspectives on school leadership were changed by participation in the PLA activities. It is also possible to conclude that a range of changes in behaviour in developing leaders, in the practice of leadership and in influencing policy, was perceived among respondents to have resulted from the activities of EPNoSL. Where this may lead to in the future is not yet identifiable.

33

While it is too early to identify fully a significant number of demonstrable outcomes on practice, the quality and impact of the initiatives and activities that were already in place or at advanced stages of planning would suggest that the Network has the potential to effect ongoing change into the future. Taking a broad overview of the four levels of impact, the overall positive slant of the ratings in the survey and the positive nature of the majority of comments, there is evidence that participation in the Network provided people with a range of challenging perspectives in relation to the practice of school leadership by exploring ways of undertaking leadership, resulting in orienting the focus of leadership decidedly towards more equitable outcomes. It is likely that the ways of looking at leadership as a practice that have been enriched by participation in EPNoSL will contribute to the ongoing discourse on school leadership in a range of contexts.

While there is clear evidence of perceptions that practice has been impacted, the primary objective of the Network was to influence policy development. It is more difficult to discern evidence of success in this area, in part because of the methodological reasons outlined earlier. There is evidence of a perception that the discourse generated by EPNoSL has contributed to the ongoing development of policy in some parts of Europe. There is also evidence of a perception that in other parts it has not. The overall conclusion appears to be that direct impact on policy is not feasible in many contexts and that the impact is more likely to have been indirect through the revised knowledge, skills and understanding of Network members as they communicate with their communities, generating a flow of ideas that may influence policy at a number of levels. There is evidence that the Network has been active and successful in generating such a discourse and in directing this toward policymaking fora.

Evaluation of the impact of the Network's activities on policy is considered in more detail in Section iii.

34

8. SECTION TWO: DOCUMENT REVIEWS This section of the report focuses on the relevance and value of the intellectual activities and products of EPNoSL's work for shaping policy and practice.

8.1 Approach to Evaluation A listing of all the documents resulting from EPNoSL activities is given in Appendix 1. This has been compiled with the assistance of all Network members to ensure completeness. It represents a large body of work and reflects the efforts of all members to contribute knowledge and frameworks to support policy development in Europe. The volume necessitated a sampling process to consider the nature and impact of the documents.

Having identified a sample, documents were scrutinised using a consistent framework.

Finally, an overview of the nature and cumulative impact of the documents was evaluated, considering particularly the overall cohesion and cumulative effect of outputs.

8.1.1 Sampling The first stage of sampling analysed and categorised the documents. Four categories were identified with a varying number of documents in each category: a) Briefing/review documents

27 documents

b) Discussion articles/keynotes

11 documents

c) Empirical studies

11 documents

d) Additional published outcomes

4 documents.

Briefing or review documents Briefing or review documents are of two kinds. Sixteen essentially describe the context in specific European nation states or subunits of states. The case studies were also reported in a single summary document designed to provide a baseline, presenting the situation in the geographic area in question at the start of the work of EPNoSL. An overarching review document considered the state of affairs in school leadership across Europe, focusing on six themes:

35



Trends and tendencies in external expectations: policies culture and governance



Translation of external expectations into internal meaning and direction



Understanding and empowering teachers and other staff



Structuring and culturing schools



Working with partners and the external environment



School leaders’ preparation and development.

This category of document therefore depicts current practice, theory, issues and aspirations in individual nation states and across Europe. The papers focus on reviewing policy, practice and/or theory, generally in relation to school leadership in Europe. Finally, two papers provide a framework for action for Network members: policy indicators and a policyinfluencing toolset.

Discussion articles or keynotes Discussion articles or keynotes were designed to provoke thought and follow-up discussion, whether at peer learning activities, national or regional events or by means of the web through, for example, webinars. Discussion documents and keynotes were aligned to themes emerging as priorities for the Network; that is, within the context of achieving greater equity though learning: • • • • •

Autonomy Accountability Distributed forms of leadership Policy response Educating school leaders and other professionals.

Empirical studies Empirical studies are generally small-scale research projects investigating a focused aspect of school leadership policy or school leader development, again across Europe or within European nation states or subunits.

Additional published outcomes Additional published outcomes are few in number, but encompass academic papers for journals and a conference using the knowledge gained through involvement in EPNoSL. 8.3.2 Selection of documents within categories Purposive sampling of documents within categories was designed to capture: a) As wide a spread as possible of European state/regions' perspectives b) Subcategories within each category of documents c) Different themes, as noted in the survey.

An overview of the resulting sample articles is presented in Table 8.1, below:

36

Table 8.1: Document sample by category and country of focus Country(ies) of focus Europe

Briefing/review documents

Discussion articles/keynotes

5

Empirical studies 1

Austria

1

Denmark

1

Estonia

1

Finland

1

France

1

Additional published outcomes 1

Germany Greece Hungary Italy Latvia

1

Lithuania

1

Netherlands Poland Portugal

1 1

Scotland

1

Slovenia Spain Sweden

1

United Kingdom

1

Total

7/27 (26%)

5/11 (45%)

5/11 (45%)

2/4 (50%)

A full listing of the sampled documents is given in Appendix 2. A lower percentage of documents sampled was necessary in the briefing and review documents, given the larger number of documents in the wide range of countries. Consequently, a synopsis document, a small sample of single state/region case studies and some European overview reviews of theory, policy and practice were selected.

37

8.1.2 Analysis of individual outputs The framework for analysis of individual outputs began with a description of the document in question and progressed through evaluation of the objectives stated within it. Next, its contribution to cumulative understanding within the full set of EPNoSL documents was evaluated. See Table 2 below: Table 8.2: Document analysis framework Title Author(s) Focus on Europe or region/ state/administrative district Category of document: context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tools Major theme(s) Objectives Degree to which objectives are achieved Connectivity to other EPNoSL documents

8.1.3 Analysis of overall cohesion and cumulative effect The final stage of evaluation was an assessment of the overall cohesion and cumulative effect of the work as a body. This considered the degree to which it had established a sound foundation, and built upon that foundation to progress thinking about both policy and policy influences within the European context.

38

9. Document Analysis Results Overall, 19 documents were evaluated. The results are given in the tables that follow: a) Briefing/review documents: 7 out of 27 documents Title

The European Scenery on School Leadership: National Case Reports

Author(s)

Kathy Kikis-Papadakis, with contributions from the EPNoSL partners

Focus on Europe or region/ state/administrative district Category of document: context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

This document is in part an analytical framework tool and in part context setting. It is intended to provide an adequate definition of leadership, followed by a series of theoretical frameworks for a number of key areas of leadership. These areas provide a framework for the analysis of reports on the state of play in selected European countries.

The major themes included are: Trends and tendencies in external expectations Translation of external expectations into internal meaning and direction Understanding and empowering teachers and other staff Structuring and culturing schools Working with partners and the external environment School leaders' recruitment, preparation and development.

Objectives

The report aims to provide an overview of the similarities and differences in trends in educational leadership in the EPNOSL countries contributing country reports. The overview is intended to provoke comment and discussion.

Degree to which objectives are achieved

Attempting to fit the diversity of European practice into a single overview is highly challenging. Success is in part dependent on the clarity of the framework for each theme into which differing practice can be categorised.

39

Some themes lend themselves more easily to clear, concise frameworks than others. As a consequence, some sections report analysis; others appear to offer more of a list of varying practice in partner countries.

Overall, the degree of variation in approach to policy and practice is captured well and some themes are presented so that the reader is enabled to consider trends and where their own country or individual practice fits and why. In the case of other themes, the variation overwhelms any sense of overview, making trends difficult to discern.

Connectivity to other EPNoSL documents

This document provides context information in relation to key themes, some of which are pursued in greater depth in successive EPNoSL documents, including system/distributed leadership and ethical issues.

Title

Typology of European space for Improvement on School Leadership Policy and Practice

Author(s)

Ana Paula Silva and EPNoSL partners

Focus on Europe or region/ state/administrative district

European Focus – 17 EPNoSL Partner countries

Category of document:

Context setting

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Objectives

40

This review addresses a number of factors that contribute to the leadership landscape in each of the 17 partner countries.

To provide a synthesis of common issues and differences in the following areas: Views on SL Models of school leadership in individual countries including variations in models Recent changes in SL Recruitment and training

Degree to which objectives are achieved

Stakeholders in leadership Evidence of impact of leadership on education Leadership and inclusiveness Strengths and weaknesses in each system.

The synthesis provides a clear sense of the commonalities and differences across, and sometimes within, different national systems. These have very strong messages for the Network and for the manner in which different models of leadership is considered.

A number of issues are raised by this document, and the range of fundamental/ baseline structures in terms of policy, governance, selection and recruitment (among others) is very varied. This creates a space where many different views are constructed on what are school leaders. This has a particular impact on the discourse relating to SL, where perceptions relating to the idea of school leadership range from a view of leadership as the head to more distributed views. Many frame their perspectives in leadership within the more traditional higher hierarchical positions in the organisation. Broadly, the idea is either framed very decidedly within a management framework (in 10 countries) or, in some cases, leadership is not used as a term at all. Other countries incorporate a broader range of views on leadership. The study identifies a gap in the evidence in relation to the links between school leadership and overall improvement. The correlation is underresearched and poorly considered in many countries, which should be an issue of concern for all partners in EPNoSL.

Connectivity to other EPNoSL documents

41

The document complements the previous thematic work reviewed above. It also provides a key source of reference for many of the empirical studies carried out in national contexts. It contributes specifically to the Network by providing a synthesis of the leadership landscape in each of the participating countries.

Title

Consensus Report on European Policy and Practice on Leadership: Moving Towards Improved School Performances

Author(s)

Nóra Révai and Mária Szabó

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Context setting

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Stimulus paper

Objectives

The main objective was to share views and experiences of different target groups, so as to build a consensus to facilitate better policy recommendations. This report seeks to capture the key outcomes of this process.

Degree to which objectives are achieved

The report indicates that the eLearning forum was successful in revealing the level of complexity encompassed in the topic of school leadership. It also points out that a common focus on the centrality of the links between leadership and the moral purpose of education emerged from the seminars. The bringing together of researchers, scholarship and practitioners as a key component in building a better shared understanding of issues was achieved, albeit to a limited extent as, at times, the numbers participating were small. The overall design of the webinars was particularly effective in this regard and the work was supported by keynote articles and speakers, online discussion fora and an extensive reading list, should participants want to explore any of the issues more comprehensively.

This is a report on the eLearning forum from the first year of EPNoSL’s work.

The topics for the webinars were well chosen and dealt with key areas where dialogue was necessary as identified by background/state of the art reports. The topics covered included autonomy and responsibility, governance, leadership competence and development 42

and system leadership. A number of key issues are raised that have specific implications for ways of working within the Network: Language – the predominance of English as the lingua franca of the Network presented difficulties for those for whom English is not their first language Discourse and conceptually dense language sometimes needs to be unpacked, and regularly requires relating to other languages and contexts The impacts of contexts and cultures at many levels.

Connectivity to other EPNoSL documents

This links explicitly to the state of affairs document, produced to capture the range of practices among participants. The eLearning platform was a key process in seeking to promote dialogue and develop a common/ shared understanding of SL for the purpose of policy making.

Title

Lessons Learnt: Contribution to EPNoSL

Author(s)

Kathy Kikis-Papadakis, Cynthia Villalba

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Context setting

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Stimulus paper

Objectives

To provide a synthesis of the work of the Network over the first year of implementation

This report is based on the experience and knowledge acquired during the first year of implementation, 2011– 2012.

To identify areas for policy development in relation to schools leadership.

43

Degree to which objectives are achieved

The synthesis of the work of the first year of implementation is clearly articulated and a number of areas emerge for consideration by the Network.

Consolidating this work was a difficult task, as the list of areas examined in the Network and summarised on pp.

19–20 of this document will indicate. Key areas for policy development are also identified and explicated throughout the paper. This again provides additional insight into the magnitude of the task of achieving a common platform from which to consider improving the work of school leaders, and the overall impact on improved student outcomes and system improvement more generally.

Five policy areas on which the Network intends to focus are explicated; this provides clarity on how it will develop. Included in its focus is recognition of the differences across systems and the commitment to use these differing contexts as a catalyst for a more enriched understanding of leadership and a point of departure for school and system improvement. Part of this work also involves providing a definition or a descriptor of school leadership. At this point in the Network it was important to be clear about the remit of this field’s scholarship and practice.

The paper opens with a descriptor of school leadership that is broad and clearly positions the focus of the Network within this wide domain of school leadership: ‘encompassing other leaders in education… members of the formal leadership team and other persons who contribute to the aims of the school… including student leadership’ (p. 4).

Connectivity to other EPNoSL documents

44

This is an excellent summary of the work of the first year of the Network. In this way it provides a clear basis for other documents and is fundamentally connected to the other work and its related documentation. It gives a clear indication of the scope of the first year’s work and the manner in which partners have engaged in these deliberations. The report also indicates most clearly areas that continue to be highly contested and in this way sets up areas to be developed in the future by the Network.

Title

Promoting the Policy Agenda on School Leadership from the Perspective of Equity and Learning

Author(s)

FORTH Team

Focus on Europe or region/ state/administrative district

Europe and the Network members and beyond

Category of document:

Context setting

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Stimulus paper

Objectives

The objective is to provide a synthesis of EPNoSL’s work to 2014, through the way publication facilitates dissemination of its work on school leadership and acts as a resource for the broad range of stakeholders in the school leadership field. It also provides a framework for the work of EPNoSL in supporting and facilitating leadership policy development and implementation in the Network’s final year.

Degree to which objectives are achieved

The document provides a concise synthesis of the main areas worked through in the Network. The document is structured with headings corresponding to the main areas of activity in the Network. Under each of the headings the main outcomes of discussions, deliberation and in some case empirical studies are consolidated in order to provide clarity in relation to what the Network recommends for future actions. A key strength is the manner in which some of the contested issues are reframed so as to allow stakeholders to engage, insofar as is possible and appropriate, given the nuances of particular national/ sectorial contexts. The policy action pathways conclude the publication in a clear and accessible way. This is a most helpful publication.

Connectivity to other EPNoSL documents

If read online, the document is interactive and links readers directly to resources produced by the Network.

In this way the level of connectivity is excellent and the manner in which the document is structured facilitates

45

The work of EPNoSL and a map of future directions

easy navigation of the Network, the documents and the key ideas therein. Additional follow-up material is made highly accessible by the interactivity of the document. It is a most inclusive and user-friendly format.

Title

SWOT to the Construction of Policy Indicators on School Leadership

Author(s)

Andreas Kollias

Focus on Europe or region/ state/administrative district

European

Category of document:

Stimulus paper Context setting Analytical framework

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Objectives

The development of a set of indicators to provide policy makers with the tools and analysis to identify areas for policy attention, and to support and enhance school leadership for equity and learning.

To provide a synthesis of the range of policy contexts within which policy is developed and implemented. To identify the SWOT components of the policy contexts and to provide a steer on future policy directions in relation to three policy goals: 1. The promotion of an enabling school leadership environment.

2. The promotion of professional standards, evaluation and research on school leadership for equity and learning.

3. school leadership capacity building for equity and learning.

Degree to which objectives are achieved

46

This document provides a comprehensive synthesis of a number of key publications. It succeeds in providing a clear analysis of many of the different variables within

different contexts in areas directly related to the policy areas to be considered by the Network. There are some strong messages here, with distinct implications for the development and implementation of policy. The selection of key data items for analysis and commentary from both PISA and TALIS is a particularly useful support to future policy directions. The document is highly successful in highlighting the complexities in establishing and implementing coherent and comprehensive school leadership policies, which consequently have critical implications for the development of a Common European Set of Indicators on policy development on School Leadership in Europe. Each of the policy goals is comprehensively explored and key directions for development recommended, based on the synthesis of many of the documents prepared in the Network, as well as other comparative scholarship.

Connectivity to other EPNoSL documents

This document connects with many of the documents that have been developed. In various ways it bidirectionally draws on and synthesises many of the context documents, as well as helps to shape the future work of the Network and the supporting material ensuing from this work.

Title

Commonly Agreed Plan on the Content and Form of EPNOSL’S Policy Influence Toolset

Author(s)

Philip Woods and Lejf Moos

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Stimulus paper

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Analytical framework

47

A focus on equity and learning in policy development in five policy areas:

1.

2.

3.

4.

5.

Autonomy Accountability Distributed forms of leadership Policy response, and Educating school leaders and other professionals.

Plus an additional three derived from the workplan:

Objectives

1. Policy coherence, 2. Processes of stakeholders’ collaboration 3. Self-assessment.

The aim of this document was to establish consensus among EPNoSL partners on the conceptual orientation, type, content, form and media of the EPNoSL policy reflection tools, and to offer guidance regarding their design, development, review and validation.

There is a dual function to the manner in which the Network will focus on influencing policy. The tools should enable reflections concerning the requirements for policy developments at the level of laws, programmes, projects, etc., and at the wider level of policy discourse(s) and its interrelationships with discourse(s), within and between various stakeholder groups.

Another objective is to facilitate reflections on the nature of the issues that are considered, the proposed policy solutions and the implications of such understandings for the development of policy initiatives on sl.

Degree to which objectives are achieved

The materials advise a menu-type approach so that contextualisation is facilitated and enabled. The policies that emerge are aimed at improving the quality and effectiveness of school leadership in a way that remains responsive to the idiosyncrasies of specific national or regional contexts. The complexity of the task is well articulated and the wicked nature of the issues at the focus of policy development, in relation to equity in particular, is clearly framed.

A clear rationale is provided for a four-stage deliberative policy analysis approach and a soft systems approach, having critiqued a number of cyclical models of policy development and a policy stream approach. As a model,

48

the latter could be viewed as overly opportunistic, arguably manipulative in its approach, taking advantage of the timing of a particular congruence of factors. It is important that there is clarity in relation to assumptions that underpin the policy tools. It is equally important to be explicit about what alternative models were considered in developing the preferred approach, so as to avoid slippage within the toolset into models of policy development that were not part of the framework.

Clarity is provided for definitions to ensure coherence and the team commitment to eight toolsets, including the general introductory set as well as a number of partners, to ensure the appropriateness of the toolsets.

Connectivity to other EPNoSL documents

This document sets the context for the policy tools. In this way it draws on several other Network documents and provides a frame for future policy documents and toolsets. It is central to other documents.

b) Discussion articles/keynotes: 5 of 11 documents analysed Title

Policy Response: A critical engagement

Author(s)

Carl Bagley and Sophie Ward, UK

Focus on Europe or region/ state/administrative district

Focus on European policy in a 'global policy ensemble of marketisation, managerialism and performativity'

Category of document:

Keynote article for discussion, 2013

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

The nature of policy The relationship of policy to education reform

Objectives

The article aims to explore the concept of policy, and to consider what is meant by policy response.

Degree to which objectives are achieved

The article concisely sets out what selected leading commentators have concluded about the complex

49

nature of policy and policy response open to school leaders. It usefully lists Ten Policy Pronouncements, which provide an overview of the context within which policy is made and received. The final contribution is reassurance that, although they may be constrained by it, school leaders are able to shape and sometimes set policy aside. Overall, a clear and challenging framework for policy implementation and response is achieved.

Connectivity to other EPNoSL documents

This is a foundational document within EPNoSL, stimulating partners to consider more deeply the phenomenon of policy with which the Network is tasked to engage. It balances a critical view with positive possibilities. As such, it provides a primary and essential basis for the policy engagement strategies that follow and build throughout the life of EPNoSL.

Title

Democratic Distributed School Leadership: Ethics, Responsibility and Authority

Author(s)

Olof Johansson, Sweden

Focus on Europe or region/ state/administrative district

Scandinavian countries

Category of document:

Keynote article for discussion, 2013

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

The understanding of and justification for democratic leadership and its relationship to distributed leadership.

Objectives

To 'make the case for democratic leadership processes that include distributed leadership in schools as a practical necessity for the current educational context' in 'conditions of social ferment and diversity'.

Degree to which objectives are achieved

The article refers to a range of concepts including authentic leadership, distributed leadership, democratic leadership and social justice, which are concerns of school leaders within the EPNoSL project. The emphasis is on a process of dialogue and

50

negotiation to reach a position that is accepted and helpful to leadership. The article successfully establishes a key role for ethics in inclusive discussion to embrace the expectations of the state and local communities. The process discusses references to various leadership theories in a fluid way, reflecting some of the ambiguities and uncertainties in the conceptual field.

Connectivity to other EPNoSL documents

The establishment of ethics and values as the basis for discussion is an important foundation underpinning much e later work of EPNoSL in learning for equity.

Title

Quality Management in Schools

Author(s)

Hasso Kukemelk, Estonia

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Keynote article for discussion, 2012

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Objectives

The nature of quality systems The involvement of all staff in implementation The appropriateness of quality systems to schools To consider 'the validity of the processes and the principles used in implementing quality systems in schools'.

Degree to which objectives are achieved

The article raises several questions in relation to the implementation of quality systems, most notably whether such processes are appropriate for learning-centred organisations. Inasmuch as it is intended to stimulate discussion, it offers key questions rather than answers. The issue about quality systems provides a background for debate, but no tools to formulate answers.

Connectivity to other EPNoSL documents

Quality systems do not appear important in ongoing EPNoSL work. However, this may be because the following discussion decided against them as an appropriate tool.

51

Title

Beyond the Reach of Leading: Exploring the realm of leadership and learning

Author(s)

Michael Schratz, Austria

Focus on Europe or region/ state/administrative district

Europe with especial reference to Austria

Category of document:

Keynote article for discussion, 2013

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Notions of management and leadership Learning as an experience Relationship of leadership to learning

Objectives

Degree to which objectives are achieved

Connectivity to other EPNoSL documents

52

The article reflects on the plasticity of the term ‘leadership’ and turns to a key focus; that is, 'debates linking leadership with learning'. The paper considers how leadership may be linked to learning, addressing three questions: 1. What is understood by the term ‘leadership’?

2. What is the role of learning in schools?

3. What is the connection between leadership and learning?

Sets the case for a more organic, socially sensitive and relationship-based approach to the complex process of achieving change in schools, summarised as 'next’ rather than ‘best’ practice. In essence it sets out a notion of leadership working though teachers and teaching, and with leadership embedded much more widely.

In challenging functionalist and technicist notions of leadership, the article lays a foundation for a more sophisticated understanding of how change might be achieved in schools and links to future EPNoSL work on distributed leadership and an emphasis on learning. As such, it is foundational.

Title

Diverse Perspectives and Hopes on Autonomy in School Leadership

Author(s)

Leif Moos, Denmark

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Keynote article for discussion, 2013

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Models of autonomy

Objectives

To stimulate discussion on models of autonomy and their relationship with equity in education

Degree to which objectives are achieved

The article challenges over-simplified and monolithic concepts of autonomy. It raises questions not only about the nature of autonomy but its relationship to equity. The questions are not answered, but it acts as a stimulus for discussion. Inasmuch as it is intended to do so, it offers pertinent questions. The discussion provides a background for discussion and limited tools to formulate answers.

Connectivity to other EPNoSL documents

Autonomy persists as a key EPNoSL concept, linking school leaders' work with policy implementation. This article provides an initial stimulus and some relevant questions to pursue, considered somewhat in ongoing work.

53

c) Empirical Studies 5/11 documents Title

Distributed Leadership in Practice: A descriptive analysis of distributed leadership in European schools

Author(s)

Ton Duif, Chris Harrison, Nicole Van Dartel, Dennis Sinyolo, European School Heads Association (ESHA)

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Empirical/context setting

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Distributed leadership

Objectives



To collect data on the extent of distributed leadership in schools



To identify and describe variations in practice by examining school-related factors, external imperatives and the personal responses of study participants.

Degree to which objectives are achieved

European School Heads Association (ESHA)

ESHA published a position paper in 2013 outlining its views on DL. The paper provides a clear summary of this position and takes a positive view of this form of leadership, because it: 1.

2.

3.

4.

Is a pragmatic response to workload Improves the quality of education in schools Focuses on interaction and practice (Spillane) Promotes reciprocal and collective responsibility and accountability.

A number of approach to DL are outlined in the first section of the paper, based on the work of Bennett; Elmore; NCSL; and the Hay Group; and the idea of a learning organisation. Seven dimensions of DL are identified (p. 8).

54

Sample size The sample size was large, with a total of 1534 respondents. This represented 10% of the full population (membership of ESHA). All the sample limitations are detailed in the document. Although some incomplete questionnaires reduced the sample to 1088, the composition of the sample mainly comprised school leaders (76%) with only 11% of teachers in the sample and low numbers of responses from eastern Europe.

Patterns in response rate led to a focus on eight countries from the full group: England, France, Italy, Netherlands, Norway, Scotland, Sweden and Spain. In addition, the use of the Eurydice education organisational classification (2013) – single structure, common core and differentiated structure – facilitated a double classification of the sample.

Findings All objectives were successfully achieved, with the caveat that it is a picture of only eight countries. Some clear findings emerge on variance in practice and factors contributing to variable views in relation to DL. Position in the school and length of service were identified as highly influential factors, with significant differences between teachers’ views and leaders’ views in a number of areas directly relating to what could be viewed as spaces for the distribution of leadership.

These included the degree of power that teachers have to make/influence decisions, their perceived impact on vision, the extent to which mistakes are viewed as learning, and the degree of collaboration and cooperation. This would suggest that, while leaders may feel that leadership is distributed, teachers do not experience this in their practice to the same degree.

There was more agreement between teachers and leaders on the levels of shared decision making and on levels of responsibility and accountability. However, this was conditional. In summary, the data point to a persistence of residual top-down unequal models of participation and leadership in schools.

Geographical classification has a strong effect and so, too, does education classification (structure). In all three

55

classifications the countries of northern Europe have the strongest perception of DL. Patterns of governance/ ownership also impact, with state-employed teachers viewing DL as less prevalent. School size also influence views, with larger schools indicating lower scores in the levels of DL, but this correlates highly with the sector.

What is provided here is a clear picture of the factors that influence levels/degrees of leadership distribution in schools. Consideration of these as models of leadership in the Network can enrich the understanding of leadership that emerges. The complexity of context is (once again) explicated very strongly and provides a clear direction for the type of dialogue needed in respect of all forms of leadership.

Connectivity to other EPNoSL documents

There are strong links between the focus of this paper and other EPNoSL documents. It develops and refines the focus of the country background reports by homing in on DL. The choice of the focus on DL is linked to much of the work on EPNoSL on this form of leadership. It also adds to the knowledge base and the understanding within the Network of DL in practice. The number of documents developed in the Network with a DL focus is considerable, and it is essential to have the views of ESHA, a key stakeholder in the field. We are provided with a clear summary of the views of ESHA on what form of DL they prefer and how practice in eight countries relates to this framework.

Title

Distributed Leadership for Equity and Learning (Germany)

Author(s)

NQL

Focus on Europe or region/ state/administrative district Category of document:

National context Regional focus – Lower Saxony

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool 56

Empirical research report

Major theme(s)

Patterns of DL in Germany using an adaptation of the Finnish questionnaire on DL and equity (previous review).

Objectives

A brief review of the German context for DL and equity The identification of the current practice of distributed leadership in Lower Saxony Sample included deputy heads and heads of departments

Degree to which objectives are achieved

Clear details emerge in relation to the differences between contexts where there are strongly hierarchical structures, where team leadership is difficult/nonexistent. It is possible to draw up a clear continuum of practices, from structures that do not support distribution to those that do.

A limited or emergent sense of DL is identified, where a lack of resources for providing additional support for students has a clear impact on supporting more equitable outcomes. An interesting point emerges in relation to the manner in which responsibility for pedagogical leadership is distributed; this, too, has implications for the equity of student outcomes. Only a selection of the data is reported here and what is being examined is a more top-down, delegated model of D leadership, with the head being viewed as the provider of DL structures. Evidence of bottom-up leadership practice is not reported. Evidence of the impact of structures to support more collaborative forms of leadership emerges from the study.

Connectivity to other EPNoSL documents

This goes into more detail in one specific context – Germany. It is useful as it was not part of the ESHA report, so we have an additional view of DL in practice.

It also fits with the equity focus on EPNoSL and is one of the main empirical studies that explores DL for a specific purpose, in this case equity. This study also supports and develops the country background reports by providing a more nuanced perspective on different aspects of leadership in German schools and drawing correlations between structures and so on. The main types of leadership are indefinable. This provides additional insight into the national picture in relation to the

57

complexity of school types and structures by providing an intra-national overview of practice. In the context of the overview study (Duif et al.), this study does not include school leaders in the sample, but focuses on senior staff.

Title

Distributed Leadership and Social Justice

Author(s)

Mika Risku and Meng Tian

Focus on Europe or region/ state/administrative district

National focus, Finland

Category of document:

Empirical research report

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Distributed leadership and social equity

Objectives

To review how studies depict DL in Finland:

Degree to which objectives are achieved

1. to synthesise the findings on DL drawing on a national leadership survey 2. to conduct a case study on defining social justice and leadership.

The meta analysis of four studies of leadership in Finland over up to 15 years is a useful piece of work providing a nuanced sense of leadership in Finnish schools.

The studies draw correlations between the broader society structures and those that prevail more locally, including in schools, and point out that Finland had changed from a system-oriented and highly centralised administration to a more fluid, collaborative model. This led to enhancing local control over education, among other changes. There was an intensification of the work of stakeholders, most especially superintendents and leaders, with the result that, in practice, the outcome was the distribution of tasks.

There are issues relating to school size, with a move

58

towards larger schools and a view that this may provide a better space for distribution. The findings of Duif et al. above would suggest that this does not always follow.

Issue emerge for creating the context for DL, providing the resources and the imperatives for this to happen.

Where DL is in place, it again seem to be a pragmatic response to workload. The most recent study in Finland, reviewed here, indicates that DL in practice is not well developed and that support is needed in terms of CPD and additional resources to move the system in this direction. The paper concludes with a strong commitment, by the authors, to the idea of distributed leadership, finishing with a series of recommendations to move Finland in the direction of a much more democratic version of DL than that currently in place.

The links between DL and social justice are not fully explored. How Finland manages issues relating to SJ are detailed. It is most appropriate that the authors do not make claims to how these two ideas are interrelated, because of the lack of evidence of DL in Finnish schools.

Connectivity to other EPNoSL documents

The context-specific issue arises again in this study and thus links all EPNoSL work, which has to take account of varied contexts in developing dialogue and especially in recommendations promoting models of leadership. The paper contributes to a more nuanced understanding of the Finnish system. In this way it complements work on the background reports and helps inform the dialogue in the Network regarding models of leadership.

Title

Distributed Leadership for Equity and Learning

Author(s)

N贸ra R茅vai and Mate Schnellbach

Focus on Europe or region/ state/administrative district

National, Hungary

Category of document:

Empirical research report

context setting literature review keynote stimulus paper empirical research report 59

analytical framework practitioner tool Major theme(s) Objectives

How distributed leadership supports equity and learning To provide and overview DL practice in Hungarian schools To examine how forms of DL impact equity and learning generally an in the Springboard case study

Degree to which objectives are achieved

Two data sources were used to support Objective 2 – a large national empirical study and a study focusing on leadership practice in a small sample of schools with regards to inclusion.

The national review clearly indicates that the formal structures, including the legal systems, create a good basis for DL. Levels of autonomy are high and the review points to a strongly positive view of how dl could improve Hungarian schools. The term ‘DL’, however, is not widely in use and there has been no research done on DL in this context.

Leaders’ aims and institutional aims were conflated – this is a limitation, as the many ways in which context mediates leaders’ own personal values do not always match; nonetheless they may be related and, as long as results are cognisant of this, the findings are useful.

The Springboard case study was helpful in providing an in-depth look at leadership practice, pedagogy and outcome links. Some features of more dispersed and shared leadership were evident, once it had become fully established and the culture had developed. A number of features of a distributed leadership-type culture is revealed in the study. The link between these leadership styles and student outcomes is reported, as indicated by a low drop-out rate. A broad range of success criteria for the student group would give a better sense of the impact of this enhancement/ enrichment programme.

Connectivity to other EPNoSL documents

60

This study provides evidence from two large data sources that seeks to draw correlations between DL and equity outcomes. The dataset itself is useful in correlating leadership in any form to outcomes, and consequently is directly related to all of EPNoSL work. It

provides a good evidence base for examining the issue and making stronger claims about this type of leadership. Again, it is worth noting that, while the collaborative context seems to be prevalent in Hungary, there is no specific reference to DL in the Hungarian context. What is being correlated is forms of leadershipparticipation in decision making, collaboration and so on – that map onto a distributed-type leadership model.

The variables examined are indicative of a form of DL, broader than many more limited top-down models.

The findings indicate that there is a positive correlation between DL and student outcomes. There are some interesting results in relation to achieving equity with negative correlation between equity aims and student outcomes because of the type of grouping/pedagogical practice that explicit equity aims lead to (segregationstreaming, setting). This indicates that the indicators related to equity and how to achieve equity need to be broader and more inclusive.

Title

Distributed Leadership and Social Justice

Author(s)

Philip Woods and Amanda Roberts

Focus on Europe or region/ state/administrative district

National Context - a case study school, East England

Category of document:

Empirical research report

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

The extent to which DL impacts on social justice and democratic practices – a case study secondary school

Objectives

To explore how participants – teachers, non-teaching staff, senior leaders and students – make sense of leadership policy and practice.

Degree to which objectives

It meets a range of objectives relating to an examination of leadership in schools and introduces a number of

61

are achieved

concepts or ways of understanding how leadership is distributed or present in schools. Ideas such as multiple distributions and the distribution of aspects of leadership, including influence and initiative, form a useful analytical lens for future studies.

There are some insightful details as to how DL fits the more traditional hierarchy of leadership models. This is useful when taken in the context of where other countries are, in relation to the degrees of readiness for DL approaches to leadership.

The link between DL and social justice outcomes for student is less developed. The need for more democratic models of collaboration, distribution of resources and so on is well argued. A need for schools based on social justice principles is clear, but working to contribute to more socially just outcomes for all students is less clear.

Connectivity to other EPNoSL documents

This is a detailed and in-depth look at the manner in which views on leadership are constructed and how these perspectives can be elicited from participants using a range of different and innovative techniques. The design of this study offers a different way of looking at leadership practice from that of the Network and in this way the study is highly relevant. The effectiveness of the data and techniques used in capturing the nuances of the complexity of leadership provides strong justification for this study. It complements many more quantitative approaches, to provide a sense of what is happening in different countries with respect to leadership practice.

It has a different perspective on DL to other empirical studies that focus on structures and systems that allow DL to develop/provide the context within which it can happen. It also takes a broader, more emergent view of DL, as opposed to a form of delegation or a pragmatic response to the intensification of the role of the head.

62

d) Additional published outcomes: 2 of 4 documents Title

What is ‘Policy’ and ‘Policy Response’? A case study of the implementation of the Leadership Standards for Social Justice in Scotland

Author(s)

Sophie Ward, Carl Bagley, Jacky Lumby Tom Hamilton, Philip Woods, Amanda Roberts

Focus on Europe or region/ state/administrative district

Scotland

Category of document:

Empirical research report, 2014

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

Objectives

Policy Policy response Use of equity standards This paper uses the implementation of Leadership Standards for Social Justice in Scotland, as set out in the General Teaching Council for Scotland policy document, Standards for Leadership and Management (GTCS, 2012) to consider the extent to which the Standards are being resisted or radically reinterpreted, and why. The empirical enquiry sought to answer two questions: Do head teachers’ views on social justice reflect neo-liberal ideology?

Do head teachers’ beliefs about social justice resonate with the managerial solutions to inequity proposed by the wider policy discourse in which these Standards are located?

Degree to which objectives are achieved

63

The paper in some respects poses a challenge to a foundational belief of the EPNoSL project, that school leadership can improve equity in education. The data is somewhat compromised by a low return rate. This reflects a limitation to the EPNoSL project: its time frame inhibits in-depth empirical work. The paper is consequently partially successful in reporting the views of a small group of head teachers. It offers some wider lessons on policy

and policy implementation within a global trend to neoliberal solutions.

Connectivity to other EPNoSL documents

As a published academic article, this paper provides comment on EPNoSL objectives rather than being directly connected to ongoing work. However, the exploration of policy and policy response contributes to ongoing development of this process, as the final phase of the project focuses on just this area.

Title

Beyond Uniformity: On school leadership policy development in Europe. Proceedings of the International Conference ‘The Future of Education’. 4th edn (pp. 379383). Florence, Italy, June 2014.

Author(s)

Kathy Kikis-Papadakis, Andreas Kollias & Pavlos Hatzopoulos.

Focus on Europe or region/ state/administrative district

Europe

Category of document:

Literature review, 2014

context setting literature review keynote stimulus paper empirical research report analytical framework practitioner tool Major theme(s)

The nature and purposes of education leadership as understood within Europe The problems and challenges faced in developing policy in relation to education leadership.

Objectives

The paper provides an overview derived from the work of the EPNoSL and identifies critical policy goals for the promotion of school leadership focused on equity and learning

Degree to which objectives are achieved

The paper usefully coalesces expert reports, ongoing discussion and empirical research to provide a summary of the conclusions of the work of EPNoSL. It provides a definition of leadership and its purpose that has emerged overall from the period of time the Network has

64

functioned. It emphasises the centrality of learning and equity to the development of leadership and related policy.

It responds to the key challenge identified, that of the diversity of systems, culture and students, by stressing the need for measured development of policy, understanding the connectivity of one area of progress with concomitant progression in others. For example, it suggests that advances in allocating autonomy must be accompanied by advances in developing capacity. The paper offers a concise and accurate summary of the conclusions on leadership and related policy emerging from EPNoSL. In doing so it inevitably reflects some of the biases and assumptions embedded in the work of the Network.

Connectivity to other EPNoSL documents

65

This paper is a summation of much of the work of EPNoSL and is a good example of how the diversity of contributions from policymakers and practitioners has been woven into concise and constructive guidance on policy development, which nevertheless generally avoids over-simplistic prescriptions.

10. IMPACT OF OUTPUTS ON POLICY 10.1 Overall Impact of Outputs The outputs from EPNoSL were designed primarily to influence policy, with additional added value as appropriate. However, in many parts of Europe the most realistic method for influencing policy is of necessity indirect, through providing material and activities that include suggestions and stimulation for new policy directions and action. These have been presented to education administrators working with ministries within the Network and beyond, and to practitioners and non-governmental organisations working closely with policy makers. Consequently, two different kinds of influencing process can be discerned with quite different timescales. The first is the direct presentation of potential policy documents or ideas, with the intention that policy makers should adopt or adapt the documents or ideas for direct use. This has a shorter timescale. The second kind of influence attempts to shift the thinking of stakeholder individuals and groups such that influence and support is brought to bear on policymakers to develop policy in particular directions. This has a much longer time scale.

The analysis of EPNoSL outputs therefore considered the degree to which the sequence, nature and overall shape of outputs affected both kinds of influencing process. It also considered the degree to which the diversity within the Network, reflecting that within Europe, was accommodated and put to productive use. Questions that were pursued included the degree to which the overall body of outputs is coherent, whether coherence is a positive or preferable characteristic or not, given European diversity, and whether, of necessity, policy relates to individual and/or regional context only, or whether to some degree it can be relevant across Europe.

10.2 Overall Sequence of Outputs The nature of the documents analysed shows distinct stages in the outputs of the Network: i.

Context setting – establishing a baseline of evidence about the nature of educational leadership in member nation states and the range of foremost ideas and theories that be relevant to shaping policy and practice

ii.

Gathering further evidence to consolidate and develop knowledge and ideas in relation to developing policy

iii.

Developing policy documents and frameworks for influencing policy.

As such, the outputs of the Network show a logical, rational sequence that established a context baseline as a springboard for developing ideas. This was then used to influence policy. Without such carefully constructed foundations, attempts to influence policy might not be well constructed. Each of the stage is considered in further detail.

66

10.2.1 Stage 1 Network members needed to assess the range and adequacy of existing relevant policy and standards in their own state/region, and the kind of beliefs, attitudes and theories prevalent in shaping thinking, practice and policy. National state case reports were analysed to identify key themes, which included: 

Trends and tendencies in external expectations



Translation of external expectations into internal meaning and direction



Understanding and empowering teachers and other staff



Structuring and culturing schools



Working with partners and the external environment



School leaders' recruitment, preparation and development.

Additionally, a synthesis of documents describing national contexts analysed the commonalities and differences across and sometimes within different national systems.

These documents served two purposes; they provided a lens through which the differing systems within Europe were scrutinised and they challenged the adequacy of policy in each context to support effective school leadership. While this section of the report on Stage i has so far referred to documents, it is clear that the webinars, with their supporting online discussion and papers, also played a part in evaluating the current state of policy and which areas of policy were most important for the future.

Inevitably, given the extent of variation within so diverse a set of countries, frameworks cannot encompass all perspectives and differences. Despite the magnitude of the task, the cumulative effect of the documents in Stage i was to coalesce the Network, and to challenge members to think more deeply about policy and practice with a wider span. The comparison with other parts of Europe that each Network member was enabled to make, considering similarities and dissimilarities, in some cases led to consideration of what is to be done locally and what might be done more widely by mutual learning and cooperation in Europe.

The key overview frameworks and the documents contributing to their formulation achieved a key objective of EPNoSL, which was to stimulate a critical appraisal of the adequacy of existing policy and to provide a foundation for its further development. The documents also acted as an aid to pinpointing where further empirical data may be required.

10.2.2 Stage ii Stage i identified a range of themes and policy areas that might be critical foci for policy development. Stage ii used a range of different media and papers to stimulate engagement in more detail with the areas identified. Specifically, Network members were supported to consider the nature of policy and the processes involved in influencing it. Empirical study, such as that undertaken in Scotland, provided an illustration of the challenges in implementing policy change. While a range of foci was evident in papers in Stage i, for example, quality, autonomy, the relationship of leadership to learning, in Phase ii 67

consideration of distributed leadership became primary. However, distributed leadership as a model of leadership practice is understood differently in the varied national contexts.

Consequently, the theory served both to support deeper probing of the nature of school leadership and also to obscure to some extent the very different understandings of what this means for practice and for valued outcomes, such as learning and greater equity. Overall, the outputs in Stage ii cumulatively create a perception of distributed leadership as a preferred model of practice. The cultural fit between distributed leadership, variously conceived, and the range of regions and nation states varies. Some consideration of democratic leadership in this stage offered alternative ways of thinking. In summary, Stage ii encompassed empirical and conceptual thinking about leadership in practice and, to a lesser extent, the process of policy shift and implementation. An apparently unifying factor was distributed leadership as a normative model of practice, though the degree to which it is commonly understood and practised may be somewhat deceptive. Nevertheless, the empirical work, communication and shared thinking drove forward deep consideration of how leadership in schools might be more effective in the varying contexts of Europe.

10.2.3 Stage iii This was an essential stage in the work of the EPNoSL Network insofar as it was here that the different deliberations of the Network were consolidated into key areas for policy development through the use of bespoke tool sets. These were designed to encourage policy makers and other stakeholders to reflect on the constituents of the policy development and implementation process. The quality of the toolset have a direct bearing on the quality of the outcome of the policy deliberations. A number of documents contributed to the work that partners undertook in this critical part of the project. It is clear from a review of these documents that the impact of the discussions and deliberations that have taken place in the various Network activities have been carefully considered in thinking about and developing the tool sets.

The degree of awareness of the contextual differences between and within the different countries provided a fundamental principle for the Network in the design phase. The awareness of complexities in establishing and implementing coherent and comprehensive school leadership policies, which consequently have critical implications for the development of a Common European Set of Indicators on policy development on School Leadership in Europe underpinned the conceptualisation of the toolsets and the manner in which they were used in a selective way not only to promote policy development itself but to ensure that the process was deliberative and comprehensive.

The premise that the tool set would go beyond one-size-fits-all solutions to more placebased solutions that are context-sensitive and take into account the existing state of school leadership policy development in each EU Member State, as well as their specific needs, constraints and potentials, provided a very strong basis for meaningful change within and between countries. The review of current leadership practice identified key areas for development in the areas of school autonomy, distributed leadership, accountability, policy responses, and preparation and continuing professional development of school leaders. The recommendations for each of these areas were ambitious and the commitment to keeping 68

equity and learning, that is, how SL can promote the quality of learning, fairness and inclusion in schools, provided the Network with a unique and potentially transformative platform for development. In addition, the model of policy development that informed the development of the tool set provided for an in-depth, qualitative and nuanced way of examining contexts and developing policy.

Assessing the overall impact of this work in different contexts will take some time. This type of change is slow and often involves a process of reculturing at micro, meso and micro levels. It is argued here that the fundamentals are in place for positive outcomes to the process. It is important for future work of this kind to ensure that how the tool sets are used and the impact of this use are monitored and that key lessons for this type of collaborative work are explicated.

69

11. CONCLUDING COMMENTS: SECTION TWO The central purpose of EPNoSL was to support the development of policy in Europe relevant to school leadership. This evaluation of the contents, that is, the documents and other outputs such as webinars, concluded that they are generally of high quality and are appropriate to supporting such a process. The policy tools and summaries that are built on the cumulative work of the Network provided a nuanced understanding of the policy process. They also successfully established learning and equity as the core goals of school leadership. The resulting models and recommendations for policy embody depth of thinking and considered responses to relevant issues, and are conceptually thought through. It is hoped that those making use of the outputs of the EPNoSL will apply them in ways that exploit their range and complexity.

EPNoSL outputs have potentially impacted to shape the thinking of Network members, some of whom are engaged with shaping policy. The majority are engaged with influencing policy; the extent to which their influence has had an impact on policy is not evident by means of scrutinising the contents and so the impact on policy by both contents and activities is considered in section three through constructing case studies within regions and nations to explore the nature and extent of EPNoSL’s impact on policy.

70

12. SECTION THREE: THE IMPACT OF EPNOSL'S WORK ON POLICY RELATING TO SCHOOL LEADERSHIP 12.1 Approach to Evaluation This section focuses on the transformation of the content of EPNoSL for policy purposes. In order to understand the nature and degree of the ways in which it impacted on policy, it was necessary to follow the trail of events and results within specified contexts. Consequently, six case examples were constructed, each based in a nation or region in a different area of Europe:      

Estonia Finland Germany Italy Portugal Scotland.

In each case, an interview was conducted with a key member of the EPNoSL Network to construct a picture of the sequence of events and their effects. Questions were posed on topics that included: 1.

2.

3.

4.

5.

6.

7.

key features of the national system impacting the policy area for development description of the impact of EPNoSL on the policy area key stakeholders involved challenges identified evidence of achieved outcomes expected outcomes overview of the policy development process.

The full record of the responses from the interviews in each case are detailed in Section 17.

Additionally, consideration was given the impact of the policy tool sets constructed by EPNoSL. An email request for evidence of use of the toolsets and any impact was sent to all Network members. Only one response was received, and it may be that it is too early to evaluate their impact, given their relatively recent publication on the EPNoSL platform and in hard copy.

The evidence presented in this evaluation report is inevitably selective. Case examples for six countries cannot depict the full range of impact amongst all 27 members. They do, however, provide insight into the ways in which EPNoSL has or has not impacted on policy development, and the factors that make this more or less likely. As such, the evidence is presented as both a summative evaluation of the degree to which EPNoSL is perceived to have had an impact on policy, and a formative evaluation from which lessons can be drawn about the conditions that enable such influence to be most strongly felt.

71

13. EPNoSL CONCEPT OF POLICY Over the life of the EPNoSL project there have been several discussions in relation to what policy entails as it relates to school leadership. Perspectives on policy and the field of policy implementation or the mediation of policy at national, regional and local levels is tightly coupled to each of the contexts represented in the Network, and the systems and structures in place that frame each policy trajectory. In this regard there was a broad understanding that policy solutions must be highly sensitive to the specific context.. This aspect of the Network will be developed further in Section 3.

The field of policy development, as it applied to our work in EPNoSL, was explored by Woods in 2014. This work is helpful in ensuring that Network members can be clear about what is meant by policy within the Network. Woods identifies two main areas for consideration with respect to how the field of public policy would impact on the work of EPNoSL members. The first of these relates to the attempts of governments to shape and define leadership practice. It is worth noting that a lack of direction from governments in relation to this matter in itself is viewed as significant, insofar as it acts as an indicator that the field of leadership is not considered a priority. The second area of policy influence identified by Woods as being relevant relates to the manner in which policy itself as a discursive formation is influenced by a range of often competing discourses that eventually lead to the articulation of a policy outcome privileging certain discourses over others. In this regard, articulating certain ideas, concepts and perspectives on school leadership can result in shifting the manner in which policy is formulated. This is a particularly powerful space, and the case studies included in this report, as well as many other reports on the work of EPNoSL, indicate that EPNoSL has had considerable influence in adding to, and at times changing, the discourse with respect to certain views of what school leadership should look like: EPNoSL has explicitly recognized the importance of understanding public policy not only as laws, regulations, strategies, programmes or projects but also as discourse and as an emergent phenomenon‌ reflections concerning the requirements for policy developments on SL both at the level of laws, programmes, projects etc., and at the wider level of policy discourse(s) and its interrelationships with discourse(s) within and between various stakeholder groups, such as school communities, professional associations, academics and researchers. (Woods 2014, p. 6) The manner in which either of these dimensions of policy intersects with each of the different contexts depends on a number of factors, not least the position or the person/s conveying the EPNoSL messages in each context. The case studies clearly indicate that point of impact and focus of the impact are strongly linked to these roles and positions. This does not mean, however, that the impact is limited to the original sphere of influence. On the contrary, there is evidence that the impact can move towards central policy or towards school policy and practice, depending on who participates in the national policy-making activity. It could be conceived as a pathway whereby the policy/EPNoSL interjection can move towards either or both zones of impact/end points.

72

There is also evidence in some of the case studies of the transfer of discourse and policy developments from practice to the more macro policy level, and from policy level to practice (Estonia, Italy and Finland) in areas such as distributed leadership and the focus on equity. In this way, the overall field is active, and there are indications that additional activity is gathering momentum in some cases.

73

14. THE IMPORTANCE OF CONTEXT There are a number of interrelated dimensions of the context in each of the participating countries that need to be explained in order to understand the varying potential that exists in the different contexts in relation to the work of EPNoSL. Specifically, the elements of context that need to be considered are: 

the needs and priorities of each of the countries with respect to educational policy formation broadly, and the specific stage of development in each of the countries in relation to SL



the overall structure and systems of governance in each of the countries represented in the Network



the level of coherence and quality of collaboration between education partners in each of the countries with respect to patterns of engagement in policy formation, implementation and evaluation



the roles and positions of each of the EPNoSL partners and the extent to which they have the authority to enact change or cause the ideas, concepts and materials developed within EPNoSL to be incorporated into national deliberations and discourse.

In the case of Estonia, relationships and connections between the different stakeholders were well established prior to joining the Network. Consequently that discursive space was already in existence, with the result that participation in EPNoSL enriched these relationships, particularly the links between researchers and the ministry. There were some specific changes to ways of working in Estonia derived from the EPNoSL approach to interagency multi-partner collaboration. This enhanced collaborative space for dialogue has resulted in a much more positive and progressive space to support critical discussions about priorities, the different options available and the preferred solutions.

Portugal’s case was somewhat different, because of the climate of fear relating to school reforms that was prevalent when the Network was initiated. One of the main policy imperatives related to increasing school autonomy, with additional demands for forms of accountability to monitor this reform. School leadership was a particular focus of these reforms, with the result that the work of EPNoSL was a timely intervention into the discourse and provided a reassurance that the demands that were part of the intended reforms in Portugal were similar to changes and reforms happening more broadly in other countries. The EPNoSL partners have been in a position to work closely with the Ministry of Education and Secretary for Educational Administration in order to support schools in responding to and successfully meeting the new demands. This relationship has developed over the life span of the Network, resulting in a situation in which the EPNoSL partners continue to provide a link between the schools and the Ministry; the Ministry formally joined the Network in 2014.

74

In the case of Italy, the policy context is less structured, with the result that developments and initiatives are often linked to particular groups or individuals. Consequently, the role and position of the individual is significant to the successful outcome of a project. The EPNoSL partners from Italy have provided strong direction to their ministry and to schools with respect to the work of the Network. The partners also provide a link between the university, the schools and the ministry and, in this way, where the priorities of the context and the Network deliverables are similar, there is a mechanism in place to ensure that there is an impact on policy and practice. A key aspect of the Italian context is the extent to which Italy is looking outwards to what is happening in other countries and is interested in engaging in international projects in order to enrich its own system. This provides a strong imperative to individuals and groups who are taking leadership roles in different projects. One of the challenges that arises from the Italian context relates to language and the lingua franca of the Network. There is a need for material to be translated into Italian in order for the work of the Network to be accessible at all levels in the system.

Scotland has a well-established policy development structure in relation to education. The most recent initiative has been to give the General Teaching Council of Scotland (GTCS) more autonomy and greater powers with respect to teacher education and leadership development. The GTCS is an integrative force in Scottish education, and there are other collaborative structures in place, including the Scottish Teacher Education Committee (STEC) that comprises the deans and heads of school in universities, Education Scotland, the government and the GTCS. This provides a collaborative space into which it is possible to bring ideas, concepts and materials from Networks such as EPNoSL. While there is only one Scottish Network partner in EPNoSL, the role and position of this partner in the GTCS is such that it is possible to bring the work of EPNoSL to the centre of the Scottish policy development discourse.

Finland, too, has well-established collaborative structures and systems in place to channel the work of the Network. It is these elements of context that result in many of the materials and ideas for empirical work to be integrated into the ongoing review and policy development in Finland. The structures here explicitly link the ministry with regional structures, academics, head teachers and teachers, with the result that there is greater likelihood that changes in emphasis, initiatives and developments have a more holistic allround impact at different levels in the system. A particular contextual factor here relates to the timing of EPNoSL in terms of lifespan. Finland is currently engaged in a process of curriculum reform and renewal. The EPNoSL partner is a key partner in this collaborative reform process. The context exists, therefore, for the work of EPNoSL to feed directly into this review work and subsequently into policy recommendations.

In Germany there is a difference between internal and external school matters with regard to who bears responsibility. The federal Land is responsible for internal matters, including curricula, staff and staffing. Local authorities are responsible, as a rule, for external matters.

In Lower Saxony, for example, the Ministry of Education ensures that the policies set out by the regional parliament are implemented, with the help of the Regional School Authority [RSA] (supervision of schools, employing new teachers, in-service education). The School Inspectorate is responsible for external evaluation; it regularly publishes reports on the 75

development of quality. Two different regions in Germany are represented in EPNoSL. The first to join the Network was the Lower Saxony State Institute for School Quality Development (NLQ), with the State Institute for School and Media Berlin-Brandenburg (LISUM) joining in the second year. Both regions now collaborate extensively in their work in relation to EPNoSL.

76

15. LEVELS OF IMPACT Policy is enacted at a variety of levels, across Europe, and at national, regional, local and organisational levels. Each of these may interact, with resulting policy change flowing in multiple directions and with different timescales. The case examples offer perceptions of each of these phenomena. At this point in the life of EPNoSL, perceptions are of policy change or intended policy change and how such change has been received, rather than evidence of the impact of policy change on learners' experience and outcomes. Although the latter is the ultimate aim of policy development, EPNoSL's brief was to influence and support policy development itself, and it is clear that there has been an impact at each of the levels outlined above.

In Estonia, EPNoSL material and activities contributed to change in policy at national level concerning teachers' workloads and contracts. The tenor of discussion at national level has changed; ‘a cultural shift appears to be in train’, with greater emphasis on human rights across several policy areas. EPNoSL is perceived to have contributed to this change. A national Network within the EPNoSL framework will continue, giving promise of ongoing influence of policy at national level.

In Finland, policy influence by EPNoSL is perceived at cross-European level in a proposal for a European entrepreneurial educational leadership programme. There is also perceived influence though collaboration with Serbia, resulting in a national principals qualifying programme; and, with Hungary, in the development of a new approach to in-service training. At national level, a programme for educational leadership focusing on directors and chairs of education boards and school leaders has been developed. There is also a perceived influence in the Nordic region of Europe in intensified research collaboration among the Nordic group through the collaborations that began in EPNoSL. At national level, a programme for educational leadership focusing on directors and chairs of education boards and school leaders has been developed. Also at national level, EPNoSL is perceived to have influenced policy development with regard to the job description, qualifications and education of principals. Given the input of a variety of players, trade unions, national associations of head teachers, local authorities and two universities, it seems likely that policy may have been influenced at local and organisational levels also.

In Germany, the inclusion of head teachers from the start in the dissemination work of the German EPNoSL partners has resulted in a highly deliberate focus on practice and on including the work of EPNoSL in discourse in schools. Both of Germany’s EPNoSL partners are also linked with the ministries in their respective regions, and this provides the added context that will allow the ideas of the Network to become part of regional discussion in both areas. There are many differences in each of the German regions with respect to the shape of education, and each region retains autonomy to determine its own priorities. It is not possible, therefore, to be certain whether the work of the Network will extend more broadly, nationally.

In Italy, EPNoSL is perceived to have had influence at national level in the establishment of a website and the development of policy on student agency. Through the work of an inspector of schools who is an EPNoSL Network member, there is a perceived development of policy in 77

approaches to leading in schools and through a Masters programme focused on educating principals. Change in practice is suggested that, arguably, could be seen as reflecting organisational policy development. At regional level, a peer-learning event for a group of schools based in Rome was convened and is perceived to be having an impact on policy and practice at organisational level.

In Portugal, the primary impact of EPNoSL has been in the implementation of policy rather than in influencing the nature of policy. It is perceived to have been successful in supporting schools to understand and implement new accountability policies in better ways. It is believed that schools ‘have a more positive environment and orientation to implementing government policy’. In the longer term, this may support policy development at national and organisational levels.

In Scotland, there is no perceived impact on national policy in terms of its direction. EPNoSL is perceived to have played a role in confirming and strengthening the value set on which policy development is based, however, and is credited with contributing to raising the profile of educational leadership at national level. EPNoSL materials have been widely distributed, but there is as yet no evidence that they have influenced policy at organisational level.

Overall, some of the reported impact of EPNoSL relates to practice in leadership and scholarly activity rather than to policy. A strict dividing line is difficult to draw, but much of such change is evident in how leaders approach leadership. In this way, EPNoSL may be seen as influencing the discourse on school leadership, which is in itself a foundational aspect of policy change. Nevertheless, there are perceptions in five of the six case examples of impact on policy at national, regional and organisational levels, and in some cases in partner states in Europe or across Europe. In all cases, the impact is indirect and highly dependent on the national or regional context, as discussed in section three.

78

16. FORMS OF IMPACT 16.1 Variety of Impact It is clear from the case examples that a wide variety of information and materials were used to stimulate debate in the policy-making community. This took place initially in Network, national and regional peer-learning events, and through the online platform. The result has been that the work of EPNoSL is related to the development of policy in a variety of ways. In some cases the work took place in parallel to policy development and was perceived to have had little impact. In most of the case examples, however, the work of EPNoSL was perceived to have had an impact. In some it was an instigator of change; more often it acted to confirm and enrich the existing direction of policy. There are no cases of policy development that can be solely and directly attributed to the Network’s outputs or activities. Rather, the impact can be regarded as indirect and contributory, but may be significant nonetheless.

The impact was perceived to have taken a variety of forms, including influencing:     

underpinning values the education/training of school leaders approaches to leadership collaborative approaches to policy development at a number of levels empowering the policy development community.

16.2 Underpinning Values EPNoSL emphasised the centrality of social justice in the work of school leadership. The term ‘leadership for learning and equity’ became a central concept of its work. Evidence regarding equity or its lack in schools, and insistence on a focus on social justice, are perceived to have had some impact on values in some case examples. For example, Scotland was already decided upon embedding such values in all national policy, but this determination was confirmed and strengthened by the work of EPNoSL: it was very helpful to have the confirmation that Scotland was in line with the mainstream of school leadership and management in Europe. (Scotland case example) In Estonia, the EPNoSL emphasis on social justice translated into a new interest in a humanrights perspective: an impact from EPNoSL is discernible in a greater emphasis on human rights across several policy areas. The thrust of EPNoSL activities has been in the same direction as other initiatives in Estonia and so, though not the sole cause, it has been one of the driving factors towards a different kind of perspective. (Estonia case example) Social justice is also reported as a focus for debate in Portugal and Finland.

79

There is no evidence that EPNoSL has had an extensive and direct impact on the values underpinning policy development, but there is evidence that the Network has confirmed and strengthened a focus on issues of equity and social justice in several countries.

16.3 The Education/Training of School Leaders One of the most common impacts of EPNoSL's work is on policy relating to the education or training of school leaders. In Estonia, EPNoSL materials were used in programmes preparing head teachers at university. In Finland, several new programmes have been developed reflecting developing policy on the education of principals, for example: the first national principals qualifying programme in Serbia; a proposal for a European entrepreneurial educational leadership programme; a new approach to in-service training in Hungary; and a programme for directors of local boards, chairs of education boards and school leaders in Finland. In Italy, EPNoSL materials are used in a Masters programme. In Portugal, a series of in-house and regional workshops have supported implementation of accountability policy and further influenced the general approach to development for leaders: A previous course conceived at the request of the ministry has been set aside.

Instead, a large international seminar is being convened to consider the way forward taking into account the education of leaders within autonomous schools. Several EPNoSL partners from other countries... are invited to present. Previous thinking was that more bureaucratically based speakers would be invited... If a new training system emerges from the international seminar and discussions with the ministry, then the shaping of leadership, policy and practice in Portugal will continue to be influenced strongly by EPNoSL's work. (Portugal case example) In Scotland, the development of national standards for leadership of schools involves consultation with a wide group of stakeholders. ‘EPNoSL was mentioned frequently during the consultation process as a reminder that what was planned for Scotland was in line with what was progressing elsewhere in Europe’ (Scotland case example).

In Germany, work with head teachers is at the core of work in both the regions participating in the Network. Leadership development and influencing what school leaders do is very clear in this context. The now annual conferences set up with the support of EPNoSL are well established and have included themes explored by the Network such as equity, social justice and inclusion. Members of the EPNoSL team have contributed activity to each of these conferences, bringing the ideas and discourses directly to the school leaders in each of the regions. Participation in EPNoSL has also resulted in a review and a revision of the processes of selection and qualification of school leaders, with some amendments having been made to the qualification programme for school leaders in Lower Saxony and a commitment to the development of new criteria for the selection of school leaders for the state of Brandenburg.

Overall, EPNoSL was in some cases an instigator of policy development in relation to the education and training of school leaders and those in related roles, such as board members.

In others, its work was aligned with, and was consequently confirmatory of, the direction of policy rather than impacting on it directly.

80

16.4 Approaches to Leadership In two of the six case examples, Italy and Scotland, there are perceptions that the work of EPNoSL has led to a greater interest in distributed leadership. In Italy, ‘The activity in the Network on distributed leadership (DL) has been widely used here. The related tool has been translated into Italian, and many schools are beginning to incorporate the EPNoSL version of DL into their school plans’ (Italy case study). Similarly, in Scotland relevant material was widely distributed: The notion that leadership is not a charismatic sole leader but dispersed amongst many is also an idea which has moved from EPNoSL to confirm thinking in Scotland and has fitted in well with how Scottish education leadership is developing.

(Scotland case example) The response from one Network member to the Network-wide request about the impact of the various toolsets suggested that the toolset on distributed leadership for equity and learning (DLE) is eliciting interest in Lithuania and Portugal, but as yet there is no evidence of impact.

Some Network members have noted that the concept of distributed leadership is ill defined and may be understood differently by those using the term. Consequently, it is not appropriate to deduce a move towards this way of leading as reflecting a uniform policy development in those taking up the idea. Rather, the work of the Network has provided a trigger for people to consider approaches to leadership, and in some cases to aspire to move towards a more inclusive and less hierarchical approach.

16.5 Collaborative Approaches to Policy Development at a Number of Levels In two of the six case examples, EPNoSL has led to the establishment of cross-national collaborations for the development of policy. In Finland, there have been collaborations with Hungary, and across the Nordic states. Additionally, a pan-European collaboration is considering the education and training of school leaders. In Estonia, collaboration has been established with the Baltic States, again to work together on the development of policy in relation to school leadership: Greater collaboration amongst the Baltic States was established through EPNoSL and this has the potential to continue informing policy development and provide a richer store of ideas and experience on which to draw specifically relevant to the Baltic States. (Estonia case example) In both of these instances there is a perception that the capacity to undertake collaboration in this way is connected to the empowering effect of being a member of the EPNoSL Network.

16.6 Empowering the Policy Development Community In five of the six case examples there are perceptions that participation in the EPNoSL Network was empowering for the individual and for the process of policy development. In 81

Finland, ‘there is a status associated with being the national representative on EPNoSL and an international stamp on the work of the Network that can be used effectively in national contexts’. In Estonia, the Network brought about a sea-change in the relationship between academics, practitioners and the education ministry, such that what was perceived previously as a bureaucratic process of consultation developed into a debate with greater engagement and depth.

The added value offered by EPNoSL was much greater interaction between personnel at the Estonian ministry of education and those involved in research. It brought people together in a way that had not been the case previously. Whereas the extent of consultation had been to look at preliminary papers, now initiatives from different groups are sent to the ministry; and collaboration has led to a search for different kinds of solutions (Estonia case example).

The process of participation was described as educative; there was awareness of the issues faced in other countries, of the solutions required in response and of the processes needed to reach such solutions.

Similarly, in Portugal, ‘Membership of the EPNoSL Network lent strength and credibility to the Portuguese partners’ (Portugal case example). This resulted in Portugal’s EPNoSL Network members visiting many schools in order to act as a bridge between education ministry and practitioners.

In Scotland, the collaboration in the EPNoSL Network of individuals from a range of nations highlighted its value as a vehicle to raise awareness of different approaches throughout Europe, and to learn of different ways of working.

In the two German regions represented in the Network, the links to their ministries provided a context whereby the work of EPNoSL could impact on policy directly. The establishment of a forum to work directly with head teachers is particularly notable, and provides for a powerful space in which the EPNoSL themes can be carried forward.

In summary, membership of EPNoSL lent authority to individual members through its status as a body funded by the European Commission; it also empowered members personally and collaboratively through an educative process, drawing on the knowledge, skills and experience available within the Network.

16.7 Using the EPNoSL Toolkits A number of toolkits were devised by the Network. These focused on:         82

autonomy accountability distributed leadership educating school leaders teacher leadership promoting collaboration. policy response policy assessment.

The purpose of the toolkits is ‘to provide policymakers, school authorities, schools, researchers and leadership training institutes with the tools to reflect upon, identify challenges and prioritise areas of policy action to support and enhance school leadership for equity and learning’ (EPNoSL, 2015: 135). It is too early to evaluate the impact of the toolsets, which have been introduced via PLA workshops and made available online only in May 2015. Nevertheless, initial responses suggest that the toolsets are considered to have potential value in impacting on policy, but they will need to be contextualised and fleshed out with examples as people work with them in their own setting.

5

EPNoSL (2015) School Leadership for Equity and Learning. The EPNoSL Tool Kit. Crete, EPNoSL.

83

17. CASE EXAMPLES 17. 1 Case 1: The Impact of EPNoSL on Policy: Estonia as a case example The initial stimulus of EPNoSL At the inception of EPNoSL, Estonia had already established communication between different stakeholders concerned with policy development. The added value offered by EPNoSL was much greater interaction between personnel at the ministry of education and those involved in research in education. It brought people together in a way that had not been the case previously. Whereas the extent of consultation had been to look at preliminary papers, now initiatives from different groups are sent to the ministry, and the collaboration has led to a search for different kinds of solutions. The Heads Association was quite active, particularly in relation to key policy issues that had been established before EPNoSL. These were:   

splitting upper and lower secondary schools the nature and involvement of parents’ associations students' rights.

These had been hot topics for some years. EPNoSL created an environment in which it was possible to discuss topics that aroused strong feelings in a more collaborative and less confrontational way. As a consequence, the solutions proposed were more acceptable for all the groups involved. The actions of EPNoSL that specifically led to this were the requirement to join together to collect ideas and solutions to be notified to the Network. Previously the ministry had consulted each group on a one-on-one basis. EPNoSL required group representatives to gather around the same table to present different perspectives and possible solutions. Before, there was antagonism, confrontation and an unwillingness to compromise. The structure of EPNoSL demanded that people work together more closely, and so confrontation lessened. This was the result of human contacts and exposure to a range of differing ideas.

In summary, there were a number of priority policy issues in Estonia, but people had not established a way of working together to address them. EPNoSL required people to come into contact with each other to work on key issues specific to Estonia, and in doing so—in part because of the Networking process itself and in part as a result of the ideas generated through the Network—encouraged a more collaborative way of working between the ministry and other groups.

Impact on policy at national level The ministry representative for education research absorbed many of the ideas and materials of EPNoSL through participation at several events and at personal meetings. This representative used EPNoSL materials to create frameworks to aid the development of his area of responsibility, which is accounting for teachers' workload and contracts. Several seminars were held at the ministry using the fresh ideas and materials that he had acquired through his involvement in EPNoSL.

84

In terms of the three key issues of secondary school structure, parents associations and students' rights, it is too early to identify policy change. There are, however, certain shifts discernible in the focus and tenor of discussion. Estonia is discussing students' rights and responsibilities, teachers' working conditions and their very long working hours. There is more awareness of a human rights perspective as policy is developed. Though it is too early to see any measurable impact on policy, and through policy on the experience and outcomes for staff and students, a cultural shift appears to be in train, the effect of which may not become evident for some time. A measurable effect may not be evident for two to three years.

Minorities have been an issue since 1992 when Estonia became independent. PISA results showed that Estonian-speaking schools were closer to the results achieved by Finland than Russian-speaking schools. Where there was antagonism between the schools there is more dialogue. There has been an influence from EPNoSL in terms of changing confrontation to dialogue.

In summary, an impact from EPNoSL is discernible in a greater emphasis on human rights across several policy areas. The thrust of EPNoSL activities has been in the same direction as other initiatives in Estonia and so, though not the sole cause, it has been one of the driving factors towards a different perspective and ways of working.

It is likely that the Estonian Network will continue to work and that the cultural change— with great emphasis on human rights—is likely to continue, because not all personnel in the ministry will change. It is judged to be unlikely that the new minister would completely obliterate the ongoing cultural change, given that the new understandings established amongst a large number of ministry personnel will continue to influence dialogue.

Impact on practice at school level Changes in practice at ministry level should, in time, have an impact at school level. There are so many factors influencing development practice in schools that it is difficult to identify the impact of one individual factor, such as EPNoSL. For example, minority schools, that is Russian-speaking schools, have improved over time, but this improvement was evident before the era of EPNoSL. It seems likely, however, that attendance at peer-learning activities and exposure to people, ideas and materials is one aspect of the elements driving improved performance in minority schools.

EPNoSL activities and outputs The range of EPNoSL activities and outputs did not impact on policy in Estonia directly, but did so indirectly. The whole process was educative, so that greater awareness grew of the issues faced in other countries and the solutions that had been brought to bear, the ideas that had been used and the processes. Participation at events encouraged thinking in different ways, expanding the range of ideas and perspectives that could be brought to bear on policy.

EPNoSL materials were used in programmes for preparing head teachers at university.

85

The recently published EPNoSL toolsets may have an impact on policy development in the future, but it is difficult to assess at this point. A new parliament has been formed, and the policy emphasis and direction will relate to the decisions of a new ministry. The previous minister was involved with EPNoSL activities, but there is no certainty that the new minister will continue to be so. Decisions by the new minister will be highly influential and could lead to a dislocation between development to date and what happens in the future.

Greater collaboration amongst the Baltic States was established through EPNoSL, and this has the potential to continue informing policy development and providing a richer store of ideas and experience on which to draw to deal with issues specifically relevant to the Baltic States.

17.2 Case 2: The Impact of EPNoSL on Policy: Finland as a case example The initial stimulus of EPNoSL A key feature of the Finnish context is the manner in which different agencies and players— municipalities, school board chairs, superintendents, head teachers—are interconnected in the development of what happens in schools. Academic institutions are connected to policy makers at regional and national level, and are also linked to superintendents and directly to schools. In this way the policy platform in Finland comprises joined-up structures and systems in which dialogue is a very strong pillar. There is a commitment to create dialogues around the state table, and in this way there is a means of getting EPNoSL materials into the conversation. There is a possibility to open up concepts and ideas, and consequently policy reforms and developments tend to be well thought through and widely owned, with the result that the implementation of policy changes is more successful. The ideas and materials discussed and developed within the EPNoSL Network had a landing space in Finland; the structures were in place to incorporate the ideas and concepts examined in the Network into national discourse and developments. In a similar way, EPNoSL attempts to bring together the disparate views and perspectives that feed into policy development.

Impact on policy at national level There are a number of clearly identifiable areas of impact in the Finnish context.

The establishment of the Network has coincided with a period of reform in Finnish education, as Finland is currently engaged in the reform of its national core curriculum as well as focusing on the idea of strategic pedagogical leadership. The national core curriculum reforms to end in 2016 are large-scale, impacting on practice significantly, with direct consequences for children’s experience of school. The current shifts in curriculum with the principle of joy at the core, using the integrative force of phenomena rather than subjects as a way of structuring learning, is a fundamental shift. These deliberations have been on-going for four years—the same length of time as EPNoSL. Many of the topics that were explored in EPNoSL were also deliberated in Finnish reform discussions.

The different EPNoSL events, the material provided and the topics discussed were aligned to these deliberations in Finland. The work in Finland has, likewise, been in alignment with what has been happening with EPNoSL, and the extent to which it provided international 86

perspectives on different aspects of schools leadership was always present in the context of current Finnish reforms. Many of the topics that were problematised in EPNoSL, including equity, distributed leadership, accountability and autonomy, as well as the whole idea of looking at educational policy as the Network does, were relevant to the Finnish national experience. Because of this, the influence of EPNoSL has been significant at a national, provincial and local level.

There is also status associated with being the national representative on EPNoSL and the international stamp on its work that can be used effectively in national contexts. One outcome that relates to this follows on from the PLA in Jyväskylä, where a local branch of EPNoSL was set up under a university-based SIG. Its purpose is to gather together national, regional and local level administrators, and also researchers and practitioners, to focus on key policy areas. EPNoSL ideas and material feed directly into this group.

Impact on practice at school level The Finnish National Board of Education has set a special group the task of reforming the job description, and the qualifications and education of school principals. This group comprises a broad representation of partners, trade unions, national associations of head teachers, local authorities and two universities. The work of EPNoSL has been included in these deliberations. Everything explored in EPNoSL was brought to the table—as a result, there are now national education policy recommendations on exactly what has been discussed in EPNoSL. EPNoSL activities and outputs The impact of EPNoSL in Finland has been highly significant. Although from a formal, more structural perspective the Network has had limited impact, in informal ways it has ‘an incredible impact’. The work of the Network has also given rise to a number of spin-off collaborations and initiatives: a. the establishment of a collaborative activity resulting in the first ever national principals qualifying programme in Serbia b. the development of a proposal for a European entrepreneurial educational leadership programme c. the development of a new approach to in-service training with the Hungarian partner d. an intensified research collaboration among the Nordic group, through the collaborations that began in EPNoSL e. the development of a programme for educational leadership focusing on directors of local boards, chairs of education boards and school leaders.

87

17.3 Case 3: The Impact of EPNoSL on Policy: Germany as a case example The initial stimulus of EPNoSL German involvement in the early stages of EPNoSL was confined to one region—Lower Saxony, with representation from the Lower Saxony State Institute for School Quality Development (NLQ). This organisation is concerned with the professional development of school leaders and has significant experience of working with international/European projects on school leadership. One of the main reasons for NLQ joining the Network was to ensure the sustainability of its previous Comenius Network, Leadership in Education. The outcome of this Comenius project contributed to early EPNoSL work, and it provided the basis for further work within the Network. The German involvement in EPNoSL was later extended to include the State Institute for School and Media Berlin-Brandenburg (LISUM), a professional development agency working with teachers and school leaders.

Impact on policy at national level Because the German system is organised at a state level rather than nationally, the impact of EPNoSL involvement will be discernible at this level only. Consequently, in responding to the questions posed for this report, for each of the states it has been identified where the impact of EPNoSL is most visible. These include: 

 

 

Knowledge exchange arising from many of the themes discussed in the Network, especially equity and learning, preparation and professional development of school leaders, and accountability. Participation in PLAs and online activity facilitated these developments.

Two national conferences drawing on expertise within EPNoSL. Head teachers and other practitioners attended these conferences. The success of the conferences has led to the establishment of an annual conference of head teachers.

Participation in EPNoSL resulted in conducting a case study about the implementation of the Convention on the Rights of Persons with Disabilities, in the context of regional qualification programmes for school leaders and school inspectors.

The EPNoSL Network initiated closer cooperation with other German states, in particular between Lower Saxony and Berlin-Brandenburg.

Some work has been done on the selection and qualification of school leaders, with amendments made to the qualification programme for school leaders, in the case of Lower Saxony, and commitment to the development of new criteria for the selection of school leaders, in the state of Brandenburg.

Impact on practice at school level It is too early to measure the impact of the work of EPNoSL at school level. The level of participation of Network members in conferences, and the roles and profiles of the attendees at these conferences, especially the head teachers, would suggest that the ideas and concepts discussed will be incorporated into the discourse at school level in a short time. How this will impact on practice remains to be seen. The impact of EPNoSL on the recruitment and professional development of school leaders should also be discernible in the near future.

88

EPNoSL activities and outputs The sustainability of the head teacher conference is clear, as in 2015 the State Institute for School and Media Berlin-Brandenburg (LISUM) and the Lower Saxony State Institute for School Quality Development (NLQ), in cooperation with the Heinrich Böll Foundation in Berlin, are organising the third school leadership conference. This year the focus of the conference will be school leaders’ response to policy. Its aim is to show school principals ways of discovering innovative opportunities in the conflict between personal expectations and reality within a given ‘framing’ of their own conditions. It is also interesting to note that this consortium is exploring ways of supporting the conference after the life-span of EPNoSL; the inclusion of the Heinrich Böll Foundation and the Green Party, which are supporting this year’s conference by providing the venue, is a way of sustaining the impact of EPNoSL.

17.4 Case 4: The impact of EPNoSL on Policy: Italy as a case example The initial stimulus of EPNoSL A key contextual factor in this region is the extent to which there is active interest in becoming involved in international projects, and an overall structure that seeks to integrate the outcome of international projects, at the level of both policy and practice in schools. The EPNoSL partners are key players in the way the work of EPNoSL is contextualised to suit the local Italian context. The political structures in Italy are such that those who work in the ministry have some autonomy in the type of work they undertake and the projects they drive. As ministers change frequently, it is often necessary for individual public service personnel to carry out and advocate on particular projects and policy directions. A key principle underpinning the work of many of these ministerial personnel is the need to ensure that the work is directly connected to schools. One of the Italian EPNoSL project partners works as an inspector of schools in the Rome area and, as the work of inspectors here is somewhat loosely defined, the partner is in a position to use her role as an inspector to exert an impact on policy at a central level, while at the same time ensuring that school participation in maximised. She also works as a lecturer on a leadership Masters programme in Roma Tre. This partner recognises the need to have key people to make connections between the work developed in structures such as EPNoSL and the local context. The second Italian partner is a head teacher in one of the schools involved with EPNoSL; thus the Network’s school perspective is closely integrated into school life. In these types of situations, it is vital to identify partners with the ability to negotiate national contexts and position the Network and Network activities firmly within the national context. This may mean that some preparatory work needs to be carried out in advance of finalising the Network membership, to ensure that the people who are in a position to make the connections are involved from the start.

There are some features of the academic world that create challenges to the manner in which academic material develops in the field of education. In Italy, the education world works in a different way from that in other countries, presenting some specific contextual factors affecting how ideas become incorporated into the discourse. Sociology of education, for example, is housed within university sociology departments, whereas education belongs in pedagogy. Consequently, the university structures that carry messages to the education 89

community in school and policy platforms are disparate, with different associations and little crossover in terms of working together or sharing focus. This calls for somebody who can consolidate the academic content of Networks such as EPNoSL and mediate how material is made available to the national audience.

An additional imperative on the need to have someone who can negotiate this context is that relating to the issue of language. One of the chief problems that has to be dealt with in the Italian context is language. Because all of the materials produced by the EPNoSL Network are in English, it is necessary to provide translations of key material in order to make it useful for schools. Skill in English at the level necessary to engage with material of this kind is not widespread, consequently providing documents and material alone is not enough. The gap, particularly linguistically, is too wide and some mediation is needed in terms of translation and making concepts, ideas and materials accessible in a sustainable way. It is clear from the impact of EPNoSL in the Italian context that the mediating processes are effective and well established.

Impact on policy at national level A number of developments supported in a number of different ways by the work of EPNoSL are already identifiable.

A website has been developed with translations of some of the material and summaries of many topics. A summary, in Italian, has been written for all of the articles, with links provided to the English versions. In order to host this work an Italian version of the EPNoSL website has been developed, so it is now possible for anyone who is interested to find information about the project in Italian; this work is on-going.

In 2012 the Italian partners organised a peer-learning event that was directly supported by EPNoSL. This conference drew on an extensive range of scholarship from within and beyond the EPNoSL Network in framing discussions on leadership. The conference proceedings have now been edited into a special edition of one of the top education journals in Italy— Educational Cultural and Psychological Studies. This is a very prestigious and highly regard publication, and it is hoped that the publication will further enhance the dissemination of EPNoSL materials. This initiative is an excellent example of how EPNoSL has instigated and enriched work that was already on-going by providing financial support, specific expertise and written materials based on empirical research and scholarly activity within the Network.

Another key component of the impact of EPNoSL is the manner in which the material is integrated into the main Masters in educational leadership in Roma Tre, one of the most prestigious Masters courses on educational leadership in Italy for the past 11 years, meaning that all the material has a direct impact on leaders’ practice through the course. The tool on distributed leadership is of particular interest in the Italian context and one that, to date, had not been well developed in Italy. Because the Masters has such a reputation, the material covered will have a strong impact in future.

90

Impact on practice at school level There is a Network of 30 schools in central Rome and these have been working for some years on a project entitled the Tony Blair project. As a result of this activity the schools are deeply interested in intercultural and international dialogue and other activities, and are open to work of this nature. The schools have also become highly active in the EPNoSL project. A key part of the EPNoSL project relates to providing an opportunity to introduce some ideas on leadership into the discourse in Italian schools. Part of this is captured in the publication of the peer-learning conference, while part relates directly to the work of the partner in encouraging schools to engage with EPNoSL’s work. The activity in the Network on distributed leadership (DL) has been widely used here. The related tool has been translated into Italian, and many schools are beginning to incorporate the EPNoSL version of DL into their school plans.

The final area of impact relates to work that is on-going on student leadership. This is a research project now linked to EPNoSL and currently being launched in schools. Members of the EPNoSL Network were instrumental in the design and development of this piece of research—a film recently been shot in Italy and most successfully used with students. This work was presented in Nice, at the PLA in September 2014. Essentially, it is about how we create and develop more opportunities for student agency. Its outcomes are already making a strong impact, as it is connected with the work of an enormous audience of teachers through the use of clusters of three or four schools across the country. It is designed to be highly interactive from the student perspective, and weekly student blogs are built into the work that schools undertake with the material.

EPNoSL activities and outputs There have been a number of solid outcomes from Italy’s involvement in the Network. The conferences, a special edition of a prestigious national journal, the Italian version of the website and the translation of materials are all evidence of a high level of commitment to the project and a strong effort to ensure that the outcomes of the Network successfully impact on policy and practice. In some ways, particularly in the case of the initiative on student leadership, the work is ground-breaking and a significant outcome of EPNoSL in Italy.

17.5 Case 5: The Impact of EPNoSL on Policy: Portugal as a case example The initial stimulus of EPNoSL In 2012, the first year of EPNoSL, a literature review was published and discussed at the first seminar of the Portuguese national Network. A total of 120 teachers responded to the invitation to attend the seminar, involving staff from university and Masters students, as well as staff from Lisbon area schools invited through teacher training centres. The ‘state-ofthe-art’ literature review had an impact on participants. People realised that policy developments in relation to school leadership that were taking place in Portugal were also unfolding all over Europe. At that time, the national policies and the related demands from 91

central government were feared by schools in Portugal. There was anxiety amongst school leaders. EPNoSL partners benefited, in that schools sought the support of Ana Paula Silva, Escola Superior de Educação Almeida Garrett (Lusophone group), and Maria do Carmo Clímaco, Universidade Lusófona, who were the Portuguese partners in the EPNoSL Network.

These Network members went into schools to answer questions and provide support in addressing issues related to policy, including:    

school evaluation school improvement social justice accountability.

The work took the format of workshops devoted to the themes of:          

school leadership and related issues distributed leadership autonomy knowledge mobilisation (theory and practice; scholarship and professional practice) decision making and action improvement and context teamwork pedagogical supervision organisational supervision equity, learning and social justice.

This was a time of perceived crisis, again the subject of discussion within schools.

Membership of the EPNoSL Network lent strength and credibility to Portuguese partners, and were invited into schools and able to answer questions to allay the anxiety of many. For the four years of the Network to date, there has been the valuable opportunity of going into 200 schools involved to work with them to reflect on the challenges faced and to move away from anxiety towards a more positive emotional environment.

In 2013 the Education Inspectorate (IGE), having notice of the work being developed with schools within the scope of the EPNoSL National Network, promoted a meeting with respective partners. It asked them to prepare a presentation on EPNoSL to integrate into the annual Inspectorate Conference, in Lisbon and in Oporto. The 200 inspectors who were involved in the event were able to address issues such as ‘What was changing in the concepts and implementation of school leadership’, and to discuss how ‘teaching and learning need to be improved’ and ‘school heads retrained’. In 2014 they extended this cooperation to the Secretary for Educational Administration.

Impact on policy at national level Throughout the life of the Network, EPNoSL partners in Portugal have worked closely with the Ministry of Education and the Secretary for Educational Administration. EPNoSL did not change the policy agenda, which was already set at the inception of the Network. The policy of increasing autonomy has been the government’s policy since it came into power, and a keystone of its educational policy. At the initiation of EPNoSL, the Ministry had decided to 92

increase the autonomy of schools, requesting that each school sign an autonomy contract.

This has been achieved with over 220 schools (from a total of 800 organic units). The majority of schools are associated within a cluster under single direction, with one school head for a variable number (6–15) of different grades, from primary to secondary. Some have since renewed their contract. Among these are TEIP schools, that is, schools in deeply deprived contexts that benefit from several kinds of supplementary support, such as from the EU, in consequence of having signed an autonomy contract. However, the majority of schools have not increased their autonomy because of fears of the implications.

The policy on accountability established the terms of accountability, but did not instruct schools on how to implement it. Many schools were concerned at the policy implications, whereby inspectors' evaluation of insufficient performance could lead to penalties such as cutting human and other resources. This, along with a general fear of the control of central government, resulted in a negative orientation to government policy. The existence of the EPNoSL project meant that partners were available to go into schools and training centres to work with them to address the issues and challenges they faced. The use of workshops and other forms of development helped schools to realise that the policy was feasible, and that they could empower themselves through learning and through networking.

EPNoSL partners acted as a bridge that developed a two-way communication process.

Schools are supported to think through how they might adapt their agenda to the Ministry's requirements, and the concerns and issues raised by schools are fed back to it by means of the Network partners. Two-hundred of a total of 800 schools in Portugal worked with EPNoSL partners, usually starting with the schools’ own agenda on organisational improvement, learning to foster interaction and learning with peers.

In 2014 the Portuguese Ministry formally joined the EPNoSL Network. It had perhaps underestimated the challenges in implementing a policy of greater autonomy. EPNoSL partners in Portugal were able to act as a fulcrum to help shape the development of policy, taking into account the concerns of schools and grounding policy adjustments in the knowledge and expertise coming from schools and other stakeholders, such as the directors of teacher-training centres. The central government’s determination to increase autonomy in a top-down way has been moderated.

A concrete example of this is the change in relation to training. A previous course conceived at the request of the Ministry has been set aside. Instead, a large international seminar is now being convened to consider the way forward, taking into account the education of leaders within autonomous schools. Several EPNoSL partners from other countries, including those with voices presenting different perspectives on autonomy, social justice and school leadership training, have been invited to present, although the previous thinking was that more bureaucratically based speakers would be invited. EPNoSL partners in Portugal, therefore, have had direct access to the Ministry for discussion of policy and its implementation. The influence on policy development is evident in the decision to hold an international seminar to further discuss ways forward.

93

In summary, EPNoSL provided initial stimulation and confidence, moreover a platform for the support and development of policy. This was better informed through partners ensuring school-level concerns were taken into account.

Impact on practice at school level Leaders in a hundred schools have engaged with action research as part of a Masters degrees focusing on issues in their schools related to:     

self-evaluation procedures and instruments differentiation school improvement planning and implementation teamwork pedagogical supervision.

The action research methodology has ensured that leaders collect empirical data, which involves devising instruments and the analysis/presentation of data, reflecting on the issues raised and conceiving measures in response to improve their schools. In this way EPNoSL has supported the development of practice in schools in the context of policy implementation. A specific example is leaders who, through this process, have worked out how to undertake a self-evaluation process that was demanded by policy, yet is a novel practice in many schools.

Such work is grounded in the individual school context.

As yet, there is no direct evidence of impact on student experience or outcomes, but there is a perception that schools working with EPNoSL partners now have a more positive environment and orientation to implementing government policy, which may, in the longer term, be of benefit to students.

EPNoSL activities and outputs The initial literature review, which was translated into Portuguese to make it accessible to local schools, was most important. At that time practitioners believed the government was in error and that it was only Portugal acting in the way that it had, which had angered people. The literature review linked policy developments in Portugal to the rest of Europe and a global stage. It presented a diversity of views that opened up the thinking of many in schools and, latterly, in the Network's life, within the Ministry.

Membership of the EPNoSL Network itself provided credibility and confidence in pursuing the activities outlined in this case example. Nevertheless, in itself that would have been insufficient had partners not pursued their work in relation to some important principles:   

schools themselves will always be responsible for undertaking activities school are supported, but never pressed or blamed the quality of support work in workshops and other development activities is axiomatic.

At this point, as the EU funding is drawing to a close, there is optimism that the influence of EPNoSL will continue beyond the life of the funding period. If a new training system emerges from the international seminar and discussions with the Ministry, then the shaping of 94

leadership, policy and practice in Portugal will continue to be influenced strongly by EPNoSL's work.

17.6 Case 6: The Impact of EPNoSL on Policy: Scotland as a case example The initial stimulus of EPNoSL During the initial context-setting phase of EPNoSL's work, policy development was progressing in Scotland as a parallel process. However, the issues emerging from the Network's context-setting emphasised that Scotland was, to some extent, ahead of others in terms of its policy development. It also confirmed to Scottish Network members, gratifyingly, that country was on the same track as many others. The kind of issues emerging from EPNoSL, with a focus on values and social justice, were exactly the kind of issues that were a focus in Scottish policy development. By attending Network meetings it was evident to them that Scotland was already implementing some of the ideas suggested in EPNoSL, but it was most helpful to have this confirmation of being in line with mainstream school leadership and management in Europe.

Impact on policy at national level Scotland legislated to give the General Teaching Council of Scotland (GTCS) independence and greater powers. It was exciting that the national standards and the development of head teachers, which had been under the aegis of the government, came to the GTCS from April 2012, during the early period of EPNoSL. The GTCS was carrying forward work on national standards and on preparation and support for the role of middle leaders and head teachers in parallel with the work of EPNoSL. There have always been values embedded in the Scottish national standards, but they were made much more explicit in terms of what was written into the revised standards. Every set of standards, from those for students about to graduate to those for head teachers, have exactly the same wording on values, thus they are written into the whole profession. The GTCS spoke to all the actors in education, including local authorities, universities, civil servants in the government, Education Scotland and parents. All these groups, therefore, knew about the central focus on values. EPNoSL was mentioned frequently during the consultation process, as a reminder that what was planned for Scotland was in line with what was progressing elsewhere in Europe. Those consulted were aware that EPNoSL Network members were discussing social justice as a vital part of the development of education policy.

Several Network members, particularly from southern European countries, were deeply anxious about the economic situation in their country and its potential negative impact on opportunities for children and young people coming through the education system. This perspective from southern Europe emphasised the necessity to ensure that education caters for all needs, to prevent the exclusion and potential revolt of young people. Such contact with other countries through the EPNoSL Network both confirmed that Scotland was right to make values central and provided a further spur to the work.

95

In summary, Scotland had decided on the direction of education policy at the commencement of EPNoSL. The work of the Network has not changed this direction, but it has provided confirmation and support. The opportunity for Scotland to work with other countries has been valuable. Sometimes the value is in observing directions or practices that are not the way Scotland would choose to go, but that contrast is of value. There are also positive lessons from working with other countries and with individuals, which have been very beneficial.

Impact on practice at school level Educational leadership has a higher profile in Scotland than previously, to which EPNoSL has contributed. The new Scottish government is fully committed to ensuring that all children get the best from their school and is making additional funds available to this end. It is also establishing Masters-level teaching qualifications. EPNoSL's work fits very well with the government's social justice agenda.

EPNoSL activities and outputs The confirmation of social justice as being central, as embedded in EPNoSL activities and outputs, is apparent in the revised Scottish national standards.

EPNoSL outputs have been disseminated:  



EPNoSL newsletters have been circulated to local authorities and universities and the unions.

Rather than holding EPNoSL events as such, communicating the work of the Network has piggy-backed on existing events, through which ideas and materials have percolated. Specific examples of practice in other parts of Europe have been raised for discussion at such events, for example, the work of the Leadership Academy in Austria.

The animation on distributed leadership was sent to universities and local authorities. Feedback from one university principal suggests that it was further distributed to senior managers within the university. The notion that leadership is not a sole charismatic leader but is dispersed amongst many is also an idea that has moved from EPNoSL to confirm thinking in Scotland and has fitted in well with how Scottish education leadership is developing.

The contrast of very different approaches has proved interesting. There are regular discussions between the Scottish EPNoSL partner and civil servants who have been informed about the work of EPNoSL. No one from government has attended a peer-learning activity (PLA); the Scottish College for Educational Leadership has, however, become an associate member of EPNoSL and will be attending the final PLA in 2015. The Scottish Teacher Education Committee (STEC) comprises the deans and heads of school in universities, Education Scotland, the government and the GTCS, and has been informed about the work of EPNoSL at regular meetings. All these channels have led to a general awareness of the work of EPNoSL, rather than it having had direct impact.

96

18. CONCLUDING COMMMENTS: SECTION THREE The evidence presented in this section details case studies from six countries – a sample from the much larger number of members of the EPNoSL Network. This partial and timelimited evidence, nevertheless, is sufficient to suggest that the work of the Network has impacted and is likely to continue to impact on policy in a number of ways. We have therefore categorised the relationship between EPNoSL and the development of policy into a typology of impact: 

In parallel: existing policy development continues unaffected by the work of EPNoSL.



Confirmation: the direction of existing policy development continues, but is strengthened and rendered more confident by the work of EPNoSL.



Adaptation: existing policy development is adjusted or refined in the light of EPNoSL's activities and outputs.



Instigation: new policy is initiated in the light of knowledge or relationships created through EPNoSL.

It would be difficult to conclude whether there are parts of the Network in which policy development is moving in parallel with and remaining unaffected by EPNoSL. The timescale needed for an evaluation of a drip-feed, indirect impact on policy is much longer than the life of the Network, and so impact may yet be achieved where it has not been perceived to date. It is also possible, however, that policy development in some parts of Europe has been and will remain untouched by EPNoSL. There is no evidence of this in the case studies presented in this report, but the evidence is too incomplete to be conclusive. The data from the six case example countries provide evidence of EPNoSL's influence within the policy development process in terms of confirmation, adaptation and instigation elements of the proposed typology.

Achieving and measuring impact on policy is a many-faceted and long-term process. At the conclusion of the life of EPNoSL, it is not possible to achieve a full, summative assessment of the overall impact of the Network on policy, as impact may continue over some years to come. Inasmuch as assessment may be made, bearing in mind the caveats about timescale and the limitations of evidence, EPNoSL has been successful in impacting on policy through changing values, discourse, knowledge, relationships and resources, which in turn have, in some instances, changed both the thinking underpinning policy and the practical means selected to achieve related goals. As such, a cautious conclusion can be reached that EPNoSL has fulfilled its brief to influence policy development concerning school leadership in Europe.

97

19. EPNoSL EVALUATION: OVERALL CONCLUSION 19.1 Introduction The European Commission call EAC/42/2010 focused on establishing a European Network of relevant individuals and organisations to implement priorities for school leadership development. The specified objectives were to be achieved over a period of four years from 2011 to 2015. During this period, three successive Work Plans focused on cumulatively building work to achieve the specified objectives. This work was undertaken by the European Policy Network on School Leadership (EPNoSL).

During the third period of the contract two evaluators, Professor Jacky Lumby and Dr Gerry Mac Ruairc, were appointed to evaluate the effectiveness and efficiency of work processes in relation to the Call’s objectives and, the relevance of EPNoSL contents to support policy development with respect to school leadership. These evaluation outcomes were achieved by deploying three different and complementary evaluation strategies. The first phase of the evaluation used a questionnaire to all of the project partners to establish the effectiveness and efficiency of the way partners have worked together. The findings of this part of the evaluation are reported in Section One of this report. The second section of the report details the peer review that was conducted on a sample of materials that were developed by EPNoSL partners in order to establish the relevance and value of the intellectual activities, the issues explored and products of EPNoSL's work in shaping policy and practice. The third section of this document reports on the findings of six case examples designed to elicit the extent to which participation and involvement in EPNoSL's impacts on policy relating to school leadership in the sample of case countries chosen.

This integrated evaluation document details a number of key outcomes that are identifiable in the different sources examined. These indicate that the impact of the work of EPNoSL is evidenced in a number of different countries and in a number of different contexts. What this evaluation has also revealed is messages for working both with and within Networks.

Explicating these messages is vital if this Network is not only to contribute to policy development on school leadership but add to a great understanding of how Networks operate and how to make the most of the opportunities presented by Networks for enhanced learning outcomes, and better and more sustainable ways of working in a collaborative and mutually beneficial way.

19.2 Project Outcomes The data in the first section of this report indicated that ways of working within the Network, including online, PLA and national events as well as the overall composition of the Network itself, were generally a positive experience. Keynotes and informal discussions were identified as the most positive ways of working. There is considerable evidence in this part of the report of a high level of commitment to the Network among members, as indicated by the level of participation in PLAs and the efforts that have been identified in the report in relation to the dissemination of the outcomes of Network themes among different cohorts. This includes developing materials for the Network, making them available online and translating materials into different languages. As the Network developed in the next 99

phase, this feature of the way members worked together intensified, and intra-Network relationships and alliances continued to be formed and strengthened. In the first phase of the evaluation, while it was too early to identify fully a significant number of demonstrable outcomes on practice, the quality and impact of the initiatives and activities in place or at advanced stages of planning suggested that the Network had the potential to effect ongoing change in the future. Taking a broad overview of the four levels of impact that underpinned the design of this part of the evaluation, the overall positive slant of the ratings in the survey and the nature of the majority of comments, there is evidence that participation in the Network provided people with a range of challenging perspectives in relation to the practice of school leadership by exploring ways of undertaking leadership, resulting in orienting the focus of leadership decidedly towards more equitable outcomes.

Over the course of the Network an extensive amount of written material was developed to support the Network and to advance the aims of the project. The evaluators reviewed a sample of this material in order to establish the overall cohesion and cumulative effect of the work as a body, and to evaluate the degree to which it had established a sound foundation to progress thinking about both policy and policy influences within the European context. The nature of the documents analysed shows several distinct stages in the outputs: 

Context setting – establishing a baseline of evidence about the nature of educational leadership in member nation states;



Gathering further evidence to consolidate and develop knowledge and ideas in relation to developing policy;



Developing policy documents and frameworks for influencing policy.

The evaluation revealed the generally high quality of the documents and materials provided within the Network. The degree of awareness of the contextual differences between and within the different countries was clearly established, as was awareness of complexities in establishing and implementing coherent and comprehensive school leadership policies that, consequently, have critical implications for policy development on School Leadership in Europe. The knowledge developed in this phase of the work and the discourse and discussions on this material at PLA events and participation in online events provided the Network with the opportunity to explore comprehensively the core concepts prior to the development of the policy toolsets. The latter have become the main outcome of the final phase of the project and one mechanism by which the impact of the Network will be sustained into the future.

The review of current leadership practice that emerged from the Network activities identified key areas for development in the areas of school autonomy, distributed leadership, accountability, policy responses, and preparation and continuing professional development of school leaders. The recommendations for each of these areas are ambitious.

The commitment to equity and learning, that is, how school leadership can promote the quality of learning, fairness and inclusion in schools, provided the Network with a potentially transformative platform for development. In addition, the model of policy development that

100

informed the development of the toolset provided an in-depth, qualitative and nuanced way of examining contexts and developing policy.

In the first phase of the evaluation it was only possible to identify early signs of impact on policy and practice. Further evidence that became available in the following phases of the evaluation, by means of the case examples, indicated that the ways of looking at leadership as a practice had been influenced by participation in EPNoSL and that the Network had made a significant contribution to the developing discourse on school leadership in a range of contexts. The evidence from the case examples indicates that a wide variety of information and materials were used to stimulate debate in the policy-making community.

The result has been that the work of EPNoSL is related to the development of policy in a variety of ways. In some cases, the work may have taken place in parallel to policy development and had little impact. In the case examples, however, the work of EPNoSL was perceived to have had an impact. In some it was an instigator of change; more often, it acted to confirm and enrich the existing direction of policy. The impact was perceived to have taken a variety of forms, including influencing underpinning values, the education/training of school leaders, approaches to leadership and, collaborative approaches to policy development at a number of levels, empowering the policy development community.

19.3 Working as a Network A key strength in the formation of this Network was the diversity of professional roles that came together to explore the issue of school leadership. This provided a tremendous opportunity for cross-sectoral learning and for building understanding between key partners on policy and practice relating to leadership. Of particular significance was the manner in which this diversity of interests converged in agreement on the need to place equity at the core of the work of the Network. This focus provided the Network with a strong moral purpose and influenced, to differing degrees, much of the work of the Network. The message here for working as a Network is the importance of time spent exploring the core purposes of the Network and consolidating the key messages between partners.

The other main messages relate to time and to having a structure that is flexible and fluid enough for partnerships to develop and evolve in a responsive and authentic way. There is considerable evidence in this evaluation of developments that arose because of the work of EPNoSL. There is also evidence of the value accorded by members to working in a Network in this vital area, predicated on the mutual support and trust of many EPNoSL members. At the final PLA in June 2015 in Crete it was clear from presentations and comments at round table and panel discussions that this dynamic is still ongoing and that the influence of EPNoSL is far from over.

101

APPENDIX 1: LISTING OF EPNoSL DOCUMENTS a) Briefing/review documents 1. EPNoSL, Del. 2.2, 2011-2012, The European Scenery on School Leadership: National case reports, prepared by Dr Kathy Kikis-Papadakis with contributions from the EPNoSL partners 2. EPNoSL, Del. 2.1, 2011-2012, State of the Art Review, Contributors: Peter Earley, Olof Johansson, Andrej Koren, Jacky Lumby, Leif Moos, Gerry Mac Ruairc, Michael Schratz, Philip A. Woods.

3. EPNoSL, Del. 2.3, 2011-2012, Typology of European Space for Improvement on School Leadership Policy and Practice, prepared by Ana Paula Silva with contributions from the EPNoSL partners 4. EPNoSL, Del. 3.1, 2011-2012, Consensus Report on European Policy and Practice on School Leadership: Moving towards improved school performances, prepared by Nóra Révai and Mária Szabó, with contributions from Jacky Lumby, Lejf Moos, Olof Johansson, Tom Hamilton, Gerry Mac Ruairc, Hasso Kukemelk, Peter Earley, Philip A. Woods, Huub Friederichs, Andrej Koren, Fred Verboon, Mikael Risku 5. EPNoSL, Del. 3.2, 2011-2012, National Platforms on Educational Policy on School Leadership, prepared by Nóra Révai and Mária Szabó 6. EPNoSL, Del. 4.2, 2011-2012, Consensus Report on Good Practices: Policy meets practice, prepared by Prof Lejf Moos 7. EPNoSL Del. 2.1. Kollias, A. (ed.) (2013) Critical factors in the discourse on School Leadership from the perspective of equity and learning. http://www.schoolleadership.eu/portal/deliverable/critical-factors-discourseschool-leadership-perspective-equity-and-learning 8. Kollias, A. & Hatzopoulos, P. (Eds) (2013) Briefing Notes on School Leadership Policy Development and Background Papers on School Leadership. Heraklion: EPNoSL. 9. EPNoSL Deliverable 2.1. Kollias, A. (2014) Commonly agreed plan on the content and form of EPNoSL’s policy influence toolset. http://www.schoolleadership.eu/portal/deliverable/commonly-agreed-plancontent-and-form-epnosl%E2%80%99s-policy-influence-toolset 10. Deliverable 5.1 Kollias, A. (2014) SWOT to the construction of policy indicators on school leadership. EPNoSL. http://www.schoolleadership.eu/portal/deliverable/ swot-construction-policy-indicators-school-leadership 11. Ward, S.C., Bagley, C., Woods, P., Lumby, J., Hamilton, T., Roberts, A. (2013) Scoping paper on school leadership and equity. EPNoSL UK Team, Work Package 2. http://www.schoolleadership.eu/sites/default/files/epnosl-scoping-paper-onschool-leadership-equity_3.pdf 103

b) Discussion articles/keynotes 1. Bagley, C. & Ward, S. (2013) Policy response: A critical engagement. Keynote article for discussion, http://www.schoolleadership.eu/sites/default/files/policy-response-acritical-engagement-2013.pdf 2. Earley, P. (2013) Leading and managing change: Why is it so hard to do? Keynote article for discussion, 2013 http://www.schoolleadership.eu/sites/default/files/leading-andmanaging-change-2013.pdf 3. Johansson, O. (2013) Democratic distributed school leadership: Ethics, responsibility and authority. Keynote article for discussion http://www.schoolleadership.eu/sites/default/files/distributed-leadership-2013_7.pdf 4. Koren, A. (2013) Accountabilities for equity and learning. Keynote article for discussion. http://www.schoolleadership.eu/sites/default/files/accountability-2013_1.pdf 5. Kukemelk, H. (2012) Quality management in schools. Keynote article for discussion, http://www.schoolleadership.eu/sites/default/files/quality-management-in-schools2012_0.pdf 6. Lumby, J. (2013) Leading for equality in a changing Europe. Keynote article for discussion, 2013 http://www.schoolleadership.eu/sites/default/files/leading-forequality-in-changing-europe-2013.pdf 7. Mac Ruairc, G. (2013) Including inclusion: Exploring inclusive education for school leadership. Keynote article for discussion http://www.schoolleadership.eu/sites/default/files/exploring-inclusive-education-forschool-leadership-2013_5.pdf 8.

Mac Ruairc, G. (2012) Working with partners and the external environment. Keynote article for discussion

9.

Moos, L. ( 2013) Diverse perspectives and hopes on autonomy in school leadership EPNoSLhttp://www.schoolleadership.eu/sites/default/files/diverse_perspectives_and_ hopes_on_autonomy_in_school_leadership_5.pdf

10. Schratz, M. (2013) Beyond the reach of leading: Exploring the realm of leadership and learning. Keynote article for discussion, 2013 http://www.schoolleadership.eu/ sites/default/files/leadership-for-learning-2013.pdf 11. Woods, P. (2012) Control or autonomy: What opportunities and challenges are being created for school leaders by current trends in the governance of education? Keynote article for discussion. http://www.schoolleadership.eu/sites/default/files/control-orautonomy-2012_4.pdf

104

c) Empirical studies 1.

Cencič, M. & Simčič, B. (2013) Equity and inclusiveness of physical environment of schools in the views of heads of Slovenian Basic Schools. http://www.schoolleadership.eu/sites/default/files/inclusion_article_2013.pdf

2.

Duif, T. Harrison, C. van Dartel, N. & Sinyolo, D. (2013) Distributed leadership in practice: A descriptive analysis of distributed leadership in European schools. http://www.schoolleadership.eu/sites/default/files/esha-etuce_report_on_dl.pdf

3.

Kasprzak, T. (2013) Accountability in the Polish School. Study report. http://www.schoolleadership.eu/sites/default/files/2013_accountability_polish_case_r eport.pdf

4.

Normand, R. (nd) Between Civil Service and Republican Ethics: The statist vision of leadership among French principals. http://www.schoolleadership.eu/sites/default/files/france_accountability_wp4_0.pdf

5.

Révai, N. & Schnellbach, M. (2013) Distributed Leadership for Equity and Learning. Final report http://www.schoolleadership.eu/sites/default/files/hu-final-report-distributedleadership-for-equity-and-learning_2_0.pdf

6.

Risku, M. & Tian, M. (2013) Distributed Leadership and Social Justice Case Finland. Final report http://www.schoolleadership.eu/sites/default/files/fi_final_report_distribute d-leadership-and-social-justice_5.pdf

7.

Ward, S.C., Bagley, C., Hamilton, T., Woods, P., Lumby, J., Roberts, A. (2013) School Leadership and Equity: An examination of policy response in Scotland. http://www.schoolleadership.eu/portal/resource/school-leadership-and-equityexamination-policy-response-scotland#attachments

8.

Woods, P.A. & Roberts, A. (2013) Distributed Leadership and Social Justice. A case study investigation of distributed leadership and the extent to which it promotes social justice and democratic practices. Final report. http://www.schoolleadership.eu/sites/default/files/uk_final_report_distributedleadership-and-social-justice_0.pdf

9.

Woods, P.A. & Roberts, A. (2013) Distributed Leadership and Social Justice (DLSJ): National Review http://www.schoolleadership.eu/sites/default/files/ uk_country_report_dlsj_.pdf

10. The Head Teachers of Educational Institutions in Lithuania about Distributed Leadership. http://www.schoolleadership.eu/sites/default/files/ lt_final_report_dl_0.pdf 11. Distributed Leadership – for Equity and Learning (Germany) http://www.schoolleadership.eu/sites/default/files/de_final_report_distributedleadership-for-equity-and-learning_nlq_0.pdf

105

d) Additional published outcomes resulting from EPNoSL activities 1.

Lumby, J. (in press) How can we understand educational leadership for equity and learning. Sodobna Pedgogika (Contemporary Educational Studies).

2.

Ward, S.C., Bagley, C., Lumby, J., Woods, P., Hamilton, T., Roberts, A. (2014) School leadership for equity: Lessons from the literature. International Journal of Inclusive Education. DOI: 10.1080/13603116.2014.930520

3.

Ward, S.C., Bagley, C., Lumby, J., Hamilton, T., Woods, P., Roberts, A. (in press) What is ‘policy’ and ‘policy response’? A case study of the implementation of the Leadership Standards for Social Justice in Scotland. International Journal of Inclusive Education.

4.

Kikis-Papadakis, K. Kollias, A., Hatzopoulos, P. (2014) Beyond Uniformity: On school leadership policy development in Europe. Proceedings of the International Conference ‘The Future of Education’ 4th edn (pp. 379-383). Florence, Italy, June. http://conference.pixel-online.net/FOE/files/foe/ed0004/FP/0723-SOE474-FP-FOE4.pdf

106

APPENDIX 2: SAMPLE OF EVALUATED DOCUMENTS Category of document

Year published

Title

Editors/authors

Nation state of author(s)

Substantive focus

Geographic focus

1

Briefing/ review

2012

The European Scenery on School Leadership: National case reports

Kathy Kikis-Papadakis, with contributions from the EPNoSL partners

Greece

European context of school leadership and policy and practice

Europe

2

Briefing/ review

2012

Country Brief, Portugal

Portugal

Portuguese context of school leadership and policy and practice

3

Briefing/ review

2012

Country Brief, Latvia

Latvia

Latvian context of school leadership and policy and practice

4

Briefing/ review

2012

State of the Art Review

Peter Earley, Olof Johansson, Andrej Koren, Jacky Lumby, Leif Moos, Gerry Mac Ruairc, Michael Schratz, Philip Woods

Varied

International context of theory of school leadership policy and practice

Europe

5

Briefing/ review

2013

Briefing notes on school leadership policy development and background papers on school leadership.

Andreas Kollias & Pavlos Hatzopoulos

Greece

European context of school leadership and policy and practice

Europe

European Policy Network on School Leadership (EAC/42/2010)

6

Briefing/ review

2014

SWOT to the Construction of Policy Indicators on School Leadership

Andreas Kollias

Greece

Indicators for policy on school leadership

Europe

7

Discussion articles/ keynotes

2013

Policy Response: A critical engagement

Carl Bagley & Sophie Ward

UK

Formulation and implementation of policy

International

8

Discussion articles/ keynotes

2013

Democratic Distributed School Leadership: Ethics, responsibility and authority

Olof Johansson,

Sweden

Distributed leadership theory and practice

9

Discussion articles/ keynotes

2012

Quality Management in Schools

H. Kukemelk

Estonia

Quality in Estonian schools

Estonia

10

Discussion articles/ keynotes

2013

Beyond the Reach of Leading: Exploring the realm of leadership and learning

Michael Schratz,

Austria

11

Discussion articles/ keynotes

2013

Diverse Perspectives and Hopes on Autonomy in School Leadership

Leif Moos

Denmark

Autonomy

Denmark

12

Empirical studies

2013

Distributed Leadership and Social Justice Case, Finland. Final report

M. Mika Risku & Meng Tian

Finland

Distributed leadership theory and practice

Finland

13

Empirical

2013

Distributed Leadership in Practice: A descriptive analysis of

Ton Duif, Chris Harrison, Nicole van Dartel, Dennis

Varied

Distributed leadership

Europe

108

European Policy Network on School Leadership (EAC/42/2010)

studies

distributed leadership in European schools

Sinyolo

theory and practice

14

Empirical studies

2013

Accountability in the Polish School. Study report

T. Kasprzak

Poland

Poland

15

Empirical studies

nd

Between Civil Service and Republican Ethics: The statist vision of leadership among French principals

R. Normand

France

France

16

Empirical studies

?

The Head Teachers of Educational Institutions in Lithuania about Distributed Leadership

Lithuania

Lithuania

17

Additional documents

2014

What is ‘Policy’ and ‘Policy Response’? A case study of the implementation of the Leadership Standards for Social Justice in Scotland

Sophie Ward, Carl Bagley, Jacky Lumby Tom Hamilton, Philip Woods, Amanda Roberts

UK

Scotland

18

Additional documents

2014

Beyond Uniformity: On School Leadership Policy Development in Europe

Kathy Kikis-Papadakis, Andreas Kollias, Pavlos Hatzopoulos

Greece

Europe

19

Additional documents

2014

Commonly Agreed Plan on the Content and Form of EPNoSL’s Policy Influence Toolset. EPNoSL

Andreas Kollias

109

European Policy Network on School Leadership (EAC/42/2010)

Deliverable 2.1

110