S U S TA I N I N G I N S T I T U T I O N A L EFFECTIVENESS: After the PRT Process - Fall 2018 Themes | Conclusions | Recommendations
THIS REPORT is the second in a series of periodic evaluations assessing the sustained impact of the Partnership Resource Team (PRT) Process on both the participating Client Institutions and volunteer PRT members. The PRT component of the Institutional Effectiveness Partnership Initiative (IEPI) provides technical assistance at no cost for those institutions identified as needing support. Prospective Client Institutions submit a letter of interest, explaining how the PRTs could help improve institutional effectiveness. Each Client Institution develops “Areas of Focus� (AOFs) and addresses them through an Innovation and Effectiveness Plan (I&EP). Through a series of three visits, the PRTs facilitate institutional discussion of the issues, provide ideas for improvement and best practices for implementation, help the Client Institution draft the I&EP, and make suggestions on how to improve implementation and sustain long-term progress.
METHODOLOGY CLIENT INSTITUTIONS REPRESENTATIVES OF CLIENT INSTITUTIONS that had completed the final visit of their PRT process by November 15, 2017 (N= 43) were asked to rate progress on each of their identified AOFs that was at least partially attributable to PRT assistance, from the
CLIENT INSTITUTION EVALUATION QUESTIONS
}} Please rate the overall progress, if any, that your institution has made since
the beginning of the PRT process that is at least partially attributable to that process, with respect to your Area of Focus. }} What aspects of the progress noted above (within or beyond your Areas of Focus), if any, were unanticipated or surprising to you? }} If your institution had NOT received PRT services, in your opinion, how much progress would it have made by now in those Areas of Focus and other structures and processes? beginning of the process though the present. They were also asked to identify major factors in the PRT process or at their institutions that helped them make progress, or limited their progress, on the AOFs, as well as any effects
that the PRT process had had on institutional structures and processes beyond the AOFs. Client Institutions were further asked about any unanticipated or surprising aspects of progress they had made, and to rate how likely progress on the AOFs would have been had the institutions not sought PRT assistance. Sixtytwo respondents provided information through the survey, representing of the 32 of the 43 institutions provided.
PRT MEMBERS PRT MEMBERS who had served the Client Institutions who completed the PRT process by November 15, 2017 were asked about the effect their participation as a PRT member had had on their own professional growth, and on their network of colleagues and resources. They were also asked whether their home institutions had applied any practices they had learned from their PRT experience. PRT members who had served on more than one PRT were asked to report the effects of that additional service on their own
Robert Pacheco, Ed.D., External Evaluator | bob@pacheco.us
pacheco.us
1
PRT MEMBER EVALUATION QUESTIONS
professional or personal growth or on their home institutions, and to rate changes in the effectiveness of the PRT process from their first experience to their most recent. Eighty-eight PRT members responded to all or part of the survey instrument.
}} Please rate the effect, if any, that your participation in the PRT process has had in your identified Areas of Focus. }} Please rate the effect, if any, that your participation in the PRT process has had in your own professional growth and development. }} Please rate the effect, if any, that your participation in the PRT process has had in your own network of colleagues and resources.
Both CI representatives and PRT members were asked to identify any specific improvements to the PRT process they would suggest.n
EMERGENT THEMES FROM THE SURVEY RESPONSES PRT MEMBERS MOST PRT MEMBERS reported that participation in the PRT process had a strong positive effect on their own professional growth (60%). Moreover, the vast majority of PRT members reported that participation in the PRT process had either a strong or moderately strong positive
INTERVIEWEE EVALUATION QUESTIONS
participation in the PRT process at their home institutions. The following themes emerged from the openended responses from PRT members, as a group, regarding the impact that participation in the PRT process had on their own professional development, their professional networking, and their work at their home institutions:
}} What were some of the major factors related to the PRT process that strengthened your progress? }} Reflecting on your experience, if you had not received PRT services, how much progress would you have made on your Areas of Focus by now? The same? More? Less? Please explain. }} Based on your experience with PRTs, how might the technical assistance model used in the PRT process help colleges make progress on the state’s initiatives (e.g. pathways, AB 705, Vision for Success).
üüPRT members valued learning how other institutions
operate and solve problems, and compared CI operations and practices to those of their home institutions as a part of their own self-reflection and home institution improvement.
üüPRT
members reported that they experienced increased connectivity with peers in the field because of participation in the PRT process, both among PRT members and with CI participants.
üüPRT members valued the collegial learning, both effect on their professional development (95%). With respect to the effect on their own network of colleagues and resources, almost half (46%) reported a strong positive effect, with the vast majority of respondents reporting either a strong or moderately positive effect (86%). Finally, nearly two in five (39%) of the PRT members reported applying practices learned through
2
Robert Pacheco, Ed.D., External Evaluator | bob@pacheco.us
in specific content areas and in peer assistance facilitation, that occurred from exposure to fellow team members with diverse skill sets from varied levels of leadership and from different fields of study.
üüPRT members noted improvement in conceptual
knowledge in key community college issues and practices such as accreditation, enrollment management, and student learning outcomes assessment.
PRT MEMBERS (continued)
üüPRT
members saw professional improvement in their own facilitation skills, including applying an appreciative inquiry approach to institutional improvement, conducting needs assessments, and facilitating complex conversations.
üüPRT members reported that they had built lasting
relationships with fellow team members (e.g., contacted team members after the visits for counsel and advice) and that these relationships would not exist but for the PRT process.
üüPRT
members developed wider perspectives on issues facing community colleges because of working with fellow PRT members and CIs.
üüPRT members brought back to their home institutions
specific areas of practice from the CIs, such as student learning outcomes assessment techniques,
planning and budgeting procedures and application of integrated plans (BSI, SSSP and Equity).
üüPRT
members who participated in multi-PRT experiences experienced further expansion of their professional skill set because of the additional site visits.
üüPRT
members who participated in multi-PRT experiences noted that additional exposure to varied college experiences broadened understanding of the issues facing community colleges and that this knowledge acquisition was a key benefit to participating in additional visits.
üüPRT
members who participated in multi-PRT experiences recognized improvement in their own ability to be a team member as part of the PRT process. n
CLIENT INSTITUTIONS THE FOLLOWING THEMES EMERGED from the open-ended responses from the CIs, as a group, to the progress on the identified Areas of Focus since the completion of the PRT process, the effect on structures and processes beyond the Areas of Focus, any surprising or unanticipated learning and whether and to what extent progress that would have occurred had the institution not sought PRT assistance:
üüClient
Institutions noted that good progress on the Areas of Focus took place because the PRT process addressed critical, larger-scope issues that the institution needed to tackle.
üüClient
Institutions reported that progress on the Areas of Focus was impacted by the level of support of, reinforcement for, and perceived value of the PRT process by leadership.
üüClient Institutions stated that the PRT process
had a positive impact on the progress in the Areas of Focus by providing structure and positive accountability for the institution and
providing funding to secure the resources needed to address the AOFs.
üü
Client Institutions reported that the PRT members had a positive impact on the progress in the Areas of Focus by building relationships with CI leaders, working as peers and not evaluators in the process, and fostering a positive mindset to address otherwise vexing issues at the institution.
üüClient Institutions that experienced limitations in progress on their AOFs identified lack of permanent leadership (presence of interims), leadership attrition (including departure of key persons who brought the PRT concept to the institution), resistance to change of existing mechanisms and structures (collective bargaining, status quo), and budgetary constraints as key impediments.
üüClient Institutions recognized progress beyond the Areas of Focus because of the PRT process, including, for example, greater engagement pacheco.us
3
CLIENT INSTITUTIONS (continued) in conversations on broader topics and larger systems issues (e.g., enrollment management) at the institution, pollination of mindset to other areas, and an openness to peer-based assistance in the future.
üüA
few Client Institutions reported low or modest initial expectations about the impact of the PRT process on institution structures and processes and were thus surprised the process delivered as much effect as it did.
üüClient
Institutions observed greater participation and responsiveness than expected by key constituent groups and administration in the PRT process.
üüClient
Institutions indicated that the participation in the PRT process accelerated progress on the Areas of Focus by providing accountability and reporting structures that fostered success. n
EMERGENT THEMES FROM IN-DEPTH INTERVIEWS THE FOLLOWING THEMES were observed in the in-depth interviews:
üüThe
yearlong approach to addressing Areas of Focus offers wide latitude for Client Institutions to consider varied alternatives and to engineer solutions; abbreviated visit cycles should be limited to targeted, narrow areas for improvement.
üüBoth PRT Members and CI leaders report growing
professionally because of the process, including learning how other colleges function, bringing novel practices back to home institutions, and working together and sharing practices with colleagues on commonly shared issues.
üüPRT
members reported similarities and differences with practices at home institutions, increasing empathy for the Client Institutions, and honest reflection on issues both at home and in the Client Institutions. PRT leads in particular reported the development of professional expertise and connectivity with
4
Robert Pacheco, Ed.D., External Evaluator | bob@pacheco.us
Client Institution leads that would have not taken place but for the PRT process.
üüOpportunities
for growth in institutional effectiveness lie in the sharing of experiences and mistakes in safe settings with peers and colleagues.
üüCommunity colleges are experiencing initiative
fatigue and struggling to complete the new expectations from the Chancellor’s Office with little time for traditional institutional effectiveness efforts.
üüAll the initiatives from the state are novel, and finding experienced leaders in the requisite areas will be challenging.
üüCommunity
colleges are spending time completing work which is not fully understood, and there is a sense that there is no time for innovation and creativity.
üüInstitutional
self-efficacy is likely to be strained, as colleges are unclear as to statewide expectations.n
CONCLUSIONS AND RECOMMENDATIONS THE PRT PROCESS has been successful in bringing together talented peers and resources and Client Institutions who seek assistance in focused areas to improve their institutional effectiveness. The increased emphasis on the part of the California Community College Chancellor’s Office to implement legislation and policy through initiatives (e.g., Vision for Success) will be challenging for institutions, particularly those experiencing high turnover and difficulty filling key leadership positions with qualified permanent candidates. The difficulties that colleges are undergoing will need to be addressed in a non-threatening environment using skilled facilitation by experienced peers and coaches. Much has been learned from PRTs about the systems and processes at the institutions, and this learning must be put to use as colleges venture into new areas of accountability and reporting. The following recommendations rest on the findings of this report, the findings of previous process reports, and a review of the emerging expectations from the state chancellor’s office with respect to the Vision for Success: 1. RETAIN THE MAIN FEATURES OF THE EXISTING PRT PROCESS for application to additional Areas of Focus, such as improving outcomes related to the Vision for Success, comprehensive plans and goal-setting, Guided Pathways and Student Equity Achievement Plans, as well as larger and more complex institutional-level issues that will support these initiatives such as integrated planning, enrollment management, and student learning outcomes assessment. 2. SUPPORT THE SELF-EFFICACY OF CLIENT INSTITUTION LEADERS by providing peer
facilitation of discussions within institutions and among institutions to address and resolve any emerging fear of failure. 3. USE THE PRT PROCESS TO PROVIDE FOCUSED SUPPORT for institutions experiencing high turnover at key leadership and constituent positions. 4. TRIANGULATE THE EFFORTS WITH OTHER TECHNICAL ASSISTANCE PROVIDERS to support client institution efforts to address the key statewide initiatives by blending the PRT process with other assistance, to avoid duplication of efforts and potential conflicting information provided by the multiple resources. 5. INSTITUTE A SEMI-ANNUAL REPORTING SYSTEM to monitor institutional progress on Areas of Focus and keep institutions apprised of the resources available to support progress on the Vision for Success and other initiatives. 6. DIRECT MARKETING OF TECHNICAL ASSISTANCE IN FUTURE CYCLES to highlight the positive model of engaging with peers in collegiality and cooperative support. 7. EXPAND THE DISSEMINATION OF FINDINGS AND LESSONS learned more widely and in more varied ways to share learning and open opportunities for greater participation. 8. CREATE VENUES FOR CLIENT INSTITUTIONS TO SHARE EXPERIENCES for mutual learning and greater connectivity. 9. SUPPORT INTEGRATION OF THE SIMPLIFIED METRICS into institutional strategic planning efforts.
pacheco.us
5
CONCLUSIONS AND RECOMMENDATIONS
(continued)
10. USE THE PEER ASSISTANCE MODEL TO PROVIDE VENUES FOR COLLEGES TO SHARE, without fear, difficulties in meeting the reporting expectations.
facing in addressing statewide initiatives. For example, develop and offer technical support in aspirational goal-setting.
11. FACILITATE ACCESS TO, DISPLAY OF, AND USAGE OF METRICS AND OTHER DATA to inform institutional decision-making and demonstrate institutional progress. 12. USE TARGETED TECHNICAL ASSISTANCE MODELS SUCH AS MINI-PRTs for tactical solutions to narrow issues that colleges are
13. USE THE PRT PROCESS TO HELP INSTITUTIONS BE SELF-ACCOUNTABLE and on task to meet deadlines for the Vision for Succss. 14. SHARE THE STRUCTURED PLANS AND RESOURCES DEVELOPED OR USED by Client Institutions to the extent possible to provide exemplars for other institutions.
Robert J. Pacheco, Ed.D. professional development consulting coaching & facilitation
415.800.4043 bob@pacheco.us
MATTHEW C. LEE, Ph.D. Project Director
THERESA TENA
CALIFORNIA COMMUNITY COLLEGES CHANCELLOR’S OFFICE SACR A ME N TO, C A L I FO R N I A 91 6 . 4 4 5 . 8752
6
Vice Chancellor, Institutional Effectiveness CCC Chancellor’s Office
BARRY GRIBBONS, Ph.D . Deputy Chancellor College of the Canyons
COLLEGE OF THE CANYONS/ SANTA CLARITA CCD SA N TA C L A RI TA , C A L I FOR NIA 661. 362. 5 5 00