IMPROVEMENTS and
INNOVATIONS
MATTHEW C. LEE, PH.D. | IEPI PROJECT DIRECTOR | JUNE 2018
I MPROVEMENTS AND I NNOVATI ONS IN T H E PA R TNERSHI P RESOURCE TEAM ( PRT) PROC E SS
INTRODUCTION This report is the third annual summary of improvements and innovations made to the PRT process based upon formal and informal feedback from the client institutions, PRT team members, and IEPI leaders. The goal of the report is to demonstrate actions taken to: »» Improve the impact of the PRT Process on the Client Institutions »» Increase the value gained by participating in the PRT Process by both the Client Institutions and the PRT Members. »» Implement training and technical assistance techniques, tools and concepts that positively impact PRT visits and best assist Client Institutions in addressing the identified Areas of Focus. The IEPI Project Director (PD), independently and in collaboration with the IEPI Technical Assistance Workgroup (TAW) and the external evaluator of the PRT component of the initiative, regularly reviews a variety of sources of qualitative and quantitative evidence, including cyclical and cumulative evaluation reports, surveys, interviews, visit debriefings, informal meetings and communications, and PRT documents (including the Letters of Interest and the Innovation and Effectiveness Plans (I&EPs)). The PD also gathers evidence on the effectiveness of training after each of these events, principally in the form of participant surveys and discussions with PRT Leads and members
VISITS The Project Director gathers information at the end of Visit 1, Visit 2 and Visit 3 (the Follow-Up Visit) in the full PRT cycle primarily through separate surveys of PRT Client Institutions and PRT members and through debriefings with PRT Leads and selected client CEOs, to examine the effectiveness of the visits themselves. In addition, the PD reviews in detail all Summaries of Initial Visits, Menus of Options (MOO), Innovation and Effectiveness Plans, and PRT Process Summary reports as sources of information on possible process improvements.
TRAININGS In each cycle, the Project Director conducts an introductory webinar as well as in-person workshops for PRT teams. Surveys are administered to attendees addressing content, process and logistical items for improvements. Workshops have been expanded and refined based on content analysis and participant feedback.
CO M M U N I T I E S O F P R AC T I C E Full implementation of the first of the Partnership Resource Communities of Practice (PRCOPs), regional groups of institutional leaders and others focused on improving in areas of common interest and need, began in December 2017. PRCOPs extend the PRT colleagues-helping-colleagues approach, with the Client Institutions providing venues for dialogue and assistance to each other, supplemented by IEPI resources. They thus build local and regional capacity to improve and sustain institutional effectiveness.
MINI-PRTS Mini-Partnership Resource Teams (Mini-PRTs) differ from full Partnership Resource Teams, primarily in that they consist of two to three volunteer content experts helping institutions address relatively narrow areas of focus in a single visit. As with the PRCOPs, evaluation efforts have just commenced, but findings are already being used for formative improvements to the process.
R E C E N T I M P R O V E M E N T S & I N N O VAT I O N S The main improvements and innovations implemented since the publication of the last report are summarized in the following tables, together with the evidence that led to each improvement or innovation. Robert Pacheco, Ed.D., External Evaluator | BOB@PACH3.CO
1
I M PR OV E M E N TS A N D I N N OVAT IO N S
INNOVATION OR IMPROVEMENT
ASPECT OF TECHNICAL ASSISTANCE (Training, PRT Visits, PRCOP, Mini-PRT)
EVIDENCE THAT LED TO IMPROVEMENT
REFINED I&EP TEMPLATE TO FIT THE MINI-PRT STRUCTURE BETTER
MINI-PRT
REFLECTION ON MINI-PRT PROCESS
ADDED SPECIFIC DEADLINE FOR CEO TO PROVIDE TREATMENT, CRUCIAL DOCUMENTS, DRAFT AGENDA WITH PARTICIPANTS, AND LOGISTICS
MINI-PRT
REFLECTION ON MINI-PRT PROCESS: INFORMATION HAD BEEN PROVIDED LATE IN SOME CASES.
ADDED REQUEST FOR A SUBSTANTIVE POINT PERSON AS WELL AS A LOGISTICAL POINT PERSON
MINI-PRT
REFLECTION ON MINI-PRT PROCESS: CEOs WOULD HAVE BENEFITTED FROM AN SPP TO TAKE ON SOME OF THE PREPARATIONS.
ADDED NEXT STEPS EMAIL TO CEO AFTER THE VISIT
MINI-PRT
REFLECTION ON MINI-PRT PROCESS: A REMINDER TO THE CEO PROVED USEFUL.
DESIGNED AND IMPLEMENTED FORMAL EVALUATION OF MINI-PRTS BASED ON SURVEYS OF CLIENT INSTITUTION AND MINI-PRT MEMBERS
MINI-PRT
MINI-PRT FORMATIVE EVALUATION
REFINED PRT PROCESS CHECKLISTS FOR BOTH CLIENT INSTITUTION AND PRT LEADS
INITIAL CEO MEETING; TRAINING; PRT VISITS
REFLECTION ON PRT PROCESS: MINOR MODIFICATIONS WERE NEEDED TO IMPROVE CLARITY.
REFINED BOTH MODEL AND TYPICAL PRT PROCESSES
INITIAL CEO MEETING; TRAINING; PRT VISITS
REFLECTION ON PRT PROCESS: MINOR MODIFICATIONS WERE NEEDED TO IMPROVE CLARITY.
EMPHASIZED TO PRTs THAT VISIT 1 SHOULD BEGIN WITH APPRECIATIVE INQUIRY TECHNIQUES TO ENCOURAGE INSTITUTIONS TO BUILD UPON PRIOR SUCCESSES
TRAINING
PROVIDED TO CEOs A MORE EXPLICIT ADVANCE RATIONALE FOR THE PURPOSE, SIZE, AND COMPOSITION OF INNOVATION AND EFFECTIVENESS PLAN DRAFTING GROUP.
INITIAL CEO MEETING; PRT VISITS
SPECIFIED A TIGHTER DEADLINE FOR THE LEAD TO SEND THE MENU OF OPTIONS DRAFT TO THE PD FOR FEEDBACK, TO HELP ENSURE TIMELY DELIVERY TO THE CLIENT INSTITUTION
PRT VISITS
REFLECTION ON PRT PROCESS
REFLECTION ON PRT PROCESS: SOME CEOs HAD REQUIRED REMINDERS OF THE PURPOSE AND NATURE OF THE DRAFTING GROUP.
REFLECTION ON PRT PROCESS: DELAYS IN PRODUCTION OF THE MOO, THOUGH RARE, HAD HAD AN IMPACT ON THE EFFECTIVENESS OF SECOND VISITS.
data continues on page 3
Robert Pacheco, Ed.D., External Evaluator | BOB@PACH3.CO
2
I M PR OV E M E N TS A N D I N N OVAT IO N S
INNOVATION OR IMPROVEMENT
ASPECT OF TECHNICAL ASSISTANCE (Training, PRT Visits, PRCOP, Mini-PRT)
EVIDENCE THAT LED TO IMPROVEMENT
EXPLICITLY EMPHASIZED TO CEOs AND TO LEADS THAT THE FINAL I&EP IS TO BE SIGNED BY BOTH CEO AND ACADEMIC SENATE PRESIDENT
PRT VISITS
REFLECTION ON PRT PROCESS; IEPI EXECUTIVE COMMITTEE FEEDBACK
HIGHLIGHTED SUSTAINABILITY OF I&EP IMPROVEMENTS IN AREAS OF FOCUS AS AN AREA OF PRT INQUIRY FOR VISIT 3
PRT VISITS
SUSTAINING INSTITUTIONAL EFFECTIVENESS: ONE YEAR AFTER THE PRT PROCESS RECOMMENDATION 3; REFLECTION ON PRT PROCESS
CLARIFIED FOR LEADS THE NATURE AND CONTENTS OF THE PRT PROCESS SUMMARY REPORT, AND THE IMPORTANCE OF TIMELY DELIVERY
PRT VISITS
REFLECTION ON PRT PROCESS: HOLDUPS IN PRODUCTION OF SUMMARY DOCUMENTS, THOUGH RARE, HAD DELAYED THE WRAP-UP OF PRT MEMBERS’ RESPONSIBILITIES, AND THUS THEIR AVAILABILITY FOR OTHER PRTS.
CLARIFIED AND REINFORCED FOR BOTH CEOs AND LEADS THE EXPECTATION THAT THE FULL PRT CYCLE PROCESS CONSISTS OF THREE DISTINCT VISITS, WITH EXCEPTIONS TO THE PROCESS BEING RARE
INITIAL CEO MEETING; TRAINING; PRT VISITS
REFLECTION ON PRT PROCESS AND EVALUATION FINDINGS (SEE, FOR EXAMPLE, SUSTAINING INSTITUTIONAL EFFECTIVENESS: ONE YEAR AFTER THE PRT PROCESS)
CLARIFIED TO CLIENT INSTITUTIONS THE NATURE AND PURPOSES OF THE FOLLOW-UP VISIT (VISIT 3) AS AN INFLECTION POINT IN THE CYCLE, AN OPPORTUNITY FOR REFLECTION ON AND CELEBRATION OF PROGRESS TO DATE, AND A LAUNCHING POINT FOR CONTINUED EFFORTS THROUGH AND BEYOND THE PRT PROCESS
PRT VISITS
REFLECTION ON PRT PROCESS AND EVALUATION FINDINGS (SEE, FOR EXAMPLE, SUSTAINING INSTITUTIONAL EFFECTIVENESS: ONE YEAR AFTER THE PRT PROCESS)
ADAPTED SPRING 2018 CYCLE PREPARATION TO UNUSUALLY LATE ARRIVALS OF LETTERS OF INTEREST (LOIs)
PRT VISITS
REFLECTION ON PRT PROCESS
IMPROVED MESSAGING TO PROSPECTIVE CLIENT CEOs REGARDING TIMELY SUBMISSIONS OF LOIs.
PRT VISITS
REFLECTION ON PRT PROCESS: UNUSUALLY LATE LOIs HAD CAUSED DIFFICULTIES IN SPRING 2018 CYCLE.
ACCELERATED CERTAIN STEPS IN CYCLE PREPARATION TO ALLOW FOR GREATER VARIABILITY IN LOI SUBMISSION DATES AND ASSURE TIMELY ASSEMBLY OF PRTs
PRT VISITS
REFLECTION ON PRT PROCESS: FLEXIBILITY IN PREPARATION SCHEDULE HAD PROVEN CRUCIAL FOR SPRING 2018 CYCLE.
EXPERIMENTED WITH ALTERNATIVE MEDIA DURING TRAININGS TO IMPROVE LEARNING EXPERIENCE
TRAINING
EVALUATION OF WORKSHOP CONTENTS AND PRESENTATION
data continues on page 4
Robert Pacheco, Ed.D., External Evaluator | BOB@PACH3.CO
3
I M PR OV E M E N TS A N D I N N OVAT IO N S
INNOVATION OR IMPROVEMENT
ASPECT OF TECHNICAL ASSISTANCE (Training, PRT Visits, PRCOP, Mini-PRT)
EVIDENCE THAT LED TO IMPROVEMENT
HIGHLIGHTED THE INTEGRATED PLANNING AND ENROLLMENT MANAGEMENT ASKs DURING THE PRT WORKSHOP BASED ON COMMON AREAS OF FOCUS
TRAINING
REFLECTION ON TRAINING: ADDITIONAL ASK RESOURCES MADE THIS ENHANCEMENT POSSIBLE.
INCREASED THE NUMBER OF EXPERIENCED PRT MEMBERS ON NEW PRTs
PRT VISITS
REFLECTION ON PRT PROCESS; LEAD DEBRIEFS; EVALUATION FINDINGS
REFINED THE SEED GRANT APPLICATION AND AMENDMENT FORMS AND INSTRUCTIONS
PRT VISITS
INTERNAL REVIEW OF FORMS AND INSTRUCTIONS
FULLY IMPLEMENTED THE NORTHERN CALIFORNIA PARTNERSHIP RESOURCE COMMUNITY OF PRACTICE (PRCOP)
PRCOP
COORDINATED PLANNING WITH REGION 1 CEOs
DESIGNED AND IMPLEMENTED EVALUATION OF PRCOP MEETINGS
PRCOP
COORDINATED PLANNING WITH REGION 1 CEOs
ASSISTED IN THE REFINEMENT OF THE NORTHERN CALIFORNIA PRCOP WHITE PAPER
PRCOP
PRCOP DRAFTING GROUP REQUEST
CLARIFIED THE STRUCTURE OF PRCOP PARTICIPATION AND POTENTIAL AGENDAS
PRCOP
PRCOP EVALUATION
REFINED SURVEY QUESTION SETS TO FIT BETTER THE UNIQUE ASPECTS OF EACH VISIT IN THE PROCESS
PRT VISITS
SURVEY RESPONSES; REFLECTION ON ESSENTIAL DIFFERENCES BETWEEN VISITS
IMPLEMENTED AN OPTION FOR REDUCED TIME BETWEEN THE FIRST TWO VISITS TO ACCOMMODATE PARTICIPANTS’ SCHEDULES AND HELP CLIENT INSTITUTIONS REMAIN ON TASK FOR PREPARATION OF THEIR I&EP
PRT VISITS
REFLECTION ON PRT PROCESS
IMPROVED SCORE SCALES TO MAKE MORE INTERNALLY CONSISTENT THE RESPONSES ON CONFIDENCE IN AND AWARENESS OF THE PRT PROCESS
PRT VISITS
REFLECTION ON EVALUATION OF PRT PROCESS
IMPROVED CONNECTIVITY WITH OTHER ASPECTS OF IEPI, INCLUDING THE ASKs AND EVALUATION OF PROFESSIONAL DEVELOPMENT
PRT VISITS
EXECUTIVE AND ADVISORY COMMITTEE FEEDBACK
HELPED RECRUIT COACHES FOR THE SEM PROGRAM FROM AMONG PRT VOLUNTEER RANKS
PRT VISITS
REQUEST BY SEM COORDINATORS
data continues on page 5
Robert Pacheco, Ed.D., External Evaluator | BOB@PACH3.CO
4
I M PR OV E M E N TS A N D I N N OVAT IO N S
INNOVATION OR IMPROVEMENT
ASPECT OF TECHNICAL ASSISTANCE (Training, PRT Visits, PRCOP, Mini-PRT)
EVIDENCE THAT LED TO IMPROVEMENT
PROVIDED FEEDBACK TO ASK COORDINATOR AND DESIGN MEDIA IN THE DESIGN AND USABILITY OF THE ASK s
PRT VISITS
REQUEST BY ASK DESIGNERS
INCREASED THE PROPORTION OF MOO MODELS AND EXAMPLES THAT REFLECT ASK AND PLN CONTENT
PRT VISITS
REVIEW OF ALL MOO s
ADVISED PROSPECTIVE CLIENT CEOs REGARDING THE TYPE OF TECHNICAL ASSISTANCE THAT WOULD BEST SUIT THEIR NEEDS
PRT VISITS
REFLECTION ON PRT PROCESS
CO N S I D E R AT I O N S FO R FU T U R E ACT IO N Ideas for improvements and innovations under consideration for implementation in the future are summarized in the following table, together with the evidence that led to each idea.
INNOVATION OR IMPROVEMENT
ASPECT OF TECHNICAL ASSISTANCE (Training, PRT Visits, PRCOP, Mini-PRT)
EVIDENCE THAT LED TO IDEA
ADD SPECIFIC GUIDANCE TO THE CLIENT INSTITUTION’S DRAFTING GROUP FOR THEIR PREPARATIONS FOR VISIT 2, INCLUDING, FOR EXAMPLE, A PRIOR MEETING TO DISCUSS THE OPTIONS PROVIDED BY THE PRT
PRT VISITS
TECHNICAL ASSISTANCE WORKGROUP FEEDBACK; REFLECTION ON PRT PROCESS
BUILD SELF-REFLECTION PAUSE POINTS INTO THE PRT PROCESS JUST BEFORE VISIT 3 AND JUST BEFORE THE END OF THE INITIAL SEED GRANT PERIOD
PRT VISITS
SUSTAINING INSTITUTIONAL EFFECTIVENESS: ONE YEAR AFTER THE PRT PROCESS RECOMMENDATION 7; REFLECTION ON PRT PROCESS
WITH CEOs’ PERMISSION, POST A LIST OF INSTITUTIONS THAT HAVE RECEIVED PRT ASSISTANCE (TOGETHER WITH THEIR AREAS OF FOCUS AND CONTACT INFORMATION) BY REGION, TO FACILITATE CONTACT BY OTHER INSTITUTIONS THAT MIGHT HAVE SIMILAR AREAS OF INTEREST
PRT VISITS
TECHNICAL ASSISTANCE WORKGROUP FEEDBACK
SHARE INTERACT VIDEO FOLLOWUP INTERVIEWS WITH PRT MEMBERS AND CLIENTS ON THE IEPI WEBSITE
PRT VISITS
TECHNICAL ASSISTANCE WORKGROUP FEEDBACK
Robert Pacheco, Ed.D., External Evaluator | BOB@PACH3.CO
5