Data Analytics Center Projects

PennOpen Pass Enhancements

PennOpen Pass was implemented as a daily symptom tracker to reduce the spread of COVID-19 across Penn Medicine and the University of Pennsylvania through daily COVID-19 screening of all employees, patients, visitors, contractors, and students.

» Met OSHA (Occupational Safety and Health Administration) and PA Department of Health regulations for daily COVID-19 screening for Penn Medicine employees.

» The PennOpen Pass dashboard was designed to aggregate the metrics of approximately 30,000 daily users to provide the data needed to help management and clinical staff better track compliance.

» The dashboard was actively used by management to view red passes issued and identify trends over time and to evaluate usage of the system as well.

» On average, this page was accessed 2,200 times per month. This information was key, not only to finding ways to improve the PennOpen Pass process, but also to provide easy access to the additional data gathered by the system.

19,504,869 SCREENINGS

126,734 SCREENINGS • DISTINCT USERS

18,760,836 GREEN PASSES

220,544 RED PASSES

46,715,320 SMS MESSAGES

405,008 INCOMING SMS MESSAGES

A mobile friendly daily screening tool that asks questions about an individual’s (staff, faculty, students, etc.) COVID-19 symptoms, exposures, and testing status. Based on the responses PennOpen Pass determines whether a person is cleared to come on site for the day (green pass) or whether they are not cleared to come on site (red pass). People can sign up for daily reminders via SMS text. The daily reminders contain a hyperlink that allows users to complete their daily screening via their smartphone or a computer. The collected data is used regularly to monitor trends in COVID-19 cases across Penn Medicine, and the University while they were using it.

These technologies and processes address the administrative infrastructure support and physical infrastructure support that includes financial services, operations, human resources, managed care contracting, billing, materials management, and other administrative services. This enterprise-wide approach designs and implements product and service processes to achieve performance gains and decrease costs while optimizing resources.

The Pavilion has been one of the most detail focused and innovative projects undertaken at Penn Medicine. The state-of-the-art technology integrated within the Pavilion transforms the manner in which Penn Medicine provides patient care and experience through detailed organization and design. Innovation and technology were the primary focus when designing each and every wall in the hospital as the goal is to deliver the treatment of today and the medical advances of tomorrow. The following projects represent the technology built to support the Pavilion and transform the patient care experience using smart building integration.

Security camera system platform was designed with new IP based camera system including hardware, software, security operations center design and staff training.

Implemented over 55 employee time clocks throughout 1.5M square feet- determined placement, configured new low voltage wiring, programming and installation.

Implemented technology supporting cafeteria systems to include point of sale stations, 10 large format food menu selection display boards and ensured system security and auditing capability for transaction history for sales accuracy.

Installed an updated parking control system which secures payment, tracks vehicle flow, and establishes pay on foot capabilities on the 6 Pavilion parking garage levels.

Go-live readiness assessments and dress rehearsals were done to prepare for technology application transition, patient movement prep, and command center logistics.

Emergency department and inpatient hospital electronic health record workflow analysis, design, build, and testing were completed.

Imaging and modality integration was coordinated for vendor installation and configuration for the following clinical imaging equipment newly placed in, or moved to, the new Pavilion: Radiology PACS build, Cardiology PACS build, associated EHR build, connectivity testing, device testing and PACS / EHR integration testing. Device modalities included (Xray, CT, MRI, Ultrasound, etc.) and diagnostic reading/tech workstations

Application updates made to EHR modules OpTime, Cardiology and Radiology, supporting new innovative workflows and technology to support our clinicians in delivering exceptional care while improving the patient experience.

Implemented new equipment within the Pharmacies (robots and carousels) and Lab spaces (microscopes & IT devices).

Penn Medicine’s project management methodologies ensured the deployment was on time while adhering to high quality standards.

Installed new virtual enterprise level Nihon Kohden physiological monitoring system.

Automated vitals data capture and EHR recording for critical care patients, ICU and OR clinical computing hub

Deployment of clinical technology devices, including clinical engineering equipment.

Implemented robotic server and software technology for pharmacy operations and delivery of medications.

Introduced patient portal bedside module so patients and families can engage in their care through educational materials and by providing their lab results, vital signs, upcoming tests, and their health care team members.

Activated 120 patient rooms early, to provide extra capacity for anticipated surge in COVID-19 patients.

Deployed 65 iPads for patient use.

Technology infrastructure constructed for 3rd floor connector bridge from the Pavilion to Perelman Center for Advanced Medicine (PCAM). Relocated peri-operative waiting room to PCAM. Peri-operative area was expanded into the old waiting room with (10) new patient bays. The old per-operative area was used as the new pathway from PCAM and onto the Pavilion connector bridge.

New dermatology suite was built adjacent to the bridge area to coincide with connector bridge space.

Patient flow management application and care team tool was implemented.

Implemented a single enterprise information platform for managing content, increasing productivity and reducing cost.

Prepared billing and claims processes, ensuring continuity in charge method, and structure and coding processes to align with health system standards

HUP Pavilion - Revenue Cycle (HIM, EMPI) – Evaluation of HIM components for the Pavilion, ensuring the availability, accuracy and protection of clinical information needed to deliver healthcare services.

Designed new electronic patient white board display for patient rooms.

Implemented technology for nurse call system for patients to request help from their bed.

Penn Medicine Opens the Largest Healthcare Equipment Sterilization Facility in the Country

Penn Medicine opened a new Interventional Support Center (ISC), the largest instrument processing and surgical supply preparation facility in the country. Located in Southwest Philadelphia, the ISC is the first facility of its kind in Pennsylvania. In this space, staff sterilize and package thousands of instruments each day in preparation for surgeries and procedures from basic scissors and clamps to advanced robotic instruments.

At nearly 110,000 square feet, the ISC was designed to process instruments from up to 200 surgical cases each day. Penn Medicine purchased the building in 2018 and built a physical plant to support the ISC’s operations. The ISC provides instrument processing services for the Hospital of the University of Pennsylvania, Pennsylvania Hospital, and the new Pavilion Hospital of the University of Pennsylvania, as well as three outpatient facilities: the Perelman Center, Penn Medicine Radnor, and the Tuttleman Center. The ISC was built to support additional Penn Medicine entities.

The opening of the ISC set a new standard for sterile instrument processing. It also marked a new approach from Penn Medicine. By moving processing operations from the traditional hospital setting to an offsite, dedicated facility, efficiency was increased in a high quality, cost effective way while keeping up with increasing demand. Plus, the ISC alleviated space at all clinical locations, providing room for hospital departments to expand their services.

In addition to providing hospital locations with much-needed real estate for patient care, consolidating these services off-site helped minimize any redundant equipment across the health system. The ISC itself was designed with a one-way forward flow approach that enables efficient instrument processing. The facility includes features that enhance sterilization efficiency, such as clean steam using reverse osmosis water. The ISC is a nearly dust-free facility - thanks to increased air changes and filtration. Airlocks throughout the building were also built to prevent cross-contamination of air between areas where dirty instruments are processed, and clean ones are repackaged.

The ISC team is made up of 140 employees from on-site instrument processing facilities at the Hospital of the University of Pennsylvania and Pennsylvania Hospital. In addition, 25 new employees were hired to staff the new building. The facility was created with ergonomic principles and employee well-being in mind, enabling a comfortable workspace for the ISC team members. All major equipment was designed for personalized adjustments for staff, such as height adjustable sinks and assembly tables.

To make this happen the IT team:

• Outfitted the new ISC building with all required IT hardware and implemented an enterprise-wide software instrument tracking system that was integrated with sterilizers, biological incubators, and an automated storage carousel system

• Standardized the instrument processing software system across the health system and integrated tracking software with our EHR, Penn Medicine’s human capital management solution software and Penn Medicine’s web-based tool for locating and viewing assets in realtime

• Established field service support for the service desk, and end-user-services

“Clean instruments are a crucial component of patient safety. Through centralized processing with state-of-the-art equipment, the ISC is an essential element behind Penn Medicine’s efforts to continue delivering high-quality, advanced patient care and safety,” said Phil Okala, Chief Operating Officer at the University of Pennsylvania Health System. “The ISC also stands as a unique, leading example for multi-hospital health systems looking to create improved processing systems and thoughtfully plan the best ways to maximize capacity for patient care.”

Tracking Surgical Instruments at Penn Medicine Princeton Health Meets Enterprise Standards

To meet regulatory requirements, Penn Medicine Princeton Health implemented a surgical instrument management system to interface sterilizer data into the core instrument sterilization system. This implementation allowed for data values from the sterilizers to be transferred to the instrumentation sterilization parameters being documented in the system, consequently meeting the State of New Jersey requirements to monitor and maintain values for seven years.

The addition of the interface standardized the process throughout the enterprise and eliminated paper tracking which ended the need for dual documentation. It also linked patient information with the case trays, which allowed the tracking of trays to the patient level, improving patient safety.

Chester County Hospital Expands Emergency Services

Penn Medicine expanded its private room footprint in the emergency department and renovated legacy space to ensure a cosmetic and cohesive flow to the new emergency department space in the Pavilion at Chester County Hospital.

This three-phased project expanded the emergency department to include:

• Twenty-nine exam rooms

• Nine enhanced safety exam rooms

• Five nursing stations

• Three triage rooms

• Two resuscitation rooms

• Two trauma bays

Each phase took six months and included installation of new cabling infrastructure and deployment of new hardware. Chester County Hospital’s emergency department now provides increased access to emergency services for the community and benefits operationally from improved patient flow.

Gastroenterology and Endoscopy Expand Footprint

To meet the growing outpatient needs of the Gastroenterology/Endoscopy department, a new and spacious facility was designed and constructed on the west side of Penn Medicine University City. The new space was built having a common reception area for both Gastroenterology and Endoscopy, ten GI exam rooms, four endoscopy procedure rooms, twelve pre/post-procedure bays and provider touchdown/office space. The enterprisestandard, state-of-the-art monitoring platform was chosen for physiological monitoring – making it the first deployment on the Penn Presbyterian Medical Center campus.

At the same time, increased demand to accommodate a wider variety of procedures for inpatient Gastroenterology services necessitated the planning and deployment of four new operating rooms, designed specifically to meet the technical needs of the department. A new operating room was part of the Cupp Hematology-Oncology expansion at Penn Presbyterian Medical Center.

Behind the Scenes - The Business of Keeping the Business Running

In the summer of 2020, Penn Medicine finance department staff were challenged to maintain social distancing guidelines at the 1500 Market Street office, while meeting the business needs of the healthcare system. The employee rotation schedule helped with social distancing, but employees working remotely had limited access to business systems. The recommendation was to outfit 700+ employees with new computers and modified workflows from a system’s perspective to perform their work remotely. To make that possible, a core IT team:

• Worked with the finance managers to identify required hardware to enable the team to work remotely

• Identified workflows that needed to be modified for remote work

• Implemented RightFax, enable OneDrive, reduce the need for printing

• Arranged for home delivery of equipment

• Setup a command center to transfer data from employee on-site desktops to new hardware

• Trained the employees on their modified workflows

The successful completion of these tasks allowed Penn Medicine’s finance division to work remotely. In turn, Penn Medicine was able to consolidate office space into three floors, reducing overall costs.

Hoteling Software Accommodates New Hybrid Work Environment

As part of Penn Medicine’s objective to support a hybrid work style and to efficiently manage real estate resources, a 12-week pilot of an office hoteling system was conducted in select Philadelphia locations. The pilot encompassed spaces at 1500 Market, the Pavilion, select areas at the Hospital of the University of Pennsylvania, and Human Resources at 3600 Civic Center Blvd. Workspaces that were a part of the pilot could be easily identified by the hoteling system label and QR code attached to the wayfinding (cube/office/room number/ name) plaque. A web portal or mobile app was used to select an available workspace from the system. Users were also able to coordinate with coworkers to reserve spaces together when they needed to collaborate in person.

The system proved to be useful in allowing users to reserve cubicles, offices, conference rooms, and other spaces rather than having fixed, permanent workspaces such as cubicles or offices assigned to them. During the pilot, the system was highly used in clinical facilities and user feedback was very favorable. It provided Real Estate Design and Construction with robust reporting for real estate planning and forecasting. The decision was made to adopt the system for production use and perform a phased implementation across Penn Medicine, expanding from the existing pilot locations.

The health system benefitted by gaining greater visibility into how office space could be used, and by whom. Some studies have shown that businesses could save about 30% on facilities costs from using hoteling. The expected benefit of the hoteling system was seen early on with a reduction in Penn Medicine’s real estate footprint and associated real estate cost. Employees were better able to find the types of workspaces and conference rooms they need when they need them.

PennforPeople Expands its Business Platform

PennforPeople entered the second phase of a five-year long program to move Penn Medicine from our previous business application platform to the next generation platform. With our initial phase that focused on human capital management completed, the goal of phase two was the implementation of state-of-the-art applications to replace disparate and paper-based workflows used to on-board employees and collect talent profile data –allowing this activity to be consolidated within PennforPeople.

After our kick-off of phase 2 of the PennforPeople expansion, it was determined that an additional system was required to ensure that Penn Medicine could implement the best possible candidate experience. So, a proactive recruiting system was added to build our career site and job application workflows.

The benefits that resulted from implementing this phase of the project:

• Enhancement and streamlining of the hiring and onboarding process by removing the need for 3rd party interfaces, thus eliminating the cost

• The elimination of manual paper-based processes from managers through PennforPeople

• Automation of the most time-consuming recruiting tasks, allowing recruiters to spend more time guiding and building relationships with candidates

• The streamlining of the offer letter creation process for our onboarding coordinator team

• Efficiencies to the candidate screening process by consolidating all human resource information into one system

• The immediate availability of employee documentation on resource records after candidates are hired

• The streamlining of workflows and the elimination of the manual licensing verification process for Penn Medicine recruiters, onboarding coordinators, managers, and candidates resulting from increased system functionality

Exit Interview Questionnaire Survey Created for HR Retention Program

In addition to the COVID-19 pandemic in 2021, Penn Medicine was impacted by the “Great Resignation”, as an elevated number of employees resigned. Penn Medicine’s leadership was interested in getting a better understanding of the reasoning behind employees exiting the health system. This initiative stemmed from the organization’s retention strategy that was created under Project Vital Advantage. Project Vital Advantage (PVA) was started as a system-wide project focused on recruitment, retention, compensation, work environment (including workplace safety), and workforce wellness and culture.

Application Development, in partnership with the HR department, created an exit survey that was sent to employees that recently resigned from the health system and employees that gave notice of resignation. The surveys were sent out as a post card with a QR code, which launched the survey when scanned. The survey was streamlined to consist of only 6 questions to ensure maximum adoption, with one question providing the employees the opportunity to have HR contact them for follow-up. A report was also built-in with the survey that allowed HR and leadership to view all submissions and track trends.

The information obtained from the surveys provided some valuable insights to leadership on some of the common reasons that employees were leaving. In addition, the organization was able to retain some personnel by following up with the employees that indicated they wanted to be contacted by HR. With the survey in place, the organization had a reportable and actionable source of information that they could use to ensure that Penn Medicine would be the place people want to stay and grow their careers.

Consolidated Retirement Plan Streamlines Employee Retirement Benefit

Before Penn Medicine consolidated its retirement plans with one of the most diversified financial companies in the United States, the health system was using three different vendors to manage employee retirement benefits enterprise-wide. With each vendor having different rules and requiring up to four interfaces each, the process was deemed too labor intensive to support administratively and technically.

Penn Medicine’s retirement department negotiated a contract with a single vendor and Information Services worked alongside them to build a revised enterprise processes that streamlined all administrative and technical support.

This combined effort resulted in:

• Efficient workflows, significantly decreasing staff support time

• A revised and simplified employee experience when accessing their retire account

• Lower administrative costs with the savings passed on to employees via lower account maintenance fees

Finance Capital Expenditure Request Process Reduces Work Burden

Over a decade ago, technology that handled the processing of capital expenditure requests allowed any staff to submit a request and lacked integrated data to handle routing and approvals. Data was manually gathered and updated behind the scenes by the finance capital budget team and approvals had to be manually re-entered into Lawson. Additionally, they managed all data and reporting through a local Microsoft Access database which required extensive manual work.

To alleviate this work burden, the IS team created a new technology solution that provided significant improvements by rewriting the process using standard Lawson development tools and integrating it within the system. The new process was designed to screen submitters by requiring them to log into Lawson and be authorized to submit requests. The review and approval processes were derived from information already set up in Lawson so that little effort is needed to enter or maintain this data. Routing to new approvers and sending back to prior users in the approval chain now had the same look and feel as other approval processes in Lawson, and action items were now managed using the same method as all other Lawson pending action items. Reporting became more automated and robust. Also, with the similar look and feel to other Lawson reporting, it allowed greater transparency in the process, and quicker turnaround to the requestor.

Most of the manual work previously performed by the finance capital budget team was replaced with built-in functionality to eliminate unnecessary work effort from staff resources. The team was furnished with power tools and reporting options that they did not previously have to do mass loads, data extraction, and ad hoc reporting. The new process was designed to be easily portable to the new PennforFSM (Penn for Finance Supply Chain Management) application, with additional features included, after implementation in June 2023.

Technology Based Patient Safety Reporting Now Enhanced

When reporting patient safety events related to technology, users previously had to submit two reports. One instance was entered into our application to report and track the safety issue, the other entry was made in the IS Service Desk system to address the technology issue. This reporting method was tedious and time-consuming for users, who were caregivers, focused on providing patient care.

Our IS department recognized the opportunity to alleviate the necessity for two entries into two different systems by combining the submissions into a single step. The IS operations team worked with the patient quality team’s system administrator to produce workable solutions. With support from our integration and networking groups, we modified the patient safety application to include an option to automatically generate a technology issue service ticket if the user selected it.

Upon making this update, an immediate increase occurred in reported technology-related safety events. Technology tickets were created, and optimization requests accumulated as well as benefits gained from system workflow issues were also addressed. User feedback was incredibly positive regarding this simplified submission process.

Technology Ticket Management Goes Mobile

It is no small task for an IS Service Desk team to consistently maintain superior customer service while responding to and triaging the high volume of incidents for a vast, highly complex health care organization. Among the ever-increasing demand for technology requests and fixes, key members of our application development and service desk teams recognized an opportunity to develop an iOS native app centered on ticket management. The concept was to design a mobile app that provided access to information about tickets, freeing the management team from being tethered to their workstations. Mobile access was attractive and helped to bring efficiency to the ticket management process, also allowing staff to track high-priority tickets without having to physically be at their desks.

The design team integrated multiple tasks into a single, cohesive experience by combining the ticket management functionality along with an existing approvals application to form a new iOS app. The necessary functionality was consolidated into this single application, along with increasing speed-to-market by leveraging the prior application’s infrastructure to avoid redundancy.

Since its introduction, the new ticket management app has been used daily by hundreds of staff members across IS. Staff can now perform approvals, and track tickets on their own time from the ease and comfort of their smart phone. Many staff have praised how the app has become a vital tool for their workflows by making tasks much more accessible and convenient.

Upgrades to Cost Accounting Aligns Services and Saves Time

Penn Medicine’s Financial Decision Support System (DSS) leverages data within Penn Medicine’s electronic health record to enhance financial processes and forecasts across the enterprise by transforming that data into knowledge, thus advancing financial decision support.

Early in 2022, the performance manager analytics application was upgraded. The application consolidated data from multiple systems into a single data model with useful pre-calculated metrics allowing cost accounting to perform sophisticated cost assignments while monitoring performance across patient populations and service lines. This upgrade allowed us to stay up to date with the latest product offerings, support, and available features.

To ensure stability, support organic growth, and provide ample capacity for future integrations, the team also migrated from physical servers to virtual servers. The upgrade and server migration resulted in improved run times of our monthly data extract by 40%, giving us more time to focus on new projects, optimizations, and enhancements. As part of the server migration, testing and development environments were built out to provide safe spaces for future needs and improvements to the existing data model. The Decision Support team has also been working with our partners in finance to enhance monthly processes and deliver useful data to support the organization’s goals.

Service Desk Thrives Using Quality Assurance Program

Penn Medicine’s technology support model grew more complex with the onset of the COVID-19 pandemic. As a segment of our workforce moved to function remotely, the Penn Medicine Service Desk expanded its program to service onsite and offsite staff, caregivers, and patients. Support for our patients involved not only support for the myPennMedicine patient portal, but also patients seeking to navigate general virus-related coordination around care. To ensure consistent service quality while also rapidly adapting to this new way of assisting our customers, the Service Desk implemented a new quality assurance (QA) program to optimize QA practices to continue providing world-class service.

This program was designed to be more comprehensive and work by using phone and screen recordings and analytics and ticket information to drive continuous improvement initiatives. Program essentials evaluated that the analyst verified the caller before discussing support needs. Agent soft skills were also monitored to gauge their demeanor when interacting with customers. Call management review was instituted to ensure that the analyst demonstrated call control and acted with appropriate responsiveness and urgency when warranted. Confirmation of issue resolution helped verify whether a customer’s issue was resolved and logged with the proper documentation.

Providing a superior customer service experience to our users translated into quicker resolution rates, allowing for positive patient experiences, and enabling our caregivers to return to meet our patients’ medical needs. Additionally, the new program improved analyst productivity and reduced the number of calls that needed to be re-routed or escalated. Monitoring customer service calls also helped to identify training opportunities for our analysts in situations that needed more coaching. The greatest result achieved was seeing an improvement in our first-call resolution rates.

Digitized Processes Brings Value, Supporting a “New Normal” Business Model

Since COVID-19 first struck in early 2020, the remote work model significantly increased, particularly among our corporate services. As a result, our health system needed to optimize business software to maintain office practices while enhancing business process flows through digital channels and mechanisms.

In fiscal year 2021, our IS finance team piloted services with DocuSign to determine if this software was viable in meeting administrative support goals. By FY22, the services had proved useful and created efficiencies that led to expansion of the pilot to other departments including our Office of General Counsel and Purchasing. We gained over 100 users and built the capacity to process over 20,000 electronic transactions annually! Using software technology and new electronic processes proved to be valuable in allowing us to maintain business continuity during the transition to a remote work environment for many departments while increasing operational efficiency.

Security Strategy Sets The Stage For Cyber Protection

As healthcare has become a prime target for cybercrime, it has become more important than ever that executive leaders, risk, and compliance practitioners, along with IT and cybersecurity personnel have a collective understanding of the potential impact that cyber risk can have - not just on the day-to-day operation, but to the very mission of our organization. To meet this problem head-on, Penn Medicine Cybersecurity program has established a clear set of objectives, and long-term goals, in support of our organization’s mission, as well as a 5-year plan of investments to guide our program enhancement over the following years. Having a strategy and roadmap in place aligns our stakeholders, resources, and investments so that all future projects and maturity enhancement initiative are effective, efficient, and support the long-term mission of the organization.

Decide On Risk Appetite

How do we facilitate informed risk decisions?

We provide the organization with accurate and actionable information about our cyber risk position, present courses of action, and record and track progress.

SECURE PARTNERS & WORKFORCE

How do we treat cyber risk that we cannot directly control?

We provide risk oversight and coordination among individuals, teams, and organizations outside of our cybersecurity program to help influence them to movie in alignment with our cyber risk decisions.

Prevent Cyber Incidents

How do we prevent cyber incidents that our reasonably preventable? We implement and operate security technology to prevent threats from disrupting our course, enrich our communities’ experience along the way, and promote operational excellence.

Respond To Active Threats

How do we respond to threats we cannot prevent?

We detect cyber threats, coordinate response, and test our defenses to quickly adapt to emerging hazards and safely and swiftly guide the organization back to normal operations after a cyber incident.

Secure Digital Transformation

How do we integrate security into organizational evolution?

We engage with business partners to understand new journeys and provide security expertise to help chart new courses that integrate the latest advances in cybersecurity technology.

Medical Device Security is Improved through Network Enhancements

The Penn Medicine network was architected as a single network connecting all medical and non-medical devices across the health system. To provide more protection for our medical devices, a new strategic approach was taken to enhance the security of the Penn Medicine network. Access control measures were implemented to ensure that only known, authorized devices could gain access to the network. Separate partitioned network segments were set up for different categories of devices, such as medical, IT, facilities, and so on. Communications between different segments was restricted with controlled exceptions. Some segments, like medical devices, have the minimum access to the internet required for proper functionality (such as access to manufacturer’s maintenance sites). Network-wide monitoring for suspicious trafficking was implemented, and processes were developed to respond to any unusual or suspicious events on the network.

New standard processes were developed to require that all new devices go through a security risk assessment and provide a network profile describing the type of required network connectivity, connections to other Penn Medicine systems, and any connections outside of Penn Medicine before being placed on the network. Additionally, all existing medical devices (5,500+ low-risk medical devices, and 600+ high-risk medical devices) are being moved to the dedicated segment, with the majority having been completed. Penn Medicine worked with the device vendors to ensure that anti-virus software was installed on devices which could support it. A specialized network security tool was also evaluated on the medical devices segment at HUP and PCAM to assess the level of protection it may offer to medical devices that do not offer on-board protection or monitoring (such as anti-virus software).

Network access control was implemented universally across Penn Medicine, ensuring that only authorized devices can access the network. Standard tools and processes were put in place for network security incidents and event management, identifying external threats to the network and providing a playbook for responses to threats. The network was segmented at Penn Medicine’s hospitals and major medical facilities like Perelman Center for Advanced Medicine, Penn Medicine University City, Penn Medicine Rittenhouse, Penn Urgent Care South Philadelphia, and the Instrument Sterilization Center.

These activities have increased Penn Medicine’s ability to respond to threats, enhanced the ability to isolate and limit any threats that do enter the Penn Medicine network, and further protect Penn Medicine’s patient-facing medical devices from external attack.

Reduced Risk of Multi-Factor Authentication Unauthorized Access

Cybersecurity implemented new logic for new user account creation that limits our phone multi-factor authentication self-enrollment window, reducing the risk of unauthorized network access from a potential cyber attacker. The change was implemented without any disruption, using change management procedures that involved coordination with the Service Desk, enhancements to new employee orientation communication, and awareness campaign for existing users.

Due to the change, Cybersecurity significantly reduced the risk associated with the self-enrollment feature, while balancing the business need to allow for employment onboarding processes to complete. In short, enabling the business – securely.

Increased Threat Protection for UPHS Websites

The cybersecurity team worked with numerous web application owners to reconfigure all UPHS public websites and place them behind a layer of advanced defense against cyber-attacks, protecting the organizations’ digital presence, brand, and customer operations. The updates were executed through testing and change management procedures to prevent any disruption to users. Having this layer of defense in place has successfully prevented disruptions from coordinated website disruption attacks against large healthcare that were conducted by cyber-criminals in the first quarter of 2023.

24x7 Phishing Detection Becomes More Advanced

Penn Medicine has established a mature phishing detection, response, and awareness program. With the increased complexity of phishing attacks, and numbers on the rise, our IS security team realized that there was an opportunity to take our phishing detection and response capabilities to the next level, thus ensuring Penn Medicine had round the clock monitoring and response capabilities on reported phishing attempts.

Penn Medicine Cybersecurity recognized that phishing attacks were happening at an increased rate, were growing more dangerous, and needed round the clock monitoring on incoming email-based threats. While the in-house team could track the emails, the amount of time delegated to reviewing phishing emails was also increasing and taking time from other incident triage and management. To better protect the organization and provide timely response activities, Cybersecurity partnered with a 24x7 phishing detection and response service that rapidly receives, assesses, and remediates phishing attacks against Penn Medicine.

All corporate users, representing over 60,000 email accounts, are now able to report suspicious emails and within less than 30 minutes receive a response with the analysis and next steps. During this time, the cybersecurity monitoring team receives necessary information so that appropriate actions can be taken swiftly to protect the organization against emerging email campaign threats.

New Security Protections for Laptops and Off-site Computers

With the increased presence of a growing remote workforce, our cybersecurity team implemented a new security solution that protects Penn Medicine managed laptops, regardless of location – at home, on public Wi-Fi, or while traveling. The laptop internet security solution protects over 10,000 active Penn Medicine managed devices, with a continued effort and process in place to ensure that inactive devices are secured as they re-connect to Penn Medicine networks. By implementing these new protections, prevention and protection is enhanced against cyber criminals trying to deliver attacks through malicious websites.

Integration of Network Standardization for Princeton Hospital

Our enterprise Cybersecurity team worked with key IS teams to re-architect the Princeton network design to integrate with our centralized health system network. The team included the insertion of segmentation firewalls, redirection of all internet traffic through the Penn Medicine firewalls, and migration of legacy Princeton hosted resources to Penn Medicine hosted environments.

By creating the alignment with Princeton’s internet access, Penn Medicine maintained a standard security posture, logging, and monitoring capability. In addition, by bringing internet systems hosted within Princeton into the Penn Medicine environment, the same level of control and security were able to be applied, which greatly reduced the risk associated with a malicious attack.

Business Continuity Assessment is an Ongoing Process

Business continuity planning was developed to ensure the health system could provide ongoing patient care in an unplanned, catastrophic event. Assessing Penn Medicine’s comprehensive business continuity plan has been a continuous process, including periodic reviews of workflow resilience for applications and technology that is used in critical patient care-related areas.

To facilitate this effort, an enterprise-wide clinical care continuity workshop was conducted with senior clinical leadership from the inpatient and ambulatory care settings. Interviews were conducted across all six hospitals, six ambulatory sites, and three data centers to identify the high-level key workflows, applications, and technologies that power critical patient care activities. Additionally, because the business continuity plan crossed over multiple departments, an assessment was conducted against industry best practices for the crisis management and disaster recovery plans.

Based on the Information Systems Audit and Control Association specifications for Penn Medicine, a baseline was obtained to measure the level of maturity for staff awareness, operational processes, resiliency, and recoverability of key systems on the Capability Maturity Model. Once gaps were identified, priority projects were recommended to increase Penn Medicine’s posture in alignment with other academic medical centers. Recommended projects centered on improvements in process resilience, operational governance, and technology resilience. A three-year roadmap was presented to executive leadership for approval, funding, and resourcing for 2022. With a goal of doubling the maturity score, the following projects were selected:

• Relocate the Disaster Recovery Environment for Ensemble to a downtown datacenter

• Improve Citrix resiliency

• Replace aging Telemetry systems at PPMC

• Assess and improve resiliency / redundancy of third-party applications

• Implement a Configuration Management Database (CMDB)

• Conduct a Business Impact Analysis

Maturing Disaster Recovery Fortifies Business Continuity and Infrastructure Resilience

Early in 2021, Penn Medicine assessed the organization’s ability to provide technology that supports ongoing patient care in an unplanned, catastrophic event. In addition, work was done to determine the resilience of workflows, applications, and technology to provide continuity of care for critical patient care-related areas in the face of technical adversity. Led by the Penn Medicine Business Continuity Leadership Committee, IS and partnered services focused on work in two areas.

The first area of focus was the establishment of clinical care continuity workshops that included representation from clinical, IS, and business leadership with the goal of identifying high-level workflows, applications, processes, and technologies – key applications and technologies that power critical patient care activities. In parallel, the second area of focus was a business continuity and disaster recovery (BC/DR) current state assessment across 15 sites including hospitals, a diverse sample of clinics and outpatient centers, and 4 data centers.

In fall 2021, the work completed above resulted in two corresponding deliverables. The first focused on rationalizing and prioritizing the applications that power critical patient care activities. The second focused on presenting a gap analysis and recommendations for improvement of technical footprint (datacenters, infrastructural services). This information was used to help focus on datacenter rationalization and strategic planning toward meeting the organization’s needs.

Wireless Telemetry Monitors Bolster Rehabilitation Services

Penn Medicine replaced the health system’s physiological and telemetry monitors. The new telemetry monitors were installed at HUP and Ravdin 6, as part of a cutting-edge system that uses single to multi-parameter transmitters to provide the ideal tool set for an individual patient’s condition. An IT team was identified to upgrade the new telemetry monitors and incorporate wireless technology, which aligned with the department’s clinical workflow for transporting and monitoring mobile patients.

Benefits of this project included:

• New high-performance bedside/mobile system; comprehensive information that clinicians need, while helping to improve workflow in real time

• Lightweight transport monitors help protect patients, while seamlessly transmitting data into the EHR, leaving no gaps throughout the patient’s stay

• Continuous vital-sign surveillance to traditionally un-monitored, at-risk patients located in medical/ surgical units

• The interface between the telemetry system and the EHR allows us to seamlessly manage, share, and store neurology patient data, thereby improving workflow

Shared Clinical Mobile Devices Upgraded to a Single Platform

The Information Services Platform Engineering group had been supporting two mobile device management (MDM) systems simultaneously. To simplify support, and render immediate cost savings to Penn Medicine, one contract was eliminated by moving to an updated mobile device management system.

All clinical users were provided with detailed step-by-step self-service instructions to make changes to their mobile device. All mobile devices using the legacy system were moved to run under the new system, and the old system was decommissioned. The system now supports over twenty-six thousand mobile devices.

Enterprise-wide Data Center is Consolidated

Penn’s Medicine’s rapid growth resulted in the acquisition and development of twelve independent data centers used to house disaster recovery environments that protect primary data centers in an emergency. Because their growth had been organic and unplanned, they represented multiple inefficiencies to the health system.

Beginning in 2018, analysis of the current data center environment was accomplished. Physical and improved logical footprint designs were calculated by the data center staff and management. Considering the use of virtualization for most systems, and the ability to use high density network and server environments, the plan migrated into action to physically move high density data center systems into the least cost data center environments.

In addition to the retirement of aging systems, system updates and conversions to high density, new racks were also accomplished. As a result, in 2022, data centers evaluated as non-sustaining or too expensive were retired. Those centers that remained viable for future needs of the health system were improved by adding power, wide area network connections, and staff monitoring. Penn Medicine’s Brownstown data center, which supports Lancaster General Hospital, played a critical role in adjusting and improving overall data center efficiencies.

The consolidations resulted in lower building operations costs, a complete overhaul of the entire system’s unit maintenance, and increased data center support. In addition, all data centers housed at entities were removed, allowing for patient focused planning space.

Because Connection is Vital; Nurse Call Gets an Upgrade at HUP Cedar

In a hospital, communication between patients and caregivers is not plainly important, it is vital. The nurse call system acts as a lifeline, providing a secure, reliable platform that scales across the entire hospital. The new system at the Hospital of the University of Pennsylvania-Cedar was selected to replace the aging system in radiology, intensive care, and on the northwest wing’s third, fifth and sixth floors. To support this system, new data cabling was run and two tele-data rooms with new network switches were installed on the fifth and sixth floors. In addition to this, the HVAC and power in the existing tele-data rooms on the third floor were upgraded to bring this into compliance with Penn Medicine standards. The Hospital of the University of Pennsylvania-Cedar now has a reliable, state-of-the-art nurse call system that ensures patient safety and operational efficiency.

Penn Presbyterian Medical Center Upgrades to New Patient Bedside Monitors

Penn Medicine replaced all patient bedside monitors with a high-performance bedside system, throughout the enterprise. One hundred and three new patient bedside monitors were installed in the medical intensive care unit, the stepdown unit, and the critical medicine unit at Penn Presbyterian Medical Center. These new monitors track a patient’s vitals from arrival through discharge. The project team incorporated technology which aligned with the department’s clinical workflow.

The benefits Included:

• Seamless integration of data into PennChart EHR

• Migration to an enterprise standard patient monitoring solution

• Improved performance and reliability of hardware with an active service warranty

• Improved patient care by providing uninterrupted monitoring of patient vitals during patient transport from one unit to the other

• Patient’s information is combined into one patient file

• Increase security by having the patient monitor system run on a more secure operating system

Media Makeover: Audio & Visual Technology Refresh and Recycle Initiatives

The media technology and production team upgraded audio visual (AV) technology rich spaces throughout Penn Medicine and the School of Medicine campus to ensure continued standardization of AV technology across departments and entities. Media technology and production was constantly challenged with supply chain shortages, manufacturer device availability, and lead time push backs (specifically linked to semiconductor microchip manufacturing). The media team evaluated existing spaces that would benefit from being upgraded and found new areas to create. We updated meeting rooms, classrooms, and conference room spaces based on priority as well as the availability of AV technology hardware from various manufacturers.

Spaces were upgraded to provide additional access to Microsoft Teams, Zoom, and Blue Jeans video conferencing as well as soft codec conference call technologies. This helped provide virtual conferencing access to diverse user groups in clinical, research, and educational departments across the health system and campus. The operation and functionality of these AV rich technology spaces is uniform in terms of standards, functionality, and operation; allowing a consistent and user-friendly environment.

Spare and contingent AV equipment was being used in spaces that required video conferencing, video capturing, and video streaming technologies. Public spaces were being used at a greater volume and frequency with the expectation of accommodating a hybrid experience. This refresh program met all users’ expectations, despite the challenge of semiconductor microchip shortages due to the impact from the pademic.

Telecommunications Technologies Upgraded – Returned Significant Cost Benefits

Penn Medicine’s footprint of clinical practices and facilities has become quite expansive and continues to grow. As healthcare business support needs changed, our IS team recognized when opportunities arose to update equipment and technologies that modernized and enabled foundational business practices. Significant changes occurred to the actual administration of healthcare with the onset of the pandemic and continued to evolve with a significantly increased remote non-clinical workforce.

To elevate communication tools used to connect with our patients and serve a greater demand initiated through digital channels, IS designed a new, state of the art integrated call center by merging several independent clinical practices to a single number integrated service. A coordinated effort was also made to create a new station rollout plan, dialing plan, and new circuit delivery, supporting the new hospital Pavilion expansion and reconstruction. Lastly, in support of the expanding group of non-clinical workforce, a new unified Microsoft Teams Dialer solution was rolled out to remote staff. The actions taken by the telephony team resulted in both increased support efficiency, lower total cost of ownership and increased patient satisfaction.

The following highlights are noted below:

• Private clinical practices now fully supported under a centralized call center platform

• Patient satisfaction scores increased by integrating with a centralized call center, lowering call wait times and adding alternative call back feature

• New lower cost circuits and efficient phone routing plans; 30% reduction in circuit cost spend

• 25% increase in phone license quantity and future cost avoidance resulted from Pavilion construction activities and internal phone licensing auditing

• Added new Microsoft Teams dialer functionality as a new remote workforce telephony standard and support model. The new Teams dialer solution resulted in remote staff no longer having to use their own cell phone to make business-related calls. This new functionality is 50% lower in cost than previous solutions and provides a unified, all-in-one communications solution for the remote and hybrid workforce

These fundamental shifts taken to innovate new technologies from traditional communications means easily replaced outdated practices with new, impactful efficiencies.

Keeping the Lights On” is No Small Feat in Maintaining a Large Health System Infrastructure

Penn Medicine’s extensive fleet of computing devices is continuously maintained for optimal performance and tracked for age related hardware and operating system updates to remain in compliance with vendor support requirements. In terms of keeping the computing environment up to date, approximately 25,000 Penn Medicine computers were deployed with the latest Windows operating system. The platform engineering team also worked with members of each hospital entity to remotely upgrade the Citrix workstation environment to the latest release at a 99.8% success rate. Additionally, constant vigilance is applied to identify the most critical updates needed to remediate security vulnerabilities.

Amidst these run-and-maintain activities, a new request was presented to introduce the rollout of mobile devices in the hospital setting. Platform engineering staff worked with members of the new Pavilion’s patient experience team to deploy 500 iPads to be used by patients in their rooms with the goal of increasing their overall satisfaction.

The achievements gained by the platform engineering team have resulted in increased support efficiency, security compliance, and patient satisfaction. Key performance highlights during this reporting period include the following:

• Enhanced security from upgrading machines to receive monthly Microsoft security updates

• Developed a process to regularly upgrade endpoints to the most current supported build. Doing so ensured that no additional endpoints were on unsupported versions of Windows.

• The new Citrix workstation release included advancements in security and stability features and was updated for an upcoming EHR upgrade in 2022.

• By working collaboratively with various entities, the platform engineering team achieved faster upgrades times by 50% over previous upgrades.

• Patients began using the iPads at their bedside, laying the groundwork for these iPads to be used for additional pilots with further integration into the Pavilion patient rooms. Additional use cases assessed include designing a whiteboard app for non-verbal patients. Deployment of a room control application; Deployment of the EHR patient bedside application in coordination with our EHR vendor.

EHRs Need Infrastructure Upgrades and Expansions to Perform Optimally

Supporting and maintaining the growth of the electronic health record (EHR) through resilient infrastructure promotes a stable platform to support our clinical staff in providing world-class care for our patients. Continuous server and storage infrastructure upgrades provided more efficient, powerful, and cost-effective hardware. Operating system and database upgrades also continued to keep the EHR environment running effectively. As the PennChart EHR application teams continued to upgrade and enhance the application for increased functionality and user features, the EHR infrastructure team continued to provide associated growth to maintain and run on a stable infrastructure platform.

Maximizing Infrastructure Efficiencies through Upgrades, Expansion, Consolidation

Supporting Penn Medicine’s consumption of storage and compute services while looking for opportunities to drive down costs is not an easy feat given the system’s size, complexity, and distributed operational model. By maintaining annual asset life-cycle management, it ensured the continual influx of more efficient and costeffective infrastructure to support the growth of the organization. The expansion of the Brownstown data center with new, high-density pods helped add new systems and ongoing consolidation of systems and related infrastructure from Chester County Hospital to the Brownstown data center. Our infrastructure team also continued ongoing work within the new virtual server farm to eliminate older, less efficient technologies across the server, storage, and backup infrastructure. The culmination of these efforts continues to provide a resilient infrastructure for Penn Medicine and promotes the strategy of integrating while driving down cost to the organization.

Network Refresh and Expansions Brought Valuable Efficiencies to Users

As Penn Medicine expanded its service area with new practices and facilities, data usage grew exponentially. As a result, network infrastructure needed to be increased to accommodate the growth. Over 14,000 pieces of network equipment and thousands of miles of cabling required constant refresh to maintain stability, in addition to improving the user experience with our existing support staff.

The network team continually updated the technical environment and increased performance by incorporating advanced technologies and performance efficiencies through design think tank sessions. These improvements were used for new expansion projects such as the Pavilion at HUP and numerous practice acquisitions. They were also incorporated into refresh activities with users unaware of the transformations taking place, such as the replacement of the core fiber backbone that ties together all major locations in Penn Medicine. Other examples included a refresh of the small practice wide area network (WAN) and numerous switch refreshes and software upgrades across Penn Medicine.

The primary benefit that this work brought was an improved user experience. Design integration activities helped to improve user access between Perelman School of Medicine and health system resources. Backbone and WAN redesigns reduced bottlenecks and improved application performance. Redundant power systems and equipment reduced outages and red/blue switch designs reduced the impact of (and in some cases eliminated) outages associated with maintenance activities. By redesigning and renegotiating carrier contracts, Information Services was able to vastly increase network capacity while reducing and avoiding operating costs, adding up to almost $1M per year.