IT Governance Business Case Compliance - Police Datacenters Refresh Police Department
Background The Police Department’s four host servers were initially deployed to support the department’s virtual environment in 2014, so they are now 7 years old. During that time, there has been growth in the number of virtual machines (VMs) supported by these hosts from around 25 to now 35. The servers are running on Microsoft Server 2012r2, Microsoft Hyper-V for virtualization, and NetApp devices for storage needs. Server hardware is normally replaced every five years. Currently there is no support on these servers by Cisco the hardware manufacturer. Additionally, there are no other vendors contracted to support the hardware if it fails.
Current Business Problem •
Aging server hardware is more likely to experience outages, leading to costly unplanned downtime for mission critical systems.
•
Police department does not have a current maintenance agreement on host server hardware
•
The current hardware is not adequately supporting the current departmental business needs. The business needs not met is the ability to add more VMs on the host without impacting performance
•
Microsoft’s extended support for the current Operating System (Server 2012r2) will end in 2023. Along with the hardware upgrade, the operating system will need to be updated so that the servers continue receiving Microsoft security patches well into the future. Otherwise, they become more vulnerable to new cyber-attacks and other security issues.
•
The Police department traditionally updated their OS when they replaced their hardware. Currently the Police Department does not have the capacity to update their OS without replacing their hardware.
Current Pain Points • Insufficient memory level of current host servers to allow operation of all VM’s on a single host in the event of an outage or need for maintenance on a partner host without transferring the VMs to the secondary data center. • The current servers no longer fully support Cisco’s management software (used for remote server management above the OS level) due to Adobe Flash end of life. This leads to much less efficient system administration. • Older server operating systems lack the latest enhancements and security upgrades available in newer ones like Server 2019. • The TS Cyber Security program has hardened the servers by putting additionally security on them but, they still have less security features compared to the city hall servers.
Other Related Pain Points based on TS Analysis
• The storage devices will need to be replaced in both data centers within three years. • All Police backups are in their secondary police data center and not in the cloud. Therefore, out of compliance with best practice with having your data backups outside of the city limits.
Desired Business Value City of Durham’s Strategic Plan Goal: Sustainable Natural and Built Environment
Initiative: Evaluate and implement maintenance and replacement plans •
Provide the department with the proper tools such as software and hardware that will help maintain and support the operational needs to accomplish its tasks
•
Upgrade aging hardware to increase efficiency and responsiveness to administrative activities and provide a better environment
•
Reduce risk of software and hardware issues that will result in unplanned downtime such as system outages and security issues for the city
Desired Business Value City of Durham’s Strategic Plan Goal: Innovative & High Performing Organization
Initiative: Develop and implement a continuous improvement model that includes evaluation and process improvement to analyze and improve City services •
Upgrade server hardware or upgrade all data center hardware to include a new backup system to meet the present operational needs, but allow for efficient growth in data center workloads well into the future without additional cost to add those workloads.
•
As better and more efficient IT tools are being implemented, the older products are being phased out. As a result, the customer support is almost non-existent.
•
It is imperative to address any shortcomings in IT systems at the inception, rather than continuing with patch work. It might be a costly fix later if ignored at the beginning. Therefore, the better value is to replace all Police data center hardware in this refresh.
Additional Desired Business Value City of Durham’s Strategic Plan Goal: Innovative & High Performing Organization
Initiative: Develop and implement a continuous improvement model that includes evaluation and process improvement to analyze and improve City services •
Additional TS recommended business value - Replace their primary datacenter storage
•
Provide the Police Department with cloud backup solution
•
Eliminate the secondary datacenter and merge their business resiliency into the City Hall solution (flex central)
•
If all data centers have the same hardware all data center engineers can support each other in emergency situations or when needed. TS senior leadership will also provide guidance and direction to the Police department if, they are on the same platform.
Future State Benefits • Host server has resource levels sufficient to support current operational needs with room for future growth • Prevention of unplanned downtime due to system outages, new cyber-attacks and other security issues. • Host servers that efficiently support both local and remote administration activities. • Maintain at least current level of live replication, allowing all servers in PD virtual environment to be brought up at alternate site within four hours upon loss of service at either Police Headquarters or replication site. • Alternative option is to replace all of the data center hardware with a new backup solution that stores their backups in the cloud and not in another data center within the city limits
Recommended Solutions and Analysis
Solution #1: Nutanix Description
Replace all Police department data center hardware and the primary data center hardware only. Provides a backup solution for the primary data center that pushes the backups into the cloud.
Pros
• Mitigates all risk of data loss by having backups in the cloud • Increased cyber security support oversight • Enterprise conformity increases the capacity of support by internal credentialed engineers
Cons
• Increases 1st year one-time cost
COST ESTIMATES Description
FUNDING Annual Run Costs (Hours)/ KTLO
Year 1 Costs (Hours)
Departmental Funding Amount:
$0
Labor $ (Hours) Internal TS:
60 hours
10 hours
Business Analysis
20 hours
0 hours
TS Services
10 hours
10 hours
TS PMO
30 hours
0 hours
Project Run Cost (External)
$0
170 hours
80 hours
*Total Project Funding Requested for Year 1
$493,038
$0
$0
$0
$0
Hardware: Nutanix
$306,033
$0
Software: Rubrik VM Ware
$136,300 $50,705
$0 $0
$0
$0
Internal Department: Police External:
Professional Services
Funding Requested (Y/N):
Project One Time Costs
*Recurring Costs
Non-Labor $
Other:
*Total Project Funding Requested (External), includes External Year 1 Run Costs and Non-Labor
Y
$493,038
$0
Solution #2: Cisco UCS 220 M5 Server Description
Replace server hardware in the primary and secondary data centers.
Pros
• • • •
Increase in long-term memory level capacity Increased efficiency for wide range of workloads Increased short-term online storage capacity Cisco UCS is a proven reliable platform in existing environment
•
Replacement Storage SAN equipment will need to be replaced in three years Risk of losing police data due to backups being local and stored in the Police secondary data center which is on the same network as the primary data center.
Cons
•
COST ESTIMATES Description
FUNDING Annual Run Costs (Hours)/ KTLO
Year 1 Costs (Hours)
Departmental Funding Amount:
$0
Labor $ (Hours) Internal TS:
60 hours
0 hours
Business Analysis
20 hours
0 hours
TS Services
20 hours
0 hours
Project One Time Costs
TS PMO
20 hours
0 hours
Project Run Cost (External)
$0
200 hours
150 hours
*Total Project Funding Requested for Year 1
$125,663
$3,432
$0
$3,432
$0
Hardware: (CDWG)
$71,580
$0
Software: (CDWG) Server 2019 OS
$38,400
$0
$0
$0
Internal Department: Police External:
Professional Services (CDWG)
Funding Requested (Y/N):
*Recurring Costs
Non-Labor $
Other:
*Total Project Funding Requested (External), includes External Year 1 Run Costs and Non-Labor
Y
$125,663
$0
Recommend Solution #1: Nutanix Description
Pros
Cons
Replace all Police department data center hardware and the primary data center hardware only. Provides a backup solution for the primary data center that pushes the backups into the cloud. • • • •
Mitigates all risk of data loss by having backups in the cloud Increased cyber security support oversight Enterprise conformity increases the capacity of support by internal credentialed engineers Requires 50% less system administration
•
Increases 1st year one-time cost
Risk Identification Description of Risk
Risk Impact
Mitigated (Y/N)
Contingency(Y/N)
Poor execution
Low
Yes
No
Project team turnover
Low
Yes
No
Vendor performance
Low
Yes
No
Cyber Security
Low
Yes
No
Mitigation Risk (Costs) •
No cost
Contingency Risk (Costs) •
No cost
Our Ask
We are asking to approve the business case for the Police Datacenters Refresh for submission to the City Manager for approval and funding through the BMS department.