ISMM1-UC0731001 Introduction to Cloud Computing
Final Project Report Creating the Extensis Environment for NYU Law
Team: Aaron C Dodd | Bryant Hernandez | Brian Yulke | Souksavath Vongbandith | Medina Ali
CONTENTS
Theme & Topic Work Plan Implementation Analysis & Conclusion Appendix
pg. 3 pg. 4 pg. 6 pg. 17 pg. 18
2
THEME & TOPIC Project Topic: Cloud Migration Analysis Specifications: • Cloud Platform Used: Amazon Web Services • Company / Department the project was designed for: NYU Law • Definite Service Implemented: Extensis Font Server Project Goal: Migrating the current Font Server at NYU Law to AWS for enhanced management and security. Project Background:
The Font Server at NYU Law runs on its local network and is solely used by the communications department. About 99% of the department works with Mac OS running the Fonts Servers on them. To meet specific and frequent support needs the department has a 3rd party support group that helps them with Mac-centric issues. Every time the communications department needs support, 3rd party has to be given access to the local network. This indicates possibilities of security risks for which the CIO of the department is uncomfortable with the current process. He has proposed moving the font server to AWS in order to eliminate this security risk. Currently all communications employees authenticate to the server via Active Directory.
3
WORK PLAN Team approached the project by following steps: 1. Analyzing the problem 2. Creating a basic outline on how to approach the technical goals of the project 3. Task distribution / Role Assignment 4. Creating milestones 5. Making technical implementations 6. Updating and project progress 7. Studying the process for Cost Analysis and Technical, Business and User based Impacts 8. Creating presentation documents for the project Role Distribution: Technical implementation of the font server with Amazon AWS: Aaron C Dodd Bryant Hernandez Brian Yulke
Analysis and Presentation Document Preparation: Souksavath Vongbandith Medina Ali Project Cycles: • Update and Record Progress using Google Drive and Emails every week • Meet in person every two weeks to understand and discuss the details of technical implementation
4
Technical Implementation: [SIMPLIFIED OUTLINE] • • • • •
Install Windows Server in an instance on AWS Install the Font Server Figure out how to authenticate with Active Directory Test functions Ultimately give keys to Support Group
Cost Specifics: Item
Cost/Month
EC2 Compute
Extensis web servers
$360.16
EC2 Compute
GlusterFS file servers
$98.10
EBS Volumes
“C:” drive for servers
$10
EBS Volumes
Storage for GlusterFS
$2
EBS Snapshots
Backups for server “C:” drive $14.26
Elastic Load Balancer (ELB) For High Availability
$18.30
ELB traffic
ingress/egress
$0.02
Route53 Hosted Zone
To host the Extensis DNS
$0.50
Route53 usage
Cost for DNS lookups
$0.20
AWS Directory Service
Active Directory for the app
$36.60
Multi-AZ RDS MS SQL
HA database setup instances $1431.06
RDS Storage
Database file itself
$4.60
RDS Backups
Backups of database file
$13.30
Network traffic in
Ingress to Extensis
$0
Network traffic out
Egress from Extensis
$0.18
Total
Description
$1989.28
5
IMPLEMENTATION
Creating the Extensis Environment, All-on-one POC Phase 1: All-on-one Server The steps below outline setting up the Extensis Universal Font Server 6 on Amazon Web Services in the following layout: In the AWS console, select “Services” and “VPC”
Choose the default option “VPC with a Single Public Subnet”
Choose a VPC-wide IP range, VPC name, Subnet IP range, and Subnet name. Leave “tenancy” as default as we do not wish for dedicated hosts, and leave “enable DNS hostnames” checked to allow for inter-machine
6
DNS resolution. “Classic Link” is not needed here, it is for linking the legacy AWS instances (“EC2 Classic”) which do not reside in VPCs to a VPC (the legacy EC2 classic is no longer supported for accounts, only older accounts are grandfathered with the ability to use EC2 classic).
By selecting “Create VPC” the subnet, Internet Gateway, DHCP option sets, network ACLs, and routing tables will be automatically created:
The generated VPC:
7
The generated public subnet:
The generated route table (notice the Internet Gateway was created and 0.0.0.0/0 directs to it, whereas 10.0.0.0/16 for the internal subnet is routed locally):
Creating the EC2 instance:
8
Select the EC2 console. The following screen appears:
Select “Launch Instance” From this screen, choose Windows 2016 Base:
Select an instance size and click “next”. For this, a t2.micro was chosen to avoid costs:
9
Select the newly-created VPC and subnet. Ensure public IP is enabled. Then, click “Next”:
10
Adjust the storage size for the root (C:) volume. Per the design, a 50G C drive will be used. When done, click “Next”:
Tag the instance. At least add a Name tag, which will be a friendly name in the AWS console. If using an enterprise billing system, it is possible to add tags such as “billing-subscription” to all resources for charge-back:
Configure the security group rules. Here, “anywhere” was used, but ideally this would be locked to the external IPs of the corporate LAN, with 3389 locked only to admin workstations. When done, click “Review and Launch”:
11
From the following screen, review the setting and click “Launch” If there is an existing keypair, choose it. For this example, a new one was created:
Be sure to download this, as it is required to decrypt the Windows administrator password. When downloaded, click “Launch Instances”.
12
A screen will appear showing the instance ID. If clicked, the EC2 console will appear showing the launch status:
Wait 10-20 minutes for Windows to fully launch (this time is shorter on an M4 series, but T2 may take some time). You will know when the instance is ready if you click “Actions” and “Get Windows Password” and it no longer says “Password not yet available”. Once the instance is fully launched, click “Actions” and “Get Windows Password”. You will need the keypair created when the instance was launched. Select “Choose file” and choose the keypair:
13
When the keypair has been uploaded, click “Decrypt Password”:
The following output will appear:
14
With these credentials, use Remote Desktop to connect and install Extensis Once logged in I: 1. Opened Internet Explorer 2. Downloaded Extensis Universal Font Server 6 3. Ran the installer Once the installer was finished I: 1. Disabled Windows Firewall (we could enable ports for Extensis but AWS already filters, so I disabled it completely) 2. Opened my browser 3. Surfed: 4. Logged in as: administrator / password 5. Entered the trial license keys: a. UCAU-060E-ALMC-HHHA-PEEL-ZRZQ b. UNAU-060E-AIBA-NCTO-CWEE-HLAC
15
Phase 2: To move to an external RDS, we need to (pending): 1. Open the admin interface on 18081 2. Navigate to Datastore 3. Change the database type to MS SQL 4. (not sure next, we need to put in the RDS info once we’ve created RDS but how do we migrate the existing DB to RDS‌) To set up the share file system (pending): 1. Create EFS endpoint 2. Enable the EC2 security group to access 3. Enable NFS 4.1 on Extensis server. Note: EFS requires NFS 4.1 so Windows 2016 is required since prior versions only supported NFS 4.0 or 3.x. 4. Mount the EFS volume as a drive letter. 5. Migrate the files to EFS 6. Re-point Extensis to the mount-point To enable redundancy (pending): 1. Create an ELB 2. Create an image of the EC2 instance with EFS and RDS configured 3. Create an autoscaling group 4. Create a launch configuration using the new AMI 5. Set autoscaling to 2 instances, tied to the ELB 6. Let autoscaling launch the two new instances
16
ANALYSIS & CONCLUSION Business Impact: • Cost Effective Test for Document Managing System. •
•
High impact on users within department. o Communications department. Allows access to 3rd party support. o No access to internal network required.
•
Proof of concept for future systems.
•
Lower impact of other systems that could be migrated.
Technical Impact: • Easily accessible for 3rd party support without compromise of internal network access. o Virtual Private Cloud •
•
•
Dedicated Bandwidth for higher availability o Failovers for reduced downtime. Security and Audit controls o Access records and logs Does not impact other services if there is a system wide failure.
Conclusion: The migration is an efficient proof of concept for future Cloud Migrations at NYU Law.
17
APPENDIX Shortcut Links to Working Documents & Notes: • • •
https://docs.google.com/document/d/1HS2vXOOzMvPFotGumH1F4NKhw1kIN K11LiXTw0kgbsk/edit# https://docs.google.com/document/d/188b47zgrdcXYZQCFuIDuhSc-y885y9sXFUQO-UtAGs/edit https://docs.google.com/document/d/1RrY7QiB5XN15dUlUC2VSvDmY7jy8gyh ejLzaMQPQYbk/edit#heading=h.n4hhwgckntd6
Architecture Details: Proposed architecture based on vendor hosting requirements
Based on initial review of the documentation, below is the proposed architecture diagram. Excluded from the diagram are the VPC subnets and VPC routing tables. For a simple deployment for the group project we can forgo the “multi-AZ” setup and install one system, but we should show how to host in a highly-available setup. Gliffy export of this is available for adjustments using Gliffy.com.
Explanation:
To achieve High Availability (HA), all components are redundant. There is no built in synchronization of the Extensis Font Server’s storage to multiple nodes, so a shared external file system will be used. Amazon’s Relational Database Service running Microsoft SQL Server in a “multi-AZ” configuration will serve as the Extensis database.
18
19
Extensis Nodes:
The Extensis Font Server operations in a “stateful” manner, therefore only one node can serve traffic for a client (a client cannot connect randomly to an available server). The HA setup will be “active/passive”, with one Extensis server acting as the primary and the second ready for service if needed. The nodes will be part of an Autoscaling Group (AG), which will be set to maintain the current server count. If, for any reason, one node terminates, autoscaling will replace it. Each node will run in a separate Availability Zone (AZ) to account for an outage of any single AZ. This proposal includes two servers running 24x7, with an Elastic Load Balancer handling the traffic. Alternatively, the client can save costs by setting the AG to have 1 node online. If that node is terminated, the client will suffer an outage for the length of time it takes the new node to come online (initial tests show a termination-to-recovery time of about 30 minutes). Load Balancing:
This proposal includes the use of an Amazon Elastic Load Balancer (ELB) in multi-AZ mode to handle directing traffic to the proper backend Extensis server. Due to the stateful nature of Extensis, the client may save costs by removing the ELB and instead relying on Amazon’s Route53 (R53) with custom health checks to resolve a main endpoint (i.e. extensis.client.com) to the current healthy server. Shared File System:
Initially, Amazon’s Elastic Filesystem was investigated. Due to the low storage needs, it would have cost only $0.30 a month for the client. However, despite requiring NFS 4.1 support, which Windows 2016 has, the service isn’t yet compatible with Windows. As an alternative, the open-source software GlusterFS is proposed due to its low system requirements and no license costs, which only adds $96/month to the client’s AWS bill. However, Windows Server may be used for this function as it supports setting up a clustered file system, if the client is willing to pay the additional instance and licensing costs to support Windows on this tier. Two file server nodes are required, with each in a separate AZ to account for an outage of any single AZ.
20
Database:
Amazon’s Relational Database Service (RDS) supports Microsoft SQL Server (MS SQL) and PostgreSQL, both of which are also supported by Extensis. The “proxy failover” documentation states MS SQL is required for failover, so it was priced out here. By using RDS in “Multi-AZ” mode, Amazon keeps one instance active and one instance as standby, with the database replicated in near-real-time between the two. In the event of a failure of the primary RDS instance, Amazon automatically promotes the standby instance to be primary, and begins recovery of the formerly “active” instance. The RDS endpoint, for the connection string in the application, does not change. In testing, failover from the primary to secondary RDS instance was seamless and caused about a two minute delay in processing. Costs for the proposal:
The costs below are for a full HA implementation. The following assumptions are made for the calculation: • Network traffic per month will be 2G out (traffic in is free) • The above network traffic will traverse the load balancer (which has an additional cost for ingress and egress) • Total amount of storage on GlusterFS will be 10G • Size of Windows root volume will be 50G, with snapshots performed weekly for backups totalling 75G each month (snapshots will only be kept for 1 month) • Windows Server license will be provided by Amazon (the client has the option for a cheaper “Bring Your Own License (BYOL)” if they have a Microsoft Enterprise agreement that covers it) • The size of the GlusterFS volume will be 10G • RDS MS SQL Server license will be provided by Amazon (BYOL is available here too) • RDS data transfer will be minimal (within the 1G free tier per month) • Pricing is in on-demand rates, but the client can save ~40% using Reserved Instances • Size of RDS server storage will be 20G with backups performed daily and kept for 7 days • AWS Directory Service will contain 20 active users • All resources run 24/7
Item
Description
Cost/Month
21
EC2 Compute
Extensis web servers
$360.16
EC2 Compute
GlusterFS file servers
$98.10
EBS Volumes
“C:” drive for servers
$10
EBS Volumes
Storage for GlusterFS
$2
EBS Snapshots
Backups for server “C:” drive
$14.26
Elastic Load Balancer (ELB)
For High Availability
$18.30
ELB traffic
ingress/egress
$0.02
Route53 Hosted Zone
To host the Extensis DNS
$0.50
Route53 usage
Cost for DNS lookups
$0.20
AWS Directory Service
Active Directory for the app
$36.60
Multi-AZ RDS MS SQL
HA database setup instances
$1431.06
RDS Storage
Database file itself
$4.60
RDS Backups
Backups of database file
$13.30
Network traffic in
Ingress to Extensis
$0
Network traffic out
Egress from Extensis
$0.18
Total
$1989.28
The AWS Simple Monthly Calculator for the above is saved here: http://calculator.s3.amazonaws.com/index.html#key=calc-9C34CCF1-AC26-47A2A3F3-DFEFAC24B70E
Proposed upgrade logic for Extensis without impacting production:
Due to the flexibility of cloud hosting, a “blue/green” deployment scenario may be used if Extensis requires upgrading in the future. In this model, the main “blue” environment is left untouched while a parallel “green” environment is spun up based on “blue”. The upgrade steps are performed on “green” and the underlying AMI is updated. Once the upgrade is ready for cut-over, a final database and file sync is performed, then DNS is altered to repoint the main endpoint from “blue” to “green”,
22
which becomes the primary environment. The “blue” environment is then spun down to save costs.
Costs for the Blue-Green Environment:
Blue/Green environments would be identical, therefore resources except for Route53, DS, and EFS would be doubled. Assume double the costs if the secondary environment is run for a full month for testing, minus ~$100 for the AWS Directory Service, Route53, and EFS that would be shared. Resources needed for functional demo: Full Stack: • •
AWS VPC Public subnet
23
• • •
• •
•
•
Internet Gateway for Public subnet Elastic Load Balancer (for HA setup, if supported) Elastic Compute Cloud instances for Extensis (at least 1, recommend 2 for HA) • Windows® Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, or Windows Server 2016 • 2.0 GHz or faster multi-core processor • 2 GB RAM • Recommend: m4.large (2 vcpu 8G RAM), t2.medium (2 vcpu 4G RAM) may be possible but t-series is not designed for sustained CPU usage Elastic Compute Cloud instances for GlusterFS (at least 1, recommended 2 for HA) Relational Database Server • Enterprise Edition users have the option of utilizing one of these external databases. Use of an external database is required with a proxy failover configuration. • Microsoft SQL Server 2012 or 2014 (Windows) • Recommend: db.m4.large, 10G database to start Route53 (optional, to host DNS entries for the endpoint, should be considered required if we cannot use an ELB and have to point DNS directly to servers, as with R53 the entries can be quickly updated instead of going through corporate IT) Autoscaling Group • If we can set the instances properly (so the app is installed in the AMI) we should set them in an autoscaling group so if one server gets terminated for any reason (AWS maintenance, etc) it is replaced
Demo Stack:
For the group project, to save costs and still show the functionality, we can set up Extensis with the following resources (removing ELB, shared file servers, and RDS): • AWS VPC • Public subnet • Internet Gateway for Public subnet • 1 Elastic Compute Cloud instances for Extensis • Windows® Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, or Windows Server 2016 • 2.0 GHz or faster multi-core processor • 2 GB RAM • Recommend: t2.medium (2 vcpu 4G RAM) just for demo
24
•
Autoscaling Group • If we can set the instances properly (so the app is installed in the AMI) we should set them in an autoscaling group so if one server gets terminated for any reason (AWS maintenance, etc) it is replaced
Resources needed for Phase 2 (integration, propose we diagram and describe but not implement) • •
AWS Directory Service AWS DS connected to on-site Active Directory (would need domain admin team to assist) • Above assume compatibility, which should be the case per AWS docs. If not, then Phase 2 requires: • VPN Gateway on VPC • VPN connection (configuration as well as setup on corporate IT VPN server) • 2 EC2 instances running domain controller roles in VPC, set to sync with AD in datacenter (at least 1, but we want a fault-tolerant solution) • Note: if the above VPN is needed, we must be sure IP range used in VPC does not conflict with that used in the datacenter
Findings • • • •
•
Setup Administration is http://servername:18081 User/system administration is http://servername:8080 Full list of ports: http://support.extensis.com/Support/58278/58411/enus/Article/View/831/What-network-ports-does-Universal-Type-Server-use Default credentials: • Server Administrator username: administrator • Server Administrator password: password Installer appears to be GUI-based, serial number is needed after installation
Questions: •
Does Extensis support HA setup (more than one app server with same code/repository) and ability to load-balance requests • Answer: yes, in active/passive using a load balancer. Clients will be interrupted during a failover, so any active connections will need to log in again. How seamless this is depends on the client used.
25
•
•
Need to understand better how clients connect. If its over http/s and stateless, the above design may work, if its a custom port/protocol or stateful we may need a different load balancer. This also ties into the client, if it manages the failover. • Answer: It appears clients use HTTP port 8080 for connection, so we should be good with this design Is installer automatable (for installation via user-data on instance start) or do we need to “bake” an AMI • Looks like we’ll just need to install on an instance, snapshot, and make an AMI of it.
Reference documents: •
• •
Installation guide: http://helpdocs.extensis.com/en/universal-type-server/serveradmin/6.1.1/index.htm#02_Installation/00_overview.htm%3FTocPath%3DInstalla tion%2520Overview%7C_____0 Requirements: http://www.extensis.com/downloads/systemrequirements/universal-type-server-6/ Installation video: http://www.extensis.com/font-management/universal-typeserver/resources/ (30 day demo is available)
Next Steps
Note:
All-on-one EC2 instance (with the required VPC, internet gateway, and other settings ste by step set up instructions: https://docs.google.com/a/nyu.edu/document/d/1RrY7QiB5XN15dUlUC2VSvD mY7jy8gyhejLzaMQPQYbk/edit?usp=sharing has the details. To log in & test: Configuration: ec2-54-211-207-180.compute-1.amazonaws.com:18081 administrator password Application admin: ec2-54-211-207-180.compute-1.amazonaws.com:8080 same credentials Remote Desktop: Public DNS ec2-54-211-207-180.compute-1.amazonaws.com User name Administrator
26
Password %iY;E88*aY9vY(4Lq!huZA*RFsacdUFb I'll leave this up for a few days. It just shows we can do this. We should use something other than my account for the demo :)
Based on the above design and findings, I propose an all-on-one EC2 instance to prove that hosting will work. If want to test the proposed HA design, we follow up based on the single-server setup. I don’t think it’s worth using RDS and shared file storage for the group project but we certainly can. For an all-in-one setup, next steps would be: 1. Obtain an AWS account 2. Set up the VPC 3. Set up the public subnet 4. Set up the Internet Gateway 5. Create the security group and access rules 6. Set up Directory Service (if we wish to include AD as part of the demo) 7. Create a Windows EC2 instance, attached to the security group 8. Install and configure the software 9. Create an Elastic Load Balancer OR assign an Elastic IP Address to the instance (ELB would help for following up with a multi-server design, but either way we need a static endpoint user’s can use, either an ELB or EIP) 10. Install the client somewhere and test To test an HA setup, we can reuse the work above and: 1. Create a RDS instance 2. Reconfigure the application to use the RDS 3. Create an EFS mount 4. Copy the files to EFS 5. Create an ELB (if we haven’t) 6. Create an AMI based on our instance 7. Shut down our instance 8. Create an Autoscaling Group tied to the ELB 9. Create an Autoscaling Launch Configuration using our custom AMI 10. Set Autoscaling for a minimum/maximum/desired count of 2 servers (autoscaling will then launch them) 11. Test the application once it’s launched
27