Client Requirement The client required a highly customized application for financial and economic data analysis. This unique tool provided for research and data analysis for economic analysts, portfolio managers, economists, traders and other financial professionals. The web application should also include tools to build custom models and charts for the purpose of comparison of securities, options, commodities, economic and user-uploaded data. The web application should also include appropriate tools to facilitate the ability to create custom data series in a Moving Average or MACD format, blended model weights, correlation studies, lead/lag analysis and performance back testing and calculations. The web application's current offerings include three unique product subscriptions - Equities, Futures & FX and Combined. The product included some striking features for 30,000+ different data series combined with versatile proprietary tools, user-friendly outputs as well as chart saving and sharing features and lots more. For all the calculation based selections, Flex layer was used. (i.e. EMA, SMA or MACD). Java and spring coding was used for the business layer interfaces and implementations. A combination of Spring and Hibernate using Spring Session Factory was used to construct the DAO layer. The application uses different data sources such as Commitments of traders and IVOL and the data for the application is loaded with cron jobs, quartz schedulers and download utilities that are configured along with the application.
Challenges The team faced the following challenges
The website should be able to handle millions of user connections at the same time without affecting performance. To enhance the user's community experience and to accommodate the numerous features for the website, all web usability principles were incorporated. Load balance must be maintained consistently even while executing various simultaneous functions. AJAX based website development. Safe and secure data transfer along with the integration of 3rd party API's, Java script compatibility and cross browser compatibility. Real time updation of content on the site required the development of a sturdy frame.
Technologies Used Operating System & Linux OS, Multi-Server Architecture with Staging & Production Server Management Environment through Version Controlling releases, Server Optimization, Security & SSL Implementation, Scheduler for Backups, Alert Monitoring System Integration, Server Performance Tuning at regular intervals, Software Firewall Configuration & Maintenance Development Environments
J2EE, Spring, Hibernate, BlazeDS Framework, Microsoft Visio, Eclipse Indigo, Java Script, CSS, HTML, SVN etc.
Database
MYSQL, DB Clustering, DB Optimization, High Availability, Master – Slave Replication, Query optimization, Slow Query Optimization, scheduler for backups, Alert Monitoring System integration, etc.
Manpower
Project Leader
1
Developers
4
Designers
2
Quality Assurance Testers
2
Planning Taking into consideration the complexity of the features to be included in the application, a 4 tier development approach was used.
Database Layer - containing MySQL Server Database, Tables, Triggers etc. Data Access Layer – Containing the DLL that is responsible for getting the data from the database. Business Layer – containing the DLL that is responsible for various business logic like Advertisers, Publishers, Banners, Payment Reports, Earning Reports etc. User Interface Layer - This forms the GUI (Graphical User Interface) for the website.
Architecture
Development Highlights The application had three distinct parts: Flex UI, Java Server and the Data sources. The data from various external sources is loaded to the databases using various data scripts. An email reader is used to fetch data from the link. Two cloud server instances are used to host the application with rack space.
The application uses a Tomcat server and is split into two basic parts ROOT and Charting Tool. The ROOT provides the code for the client's website at ROOT web app (/usr/share/apachetomcat-6.0.29/webapps/ROOT) and The services for the charting tool are provided by (/usr/share/apache-tomcat- 6.0.29/webapps/ChartingTool). The spring and Hibernate frameworks are used by the charting tool app for the business logic and database access. Services oriented architecture was used by the application. The Charting Tool exposes multiple services in the Java Services layer. The Flex UI is the origin for all remote method calls and are processed by the Blaze DS Layer. (http://opensource.adobe.com/wiki/display/blazeds/BlazeDS) it utilizes one or more Java services. Calls such as login, post chart, subscribe which originated from the HTML pages are processed by the JSP/Servlet layer which uses the Java Services. The Delegate layer has the business logic and uses the Data Extractor layer to either get Hosted Data from the Elmwood database server (eg. iVol, COT) or use specialized data fetchers and parsers for Non Hosted Data (Internet search trends, FRED, DDF) or fetches data saved on the file system used by the user uploaded data.
Email Reader
Spring scheduling was used to configure the email reader task which is a 4 hourly cron task in the charting tool. IMAPS are used to check the email ID and extract all .xls files for data sources.