Challenges in cloud based testing

Challenges in Cloud Based Testing

June 12, 2015

Contents  Introduction  Challenges in Cloud Based Testing:  Test Environment setup  Dependency on the Internet connectivity  Security  Performance  Scalability  Infrastructure Challenges  Regression testing issues

 Testing all layers in a cloud

 Case Study  Conclusion

Proprietary & Confidential. ©Aricent Group 2012

2

Introduction •

According to Wikipedia, "cloud testing is a form of software testing in which Web applications that leverage Cloud computing environments ("cloud") seek to simulate real-world user traffic as a means of load testing and stress testing web sites

•

All the leading players like Google, Microsoft, and Amazon etc. are offering their cloud infrastructures for services

•

In 2010, Gartner estimated that “cloud service market will reach $150.1 billion in 2013”.

With so much money being invested in cloud computing, it has become necessary to understand the risk, challenges & issues that cloud computing bring with itself so that the contingency plans are in place to eliminate those issues.

Proprietary & Confidential. ©Aricent Group 2012

3

Challenges in Cloud Testing

Common concerns for Cloud Computing: (Industry Sources)

5

Challenges & Concerns in Cloud Testing •

Test Environment setup There are multiple cloud technologies which support automatic provisioning of the required resources in the cloud but there are no standard supporting solutions to engineers to setup a required test set up on a cloud in a cost effective way.

•

Dependency on the internet connectivity As the application are hosted on the cloud set up, all the testing activities are halted if there are some connectivity issues due to internet

•

Security Any organisation cannot totally depend on the cloud service provider for the security of the data and the QA team need to perform standard set of security testing on cloud based environment

•

Performance With cloud computing there are several patterns of latency to consider, including: UI, local network, public network (Internet), API or middleware, processor, storage, and data, just to focus on the most

Proprietary & Confidential. ©Aricent Group 2012

6

Challenges & Concerns in Cloud Testing (Contd.)

•

Scalability One of the most pertinent issues facing cloud testing in general is the ability of various apps and web objects to effectively scale up or down in order to match the system that they’re running on; the goal being to provide equal (or near equal) access to everyone connected to the cloud

•

Infrastructure Challenges Shifting multiple legacy applications/ functions to cloud often requires a company to rely on multiple vendors. Relying on multiple cloud vendors/platforms can result in a more difficult user experience, requiring training and perhaps resulting in less productivity

•

Regression Testing issues The multi tenancy feature of the cloud may cause a difficulty to apply the existing research work in cloud testing specially for on demand software regression testing service whenever software changes

•

Testing all layers in Cloud When using a cloud service providers it becomes necessary to test all the layers in a cloud like network connections, database, Software applications etc.

Proprietary & Confidential. ©Aricent Group 2012

7

Case Study-Cloud Testing Challenges

Case Study-Project Synopsis • • • • • •

UK Based Service Provider was implementing the M2M (Machine-2-Machine) solution.

Challenge was to meet UAT timelines and physical environment was not ready.

To mitigate the risk & to meet the timelines, Cloud Environment was introduced.

The development and the testing servers were based out of Austin, Texas in USA The testing and development teams were sitting at different places like UK, India.

Project was executed successfully from different geographical location in different time zones.

9

This was one of the There were issues major issues while sometimes in which the configuration changes testing. Sometime the There were major performance issue support was not were getting lost and it throughout the testing phase. Sometimes was needed to again available from the cloud the servers were responding very slowly service provider when make the required Configuration and sometimes there were severs went down and changes. changes memory issues and it became necessary to Unplanned Support some manual increase the memory size which has results outage intervention was in halting all activities (Dev & Testing) for needed immediately. If few hours we don’t have right Support Mechanism The testing activity was (e.g. Mail, Chat & 24*7 impacted due to the on call) then we may unplanned outage end up in phrase “Saving which lasted for 2-3 Server hosted in cloud Penny, Spending $$” days. Such outages are biggest concern for in USA Cloud environment and should be handled carefully in SLA Contracts.

Cloud Env. for M2M Project…

Non availability of cloud environment during testing phase impacted the testing Performance schedule.

Availability

Connected to cloud via secured web VPN

Connectivity from India

Connectivity from UK

Conclusion •

Cloud testing is widely discussed topic in the cloud computing and software community.

•

Cloud based testing approach has its own set of benefits but it definitely comes with some serious challenges. It is necessary to understand the existing challenges before moving to the cloud service providers.

•

Agreed service level agreement with the cloud service providers covering important points like security, performance, availability, support etc. reduces the impact

•

Most innovative testing techniques and solutions & QOS standards are needed to support on demand testing services in a scalable cloud infrastructure.

• Please Note: This overview is just an effort from our side w.r.t. the challenges faced during Cloud Based Testing. We are not here for advocating any particular pros or cons for the cloud based services, rather it’s an experience sharing with the Industry Gurus.

Proprietary & Confidential. ©Aricent Group 2012

11

Thank You!

Proprietary & Confidential. ©Aricent Group 2011 12