Test Magazine September 2015

TEST MAGAZINE SEPTEMBER 2015 VOLUME 7 ISSUE 4

SEPTEMBER 2015

AGILE ASSURANCE BUSINESS ORIENTED TESTING

WWW.TESTINGMAGAZINE.COM

20 LEADING TESTING PROVIDERS

SPECIAL

READY, SET AND BE PREPARED

1

C O N T E N T S

T E S T

M A G A Z I N E

S E C T O R

F O C U S :

|

S E P T E M B E R

2 0 1 5

T E L E C O M M U N I C AT I O N S NEWS

Global Software Testing News 5 EDUCATION

Future Testers

9

THOUGHT LEADERSHIP

Agile Assurance

10

TELECOMS

The Future of Telecoms

14

COVER STORY

The Forces Behind 4G

20

TELECOMS

Business Oriented Testing in the Telecommunications Industry 26 STANDARDISATION

40

Are Standards Stifling Innovation?

30

TEST PROCESS IMPROVEMENT

Ready, Set and Be Prepared 36 ETL TESTING

36

Becoming More Reactive to Change 40 SPECIAL

20 Leading Testing Providers 44

10 30 44 T E S T M a g a z i n e | S e p t e m b e r 2 01 5

3

L E A D E R

INCREASED FOCUS ON QA FOLLOWS HIGH COMPETITION IN TELECOMS

T

he global telecoms sector has seen a trend of service providers offering cloud-based testing solutions, resulting in a growth in the market.1 By moving IT infrastructure to the cloud, telecom companies can cut the capital investment as well as the cost of maintenance, security, and infrastructure.

This is having a knock on effect on the software testers as well.

When the testing takes place over the cloud, then the testers can work independently to reduce the testing time by accessing the tested software on the cloud even in the product development stage.

Today’s customers are fickle. They expect a strong service, faster delivery than ever before and have no qualms about voicing criticism on social media. As the competition increases amongst telecom providers, it’s clear that it’s not enough to bring products quickly to market, they must also invest in QA to retain customer loyalty and maintain market share.

Our September issue takes a deep look at the telecoms sector, considering future trends such as 1.

CECILIA REHN EDITOR TEST MAGAZINE

how far wireless broadband can take us, to an in-depth interview with John Charlton at EE who reveals how setting up and testing the UK’s first 4G network came about, and a discussion on business oriented testing in the telecoms industry from Amdocs.

We are also showcasing articles on ETL testing, standardisation, along with a thought leadership piece on agile assurance from TCS. This issue also concludes with our popular 20 Leading Testing Providers, a comprehensive list of the most relevant software testing vendors on the market today.

Finally, I must draw your attention to our new design; it’s been a couple of months’ in the making. The same TEST Magazine you know and love, but with a different look. Any thoughts on the new style, content or if you just want to say hi, please don’t hesitate to get in touch!

© 2015 31 Media Limited. All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor of TEST Magazine or its publisher, 31 Media Limited.

ISSN 2040-01-60 EDITOR Cecilia Rehn cecilia.rehn@31media.co.uk +44 (0)203 056 4599 ADVERTISING ENQUIRIES Anna Chubb anna.chubb@31media.co.uk +44 (0)203 056 6945 PRODUCTION & DESIGN JJ Jordan jj@31media.co.uk 31 Media Ltd, 41-42 Daisy Business Park 19-35 Sylvan Grove London, SE15 1PD +44 (0)870 863 6930 info@31media.co.uk www.testingmagazine.com PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA @testmagazine

cecilia.rehn@31media.co.uk

TEST Magazine Group

TechNavio, ‘Global Software Testing Services Market in the Telecom Sector 2014-2018’ (October 2014).

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Get the UX factor Put the user at the heart of your testing Ensure the customer remains at the heart of your software development, and your organisation stays ahead of the competition.

Our UX certification combines an exam and portfolio assessment to deliver valuable, practical skills.

Š BCS, The Chartered Institute for IT, is the business name of The British Computer Society (Registered charity no. 292786) 2015

BC1245/LD/AD/0615

bcs.org/ux

S O F T W A R E

T E S T I N G

N E W S

5

UK NUMBER ONE, AS EUROPEAN TECH SPENDING GROWS According to Forrester Research, technology spending in Europe is expected to hit 5% this year, up 0.9 percentage points on last year.

The best growth, at over 5%, will come from the Nordic countries, particularly Sweden, along with Ireland and the UK. France, Germany, Italy, Spain and the Benelux countries will see a more moderate rise in technology spending, with growth of between 1 and 3.5%.

Interestingly, most Eurozone countries will see technology spending rise by between 1 and 2% this year, while non-euro countries are by and large seeing higher rates.

Currently, Europe's technology market remains larger than Asia Pacific's but is only around two-thirds the size of America's. Measured in euro terms, businesses and governments in the Americas will spend €1,105 billion on technology purchases in 2015, compared with €684 billion in Western and Central Europe and €512 billion in Asia Pacific. According the research company, software purchases will be "ahead of the pack", growing by a more than a respectable 7.4 % in 2015. Spending on technology consulting and systems integration services is expected to rise by 7% over the course of this year, marking a strong area of growth. This growth is a result of the increases in spending on mobile apps, analytics and big data, and customer-oriented applications, says Forrester, along with the related services for helping firms select, implement, and "maximise value from these software products".

Additionally, the outsourcing sector is pegged to grow nearly as fast as the overall market, with continuing economic weakness in the Eurozone fostering an expansion of demand from the UK into other countries.

UNMANNED AIR TRAFFIC CONTROL TOWERS GROWING IN POPULARITY Since April 2015, Sweden’s Örnsköldsvik airport has employed a remote system – which uses cameras and sensors – in its air traffic control tower. The same system has just been rolled out in Leesburg Executive Airport in Virginia, USA. And according to the UK’s National Air Traffic Service (Nats), the technology could change air traffic services as we know them.

Designed by Swedish defence firm Saab, the remote system includes 14 high-definition cameras and sensors that can see aeroplanes in all weather conditions.

Currently, at Örnsköldsvik, the planes are controlled by a person sitting 90 miles (144 km) away at a different airport, Sunsvall, which is due to install the same system later this year.

Speaking to the press, a spokesman for Saab said the technology could be a huge benefit to air traffic control,

SQS SOFTWARE QUALITY SYSTEMS AG BUYS US-BASED SOFTWARE TESTING CONSULTANCY SQS Software Quality Systems AG has reached a conditional agreement to buy US-based software testing consultancy Galmont Consulting LLC for up to US$22.0 million in cash and shares.

The acquisition also will boost and diversify SQS's revenue base by materially expanding its US business.

reducing costs as small airports could pool controllers.

The technology does a better job than humans, he said. “The cameras and sensors pick up and see aircraft in any environment – in fog, rain and the dark.

It is better than the human eye.”

There is also a military use for the system. Pop-up airports that the military could deploy in war zones are now possible, by attaching the camera infrastructure to trucks rather than towers.

Saab said that interest in the system is increasing, with some major airports considering installing remote towers as back-ups and added that the company is in talks with UK airports.

The UK’s Nats said that it is in discussions with a number of manufacturers about offering the service in the UK. “The introduction of remote control towers is one of the most exciting technological developments in the history of our industry,” said Nats General Manager of Operations Paul Jones.

SQS will pay an initial US$3.5 million in cash and the same amount in shares for Galmont. The issue will comprise 375,134 shares in SQS priced at 598 pence per share.

"This is SQS's second acquisition in the US and follows the successful integration of US-based management consultancy business Trissential, announced in April. The acquisition of Galmont, which has built an outstanding and established reputation, significantly expands our presence in this large and growing market for us," said Diederik Vos, SQS's Chief Executive.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

6

S O F T W A R E

T E S T I N G

N E W S

NEW ACCENTURE AND SOASTA ALLIANCE Accenture and SOASTA are forming an alliance to provide faster and more reliable digital performance management solutions to clients.

In addition, Accenture has made a minority investment in SOASTA. Combining Accenture’s capabilities with SOASTA’s tools will allow clients to increase end-to-end performance of their web sites and mobile applications while delivering exceptional customer experiences.

This ability to monitor customer experience in real time can help predict and solve performance problems from virtually any location and on any device around the world. This approach can provide significant business visibility and drive revenue impact, such as in online retail where even momentary latency can result in significant drops in page views, shopping cart abandonment and ultimately sales conversion.

“In the hyper-competitive online marketplace where performance is critical and milliseconds can equal millions, providing a better customer experience is the only way to win,” said Tom Lounibos, CEO and Co-Founder, SOASTA. “We’re thrilled to have established an investment and alliance agreement with Accenture, enabling us to provide even more business value to our joint clients by offering actionable intelligence that can help them quickly understand what’s working, what isn’t and what to do next.”

“Businesses are increasingly dependent on software and, therefore, on software performance,” said Kishore Durg, Managing Director and Global Testing Lead for Accenture.

“Our alliance with SOASTA provides a step-change improvement in our performance diagnostic and monitoring capabilities, which are critical for today’s digital business.

SOASTA is one example of how Accenture is advancing performance engineering and testing by incorporating the latest analytics tools and intelligent automation into our service delivery.”

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

www.softwaretestingnews.co.uk Featuring stimulating, intriguing articles and features from experienced software testers and leading vendors, you can be sure that you will stay up-to-date with the software testing industry.

THE UK’S FIRST 4G-CAPABLE DEDICATED M2M PLATFORM

read more online

The UK’s largest digital communications company EE, has launched EE Connect, the UK’s first 4G-capable dedicated Machine to Machine (M2M) service. EE Connect provides the foundation for businesses and public sector organisations to develop the connected products and services of the future on 4G, 3G and 2G network.

INDIA UNVEILS ITS 98 SMART CITIES OF THE FUTURE

read more online

India’s Urban Development Minister M. Venkaiah Naidu has announced the list of 98 cities nominated for the Smart City Mission, which will see 130 million citizens covered under the initiative. “There are 13 cities from Uttar Pradesh, 12 from Tamil Nadu, 10 from Maharashtra, seven from Madhya Pradesh, three each from Bihar and Andhra Pradesh included in the list,” said Naidu.

SPEED OF TECHNICAL CHANGE POSING PROBLEMS FOR BUSINESS, WORLD QUALITY REPORT 2015 NOTES

read more online

The annual World Quality Report, published

by Capgemini and Sogeti, its local professional services subsidiary in conjunction with HP, observes the state of application quality and testing practices across multiple industries and geographies.

Automation that makes sense to your Test Engineer… and your Grandma.

Tricentis Tosca Testsuite’s Model Based Test Automation Scan. Arrange. Test. That’s all it takes.

www.tricentis.com/MBTA

8

S O F T W A R E

T E S T I N G

N E W S

INDIA AND USA JOIN TO COMBAT CYBER CRIME

NEW SMART CONNECTED POE LIGHTING ON THE MARKET One of Europe’s first true Power over Ethernet (PoE) lighting installation will be unveiled early October with a public demonstration of how smart connected lighting can deliver massive operational and installation cost savings.

The technology, which promises up to 75% energy savings in commercial and public installations,

by British tech company amBX. Working alongside Cisco, the company has developed software that delivers high performance lighting control without the need for costly programming and complex technical interfaces.

Smart connected lighting control will mean that lighting systems will become much quicker and easier to install and faster and richer in configuration. amBX technology works by controlling the lighting in a single space or multiple spaces based around end-user needs: environmental factors such as levels of daylight, time of day, temperature, whether the space is in use or not and any other measureable parameter. The system software is then able to take account of all these factors and produce, in real time, the best implementation possible of the desired outcome using the light

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

fittings that are available. In effect the software produces the perfect virtual implementation of the desired outcome and then projects the best representation possible of that in the real world using the lighting available.

“Complex programming has long been the Achilles heel of lighting control. The amBX system supports an extremely wide range of ways to interact with lighting making it easy to manage both simple and sophisticated lighting experiences.

Controlling lighting using PoE lights in this way will enable building owners and users to use richer, bio-adaptive lighting in new ways to create healthier, more productive workspaces and environments,” explains Neil Macdonald, COO, amBX. “This will lead to better humancentric lighting at desks, in meeting rooms and public spaces, in terms of intensity and colour changes,” Macdonald added.

Although transferable to other sectors and services, development of the amBX technology has been primarily concentrated on the commercial lighting market. It has been developed into control systems that address three key sectors of the global lighting market: Connected lighting (lighting connected on ethernet based networks), Commercial lighting (lighting projects, installations and systems), and Bio-adaptive lighting (human-centric lighting experience).

India and the US, faced with evolving cyber challenges, have decided to join hands in combatting cyber crime.

"Cyber security is fundamentally a team endeavor, and it is essential that international partners like India and the US work together closely, along with industry and civil society, to raise our cyber defences in both the short and long term, to disrupt and interrupt malicious actors in cyberspace, and to improve our ability to respond to and recover from cyber threats," Michael Daniel, Special Assistant to the President and Cybersecurity Co-ordinator, said.

"We are hopeful that the governments and industries from both the countries can work together to chart the way forward for a successful US-India partnership in responding to the evolving cyber challenges," said Arvind Gupta, India’s Deputy National Security Advisor.

The two delegations identified a variety of opportunities for increased collaboration on cyber security capacity-building, cyber security research and development, combatting cybercrime, international security, and internet governance, and intend to pursue an array of follow-on activities to bolster their cyber security partnership and achieve concrete outcomes.

The two countries decided to hold the next round of the Cyber Dialogue in Delhi in 2016.

The US is also currently engaging in cyber security discussions with China, a topic that has become a point of friction in US-Chinese relations.

9

E D U C A T I O N

FUTURE TESTERS New software engineering degree programme with ample focus on software testing at Cardiff University

A

National Software Academy, set up to train and educate the next generation of software engineers in Wales, has been launched this autumn by Cardiff University and Welsh government.

Run in partnership with incubator Alacrity Foundation in Newport, the three-year degree programme (BSc Applied Software Engineering) has been set up to address the current unmet demand for skilled software engineers in Wales.

Cardiff University

Dr Wendy Ivins, Director for the new programme and a lecturer at Cardiff University, spoke exclusively with TEST Magazine about the new programme’s focus on software testing.

Graduate students in Cardiff will learn that testing is not an afterthought.

Its importance will be emphasised throughout the three year course, with students in the first year working in small teams to review code, learn to recognise errors and debug programs.

By the second semester automated testing, behaviour driven testing and error reporting protocols will be taught.

“Instead of being a research-driven course, like many traditional computer

science degrees, it’s a practice-driven curricula,” Dr Ivins said. “Testing is embedded. It’s a key skill.”

DR WENDY IVINS DIRECTOR NATIONAL SOFTWARE ACADEMY CARDIFF UNIVERSITY

Dr Wendy Ivins has been a Lecturer at Cardiff University since 2000, and has a background in testing. She currently

SOFTWARE TESTING COVERAGE IN THE CURRICULUM

teaches software engineering, information systems management at The School of Computer Science.

Research interests are in computer

Indeed, the National Software Academy was set up to work closely with industry, meaning graduates will work on ‘real life’ projects throughout their studies under the mentorship of experienced software engineers from industry.

By the time the students have finished the second year, their software testing skills should be more applied, with knowledge of DevOps, automated deployment, test frameworks, security and penetration testing.

“In the third year, the students will be working on large projects with multiple teams, to emulate real life,” Dr Ivins said. “We will expect the students to take ownership of an entire project, and that includes taking ownership of the software quality as well.”

Third year students will be exposed to approaches such as test stubs and carry out more functional testing.

“The goal is for the students not just to understand the concept behind testing, but experience key practice,” Dr Ivins concluded.

INDUSTRY INVOLVEMENT The course is designed to give the students practical skills, and they

support for co-ordinating distributed teamwork.

will be working with professional tools throughout. “We will be guided by industry in terms of what tools and programmes to use,” said Matthew Turner, Project Manager at the Software Academy. “There will be regular chances for industry partners to outline their needs and in partnership they can help keep the course up to date.”

The Academy is actively courting industry partners to help shape the course, based on their requirements and input. There are opportunities for guest lecture spots, mentoring and internships.

“We’re talking to all companies: startups, big software firms, public institutions with large ICT departments. We want to expose our students to all facets of the software industry,” Turner added.

There is some talk of expanding the programme, which for this academic year will see 20-25 students enrolling full time. A master’s programme might be in the future.

Organisations interested in getting involved in the degree programme can contact Matthew Turner (TurnerM1@cardiff.ac.uk).

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

AGILE ASSURANCE: TO DO OR NOT TO DO, THAT IS NOT THE QUESTION Siva Ganesan, Vice President and Global Head of Assurance Services, Tata Consultancy Services, addresses the agile revolution and how it has become de rigueur in assurance.

He brings in a dimension whereby he goes beyond ‘why’ and proposes ‘how’ to best embrace agile assurance.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

T H O U G H T

11

L E A D E R S H I P

R

ecently TCS held the Grand Finale of TESTimony, its Software Assurance contest for students. The guest of honour at the event spoke about how software development is now becoming a true engineering discipline with its move towards in‑process quality assurance rather than quality control, a reactive model that served us well during the times of manual testing in legacy systems.

In recent years we’ve seen the model shift left to prevention rather than only cure. Today, when IT departments and organisations are under pressure to rapidly deliver projects and the first‑time‑right mentality prevails; agile assurance is revolutionising the QA industry.

As we all tackle today’s reality, it is time we move beyond the debate of should we focus on agile assurance and instead discuss best practices for implementation. This article will examine the expectations of agile assurance and introduce some clear steps to take towards successful implementation and achieving speed‑to‑market without jeopardising quality.

SPEED‑TO‑MARKET Assurance is about enabling teams to deliver speed‑to‑market without compromising quality; agility is used to emphasise the velocity

of delivering feature functionality and engineering. However, there is a dichotomy at play here: quality needs to be balanced with agility and velocity. Thus, there has to be an emphasis on quick, first‑time‑right delivery, but this cannot compromise quality. Indeed, assurance needs to be integrated into the agile software development lifecycle in a fluid manner without compromising on the discipline that robust engineering demands.

Notably, implementing assurance to reduce the overall speed‑to‑market stretches beyond the assurance cycle. TCS’ Digital Test Value Chain™ means organisations can automate the entire value chain, including test design and mechanising overall testing activities in a true factory model, to increase speed. Additionally, there is a new focus on the overlap between the Development‑Assurance‑Operations team, as with release automation.

In the DevOps delivery model, automation across the value chain, from assurance to release is highly important. Considering the marketplace in 2015, for channels with consumer‑facing objectives, agile is the way to go.

The transition to an agile assurance methodology is all about the implementing the different types of testing tools, and realising how they all come together to deliver a superior automation experience.

THE REVOLUTION IS TOO FAST TO BE TELEVISED Today, the default is near‑instantaneous delivery. There is almost a sense of urgency amongst this growing customer base, which now expects the best‑of‑best features and performance dispensed in high velocity cycles.

The software testing community is vibrantly adopting this revolution and can see how it builds upon the advances in test automation.

They are accepting the new reality: requirements and design are not always in place; builds are no longer monolithic and long drawn but more incremental. There is a high refresh rate. Agile assurance is a natural consequence of this evolution.

SIVA GANESAN VICE PRESIDENT AND GLOBAL HEAD OF ASSURANCE SERVICES TATA CONSULTANCY SERVICES

Siva runs the Assurance Services Unit (ASU) business for Tata Consultancy Services (TCS), which serves a large, diversified customer base globally. With over 23 years in the IT industry, Siva has considerable experience in building relationships ground up and helping customers unlock tremendous value from their existing testing estate.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

12

Testing needs to happen near real‑time, to ensure quality products are delivered, in a real time, agile manner.

The context of agile assurance being adopted to legacy systems is also underway. That said, systems developed in the last century have their own set of imperatives and priorities which call for agile assurance to adapt to this stack in a different manner.

HOW TO IMPLEMENT AGILE ASSURANCE For organisations looking to adopt agile assurance, there are several points to consider: • Business outcomes: Consultants can demonstrate how outcomes will be affected by a change to an agile assurance methodology.

• Metrics: There is a different/ complementary set of measures that come in the agile world, and organisations need to be preparing to manage these.

• Organisational change management: There has to be a certain amount of retooling in the ecosystem to adapt to agile assurance.

• Stakeholder management: Unlike the testing world of the past, agile assurance engineers work across the full value chain.

• Technology/retooling: Before, testing tools were mainly for test script automation. Today, there is more sophistication around virtualisation, test environment creation, automation, provisioning, test data fabrication, and test execution or test automation in general.

• Experience: How they can deliver an improved consumer experience.

The transition to an agile assurance methodology is all about implementing the different types of testing tools, and realising how they all come together to deliver a superior automation experience.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

T H O U G H T

NOT A BLOCKER, BUT AN ENABLER TCS can highlight some channels of engagements where we have been able to shorten the launch time of systems by 30 ‑ 40% by adopting agile assurance methodologies.

For one large healthcare provider in the US and an online retailer in the UK, testing evolved into assurance. This meant that testing was never a blocker for launching software, but instead an enabler.

Speed‑to‑market was achieved without compromising on quality, as the assurance objective was a clearly defined priority right from the start.

TO DO OR NOT TO DO, THAT IS NOT THE QUESTION Speed‑to‑market is non‑negotiable today and adopting agile assurance is not the question; it is instead, how fast can you implement it and satisfy the challenging demands required by today’s marketplace? Why not go down the route of agile to ensure first‑time‑right delivery?

Finally, agile assurance is not isolated to deliver outcomes for the IT department. By positioning assurance as a project enabler, there will be faster delivery of a high quality project, ultimately enhancing the consumer experience and positively affecting a business’ bottom line.

Reference 1.

‘Code Improves Code, and Certifies Certainty’, http://sites.tcs.com/blogs/Think-Assurance/ code-improves-code-certifies-certainty.

L E A D E R S H I P

Code tests code There is another revolution underway, in terms of the role of the software tester.1 Now that we’ve moved away from the days of predominantly manual testing and unhurried software releases, to today’s test automation and continuous updates, testers are evolving. Testers should now be considered software professionals, who can, and must, craft software in order to test and assure software.

For test automation to go beyond mere test data, we need efficient scripts. This means only code can improve code.

Testing is about architecture, instrumentation, validation, verification, certification, and structural quality. And testing is also about coding the right testware – testware that certifies certainty – and makes software perform as expected – anytime, anywhere.

THE FUTURE OF TELECOMS Richard Jones, Partner - Ventura Team LLP, reports on the future trends in the global telecoms industry.

T

elecoms is a broad and exciting field. Work by the McKinsey Global Institute suggests that cloud, mobile internet, the automation of knowledge work and the Internet of Things will be four of the top five disruptive technologies by economic impact by 2025, with revenues in the trillions of dollars.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

SUBSCRIBERS WANT FASTER BROADBAND In work for the FTTH Council Europe, my colleagues looked at the evidence behind Neilsen’s Law. This ‘law’ states that the fastest residential services in a country will

double in speed every two to three years.

Figure 1 shows some best fit examples of this from the real world which support the law’s validity up to today.

Residential broadband services as fast as 2 Gbps (Hong Kong) are now available and, with Ultra

15

T E L E C O M S

High-Def TV having 16 times the pixels (and therefore bandwidth requirements) compared to a current generation HD TV, there is no reason to believe Neilsen’s Law will stop being valid.

Fibre to the Home (FTTH) is currently the ultimate technology for delivering broadband and you could download the whole of the Library of Congress in 2.5 seconds at the current speed limits over fibre achieved in a lab. It would take a little longer over poor ADSL out in rural Devon!

It’s worth nothing that individual countries may be constrained not by demand, but by a lack of investment in improving infrastructure from traditional copper to fibre. For example, vectored VDSL2+ uses the last link of copper to homes but only achieves just over 100 Mbps. A superior technology, G.Fast can hit 600 Mbps, but it requires serious investment in extending fibre ever closer to homes.

In Africa, the problem is that there is little copper infrastructure so communications in the continent is generally powered by mobile.

However, pockets of faster speeds are emerging and my colleagues recently launched services in South Africa that offer the first residential, gigabit broadband service in the continent (we believe).

Wherever a country is in terms of speeds today, Neilsens’s Law underlines the simple fact that subscribers want faster broadband and will continue to do so.

WHAT WILL THEY PAY FOR FASTER BROADBAND? The chart in Figure 2a shows how users are paying slightly higher prices for much more broadband speed.1 The chart in Figure 2b underlines that users will pay more for increased speeds but not a huge premium.2 This is not great news

Figure 1. Examples of the evolution of fastest residential services by country (Data from Ventura Team LLP).

for some operators and I would argue that mobile only players in competitive markets will exist in five years. There is only limited upside in maintaining the status quo in mobile broadband. LTE illustrates how operators have increased capex on top of what is fundamentally a doomed business model. The impact of data volumes exploding against flat/falling ARPU’s is another discussion entirely, but once all parties have the ‘new thing’ then there is no differentiation created by it.

Telecoms is also a key driver of economic development: a 10% increase in mobile coverage can lead to a 1% increase in GDP. Doubling the average broadband speed adds 0.3% to GDP and doubling again nets the same benefit.

YOU CAN’T BUILD FIBRE EVERYWHERE The charts in Figure 3a and 3b estimate the cost for doing GPON and P2P based FTTH as well as FTTC in different cities in Sweden (used as an example). The denser cities show lower costs per home passed than smaller towns and villages. The chart on the right synthesises this data to show how the incremental cost of covering from 0 to 100% of the population increases substantially as you leave the more dense areas.

An interesting piece we did for the FTTH Council Europe shows a lack of available equity funding for FTTH projects. It outlined alternative means for project funding but also estimated that at the current investment rate in fibre, it would take 92 years for Europe to be fibred completely.3

RICHARD JONES PARTNER VENTURA TEAM LLP

Richard Jones co-founded Ventura Team, a leading telecoms consultancy that has led projects deploying national broadband networks and creating the first gigabit incumbent operator. He is Chief Commercial Officer and co-founder of Ventura Next which manages fibre networks in developments and smart cities and has helped create the first gigabit services in Africa.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

16

T E L E C O M S

Figure 2. The premium for faster connections lags the speed increase provided.

YOU CAN’T BUILD EVERYWHERE IN A TOWN OR CITY In work carried out in Saudi Arabia as an example, the difference in capex costs for the same number of homes passed can vary by 150% from area to area (Figure 4). This models a number of costs for areas of different density with point A representing an average density area and B showing the capex at an area of 90% occupation of apartments. The business case for FTTH is generally very challenging.

The simple rule is that FTTH will only be built in widespread deployments where some of the following are true: • Broadband/TV revenues are high.

• There are new developments – the business case for FTTH in greenfield developments, smart cities etc. is a no brainer.

• Countries have taken a political decision (e.g. UAE, Qatar).

• The number of people in multi-dwelling units is very high (leading to far lower capex compared to where most people live in individual homes).

• Municipalities have invested in fibre networks already (e.g. much of Sweden).

• Dig costs are very low and/or aerial fibre can be deployed (the latter explaining Verizon’s low

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Figure 2a. Some will pay a premium for extra speed.

cost per home passed).

• There is a very high availability of ducts that can be re-used at low cost.

WIRELESS WILL ALWAYS HAVE A BIG ROLE TO PLAY IN FUTURE BROADBAND DEPLOYMENTS You can view broadband speed in countries like cross sections of a mountain. The peak will be those with fast fibre descending through increasingly broad areas with G.Fast, vectored VDSL2+, unvectored VDSL and down to ADSL. Mobile will be additional to this (where these fixed services offer reasonable throughputs) but will be the main offering in rural areas and white spaces (places with no coverage).

WIRELESS BROADBAND IS IMPORTANT BUT THERE IS NOT ENOUGH SPECTRUM TO PLAY WITH There is only a finite amount that can be done with mobile before you start needing to break the laws of physics.

Even refarming spectrum (taking it say from analogue TV and using it for mobile), improved antennae, smaller cell sizes and the evolution of

LTE are not going to be enough. The availability of spectrum to support the explosion of data usage (e.g. a 40 fold increase in the GCC countries between 2008 and 2013) is simply too limited.

A quick back of envelope calculation suggests that in some urban areas, the limitations of spectrum will cap the average user at 30 GB of download per month.

That might seem high where mobile complements fixed broadband but wherever it acts as the main source of broadband, then usage will be higher. WiMAX customers in Jordan were already at 20 GB average per month five years ago with one customer hitting 200 GB.

SMART CITIES AND THE INTERNET OF THINGS Sometimes when I’m building a new telecom business model, I try to think about what really is five or 10 years out. The challenge is that interesting things will happen along the way and revenues appear from some unexpected quarters.

The Internet of Things is a little like that. The idea that lots of devices could be connected together has existed for years. You can even see a real kitchen computer workstation designed for the smart home… in the 1960s! However, we’re still very much in the phoney war stages

Believe it or not, Pete in Q&A loves his functional testing kit

Shame the deployment failed anyway...

I T R Y N E G Y A D V

Virtual test networks are the smart way

Mimic every perceivable network

to manage the risks of deploying critical

topology and scenario. Tweak and

applications into potentially hostile

repeat.

network environments. iTrinegy is the leading provider of networked application risk management solutions, trusted by governments, military organisations and enterprises around the world.

Contact us today to arrange an online demonstration or trial ( + 44) 01799 2 52 2 00

W WW.IT RINEGY.CO M

INFO @ITRINEGY.CO M

18

T E L E C O M S

Figure 3a. Passive infrastructure capex (GBP) per home passed by geotype and technology for Sweden.

Figure 3b.The data in Figure 3a has been synthesised to show how the incremental cost of covering from 0 to 100% of the population increases substantially as you leave the more dense areas.

Figure 4. Percentage of population in multi-dwelling units.

and the challenges of security and cost are still tricky. Do you really need to control your lights at home using WiFi when you’re not there? Over time, applications and services that really do capture our imagination will appear and be perfected. For the moment, we’re still in that ‘it’s clever but why do I need it’ phase with many services.

There is a greater question in the telecoms world that covers IoT and also smart cities. There is a difference between doing something to jump on a bandwagon and actually doing something because it’s a committed part of your strategy. I know of 300 different applications for smart cities built around a core of a (preferably) fibre telcoms nervous system. Not many of them actually

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

make a difference. Further than that, some smart cities just call themselves smart for marketing purposes rather than commit to doing anything even remotely smart.

As with many things, telecoms in the future will be a battle between reality and marketing hype. For those on ‘fibre’ broadband in the UK, you perhaps don’t realise that you’re not getting true fibre to the home. Instead, the presence of fibre in the network allows operators to make this claim and the ASA has failed to correct this shall we say, misleading perspective.

FINAL THOUGHTS Obviously people love what they can do with telecoms. We stream

Netflix, stay in touch on WhatsApp and occasionally even make the odd phone call. However, telecoms is also a key driver of economic development. A 10% increase in mobile coverage can lead to a 1% increase in GDP. Doubling the average broadband speed adds 0.3% to GDP and doubling again nets the same benefit.. again! Another study suggests a 1.1% increase in GDP for a 10% increase in broadband penetration.

If you look at Australia’s attempt to accrue some of these benefits with its National Broadband Network, you can see an abject lesson in success or failure – depending on which shouting politician you choose to listen to. UAE, Qatar and Oman have recognised the need to make investments into National Broadband Networks as it is an important area and governments need to work on ensuring policies support delivering services to less dense and poorer areas – not just the wealthier cities and towns.

References 1.

2.

3.

Ventura Team LLP, Point Topic, Detecon.

Ofcom.

‘Financing Stimulus for FTTH’, FTTH Council Europe, http://www.ftthcouncil.eu/documents/ Reports/FTTH_Finance_Report.pdf

THE FO

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

T E L E C O M M U N I C A T I O N S

/

C O V E R

S T O R Y

21

RCES BEHIND 4G In 2010, Orange and T-Mobile were brought together and EE was born. Within two years, the new company had launched the EE brand and the UK’s first 4G network.

Today EE is Europe’s largest 4G operator with over 10 million customers, whilst its 3G network covers 98% of the UK population.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

22

C O V E R

Even though T-Mobile and Orange were in the same industry, they had completely different test operation set-ups. Team structures, delivery models, test frameworks and test environment set-ups were all very different.

T

his past July EE celebrated a 4G landmark, switching on its superfast mobile network in Exmouth in south Devon, the 600th large town or city to receive this service. EE’s 4G is also available in over 6000 villages and small towns across the UK. In three short years, EE has achieved what it set out to do: be at the forefront of the 4G revolution.

Cecilia Rehn, Editor of TEST Magazine talked to John Charlton, Senior Test Service Manager at EE, about testing trials and triumphs at the UK’s largest digital communications company.

Tell us a bit about yourself. How did you end up working for EE in testing?

I’ve been lucky enough to be involved in the company for 15 years. I started my career at T-Mobile in their customer service department. The shift work suited my lifestyle at the time. Then came the opportunity to get involved in test, which was something I really enjoyed. The investigatory element and the chance to be a part of a large project, where I could see the end result of my efforts, really appealed.

JOHN CHARLTON SENIOR TEST SERVICE MANAGER EE

John Charlton has over 13 years of experience in testing, service management and programme delivery covering a multitude of domains and technologies related to the telco industry.

Based in the northeast of England with his wife and two children he is also co-founder of Uatopia, which offers free testing advice & guidance for new start-ups.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

I started in user acceptance and progressed to various test department roles, to test management, and into my current role within the service management area, focusing on both IT and network deliverables. I’ve been in this current role for around three years and it’s something I get massive enjoyment out of.

In 2012, EE was launched and two large firms, T-Mobile and Orange, were brought together. What was the merger like and how were different testing departments integrated?

It was a lot of work bringing the two companies together. Especially since we were launching the EE brand to

S T O R Y

the UK market shortly after the initial merger. Even though T-Mobile and Orange were in the same industry, they had completely different test operation set-ups. Team structures, delivery models, test frameworks and test environment set-ups were all very different. We had to integrate the departments and systems in order to meet EE’s goal of being first to market with the new 4G technology.

We started by identifying where we wanted to go as an organisation.

Testing then naturally followed suit, in order to support delivery.

One early example was discussing how integration environments would operate in the new company.

Orange’s legacy process allowed multiple integration environments, essentially letting projects work independently through this stage, without the need to merge code or build together until after this stage.

This gave flexibility but there was a significant cost attached, and some disadvantages, to the model.

We chose to follow a model similar to T-Mobile’s legacy framework, which utilised large-scale integration environments. This allowed us to run multiple projects in a single environment. There are a lot of dependencies there, but critically this model minimises costs from an environment perspective. We are also able to merge code early and test projects together, ensuring potential risks are highlighted earlier in the lifecycle, improving overall quality and time to market.

In the last year, EE has been merging its IT and Network test departments. How is this working out and what resources are at your disposal?

Once the two organisations were brought together, we had transitioned into a single IT test function and a single Network function. Then 12 months ago,

24

C O V E R

S T O R Y

we began working towards combining the test and network departments. Traditionally, these have been separate, with their own cultures and frameworks. But at EE we recognised the benefits of moving towards a single flexible framework, as these departments are more interlinked than ever in terms of deliveries. It makes sense that we should work together from the get go.

In terms of resources, we don’t have a set number of resources; instead we have a fantastic core internal team, coupled with a number of strong external partners. This enables us to operate using a very flexible operating model where we can pull in additional experienced resources as and when we need them.

EE was first to market with 4G technology. What were the challenges facing the teams behind the scenes?

It has been a challenge but a worthwhile one. We not only successfully tested and delivered this new technology but it also had to be done in conjunction to launching EE, a new brand. We had to deliver both elements at exactly the same time for this to be successful. In terms of 4G, as it was new technology it meant we had very little to go on, nor could we learn a huge amount from other experiences in other parts of the world, since very few organisations had done anything like it at the time and certainly not in the timescales we were aiming for. This resulted in a very fluid delivery model and a huge amount of effort and flexibility from the people involved.

customer journeys, which enabled a workable solution to be delivered in a very short space of time.

Since we wanted to be first to market with 4G and deliver in a condensed timeframe, it meant that testing had to be prioritised and we followed a risk based approach. Our customers expect high quality products and user experience is very important at EE so the approach was focused around key

What does the future hold for EE’s testing and network department?

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

It was different at T-Mobile, since the company didn’t aim to be first to market for everything. Testing was important of course, but there wasn’t the same push to market nor speed of delivery. But EE wants to be at the forefront and continue to push the boundaries with technology which we’ve seen recently with LTE Advanced, Wifi Calling and our Consumer Shared Plans. As a result there is a huge level on focus on test but also plenty of support, to make sure everything delivered is fit for purpose.

It is widely known that BT Group plc has agreed to acquire EE. Hopefully that’ll progress over the coming months, subject to regulatory approval. Pretty sure it’ll be exciting

times from the test point of view, looking at how we can move forward as a bigger organisation and create the UK’s leading communications provider.

Other than that, we have a number of new deliverables coming up which I’m really excited about. We’re hitting that test execution period now, after the planning and preparation stages over the summer months.

We’ve also been growing our EE apprenticeship programme to include and help develop the new generation of test staff. We’ve just gone through a recruitment cycle, with apprentices now working in both the test execution and environment and labs areas.

Supporting newcomers to the industry is a very important part of the job and hopefully we can help them develop. It also enables us the chance to get a fresh pair of eyes or perspective on something which can be priceless.

BUSINESS ORIENTED TESTING IN THE TELECOMMUNICATIONS INDUSTRY Dr Gilli Shama, Testing Scientific Director, and Ighal Szyk, Testing Product Line Director, Amdocs, describe how business flow testing helps CSPs to better meet business goals.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

27

T E L E C O M S

T

he communications industry is one of the most competitive and fastest evolving industries in the world. With over 5 billion active subscribers, and an exponential growth in multi‑device data usage, the need for quality deliverables is becoming more and more critical. A one day delay in a launch to market may cost a single tier‑1 telecom company as much as €9 million in lost revenues, underscoring the demand for shorter time to market. Service providers recognise the momentous impact of end‑to‑end quality on their business, and are therefore continually motivated to invest in innovative acceptance testing methodologies and tools that are aligned with their business goals.

Acceptance functional testing is typically done in applications, where the focus is primarily on expertise

Search Customer

Provide - Start Order Run Voice Event Provide - SOAP

Run SMS Event Provide - NCP

Provide - NCD Create Charge Level Credit Provide - Credit Vetting Screen

Provide - Order Summary

Figure 1a. Example of a telecom business flow.

and knowledge of the applications in scope. Offering acceptance testing that focuses on business flows rather than on applications can bring high quality results through lower testing efforts, and invariably leads to a better understanding of the end user experience.

This article offers a comprehensive approach to testing by business flows.

USING BUSINESS FLOWS A business flow is a description of a given business ‘story’. The business flow is composed of general activities connected together from start to end, and includes all realistic options that may arise through its use.

TEST CASES VERSUS ACTIVITIES Testing is typically carried out using test cases (TCs). For example: ‘search for offers that include both wireline and TV’, ‘scroll down the search results and select the 10th result’, or ‘select wireline offer from the search results list.’

Activities are the building blocks of a business flow. Unlike test cases, activities are open and may have various parameters. For example, the activities ‘search for an offer’ and ‘select an offer from search results’ match the example test cases above, and many other possible test cases.

DR GILLI SHAMA TESTING SCIENTIFIC DIRECTOR AMDOCS

Dr. Gilli Shama has been working in Amdocs for last 15 years in various managerial roles, including performance management and quality management.

In January 2015, Dr. Shama won the Quality Innovation of the Year Award for Amdocs BEAT. Prior to her work at Amdocs, she was a researcher at Maryland University, and she received her Ph.D. from Technion, Israel institute of technology, in technology and mathematics education.

BUSINESS FLOWS VERSUS SCENARIOS A business flow is a graph of activities that are likely to appear in real life – describing as closely as possible the actual customer experience – with a variety of parameters and flow splits.

The business flow graph starts with an activity (e.g. ‘search for a customer’) and includes several paths to end state.

An example of a telecom business process, based on eTOM – the telecom business process standard framework.

This can be seen in Figure 1a.

A scenario is a list of matching test cases to be performed in sequence using a realistic detailed example (see Figure 1b).

IGHAL SZYK TESTING PRODUCT LINE DIRECTOR AMDOCS

Ighal Szyk currently leads the research and development of the award winning Amdocs BEAT testing platform. He has 16 years’ experience in telco software development, managed services, project and program management, as well as testing tools and methodology R&D. Mr. Szyk holds a B.A. double-major in History and Western Civilisation from Concordia University, in Montreal, Canada.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

28

T E L E C O M S

TESTING BY BUSINESS FLOWS Traditional testing is typically carried out using test cases performed on applications, with scenarios integrating between the different test cases. Testing by business flows entails a different testing structure, tester role, and test methodology, as well as a unique set of testing activities, starting from high level test design all the way to the test status reports.

Customer Type

Business

Product Type

Digital Voice...

Provide - Start Order

Customer Type

Business

Provide - SOAP

Product Type

Digital Voice...

Account Type

Personal

Resource Type

SIM

Service Type

APN Service

Service Type

Allovance Service

Bill Method

CSV

Payment Method

Cash

Credit Evaluation

Accept

Yes / No

Yes

Customer Type

Business

Product Type

Digital Voice...

Tax Code

0

Search Customer

HIGH LEVEL TESTING STRUCTURE Testing by applications requires testers’ in‑depth knowledge of the application in scope. Such testing is done mainly in silo teams per application and/or product. The role of a Testing Center of Excellence (TCoE) is such testing. is to define the test scenarios and co‑ordinate the integration points and test environments between the product testing teams.

Testing by business flows requires testers’ in‑depth knowledge in the tested telecom’s business. Such testing is performed in teams per business flows. For example, one team for a flow from ordering to activation, and another team for collection flow, including payment. The testing teams are part of the TCoE, as their expertise is with entire business flows and not in unique applications. For example, the ordering testing team knows the ordering business and how to test it end‑to‑end, regardless of the exact ordering systems and other integrated systems.

Another implication of the business flow approach is insofar as the testing methodology used. Testing on a waterfall project starts with high level planning, and splitting of work to test calendars. The calendars may match business flows, such as eTOM business processes, and would enable providing recommendations on testing effort based on customer priorities, and risk history (Figure 2). Testing by business flows is an engine for agile methodology of acceptance testing driven development (ATDD), so as to match development that is carried out in sprints of business flows.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Provide - NCP

Provide - NCD

Provide - Credit Vetting Screen

Provide - Order Summary

Figure 1b. Example of a telecom business scenario.

29

T E L E C O M S

Figure 2. High level planning system by telecom business processes.

TESTING TASKS BY BUSINESS FLOWS Testing activities are performed through a business flow approach, starting from a detailed test design of a business flow, through the test execution, and test automation by business flows.

Work in business flows enables greater transparency of testing status to the business, by seeing those business areas which are stable, and those which need further testing, as demonstrated in Figure 3. By providing the customer’s business units visibility into testing progress from a business flow perspective, testing teams can better communicate with the business in a language they can understand much more easily and effectively.

Testing by business flows entails a different testing structure, tester role, and test methodology, as well as a unique set of testing activities, starting from high level test design all the way to the test status reports.

high efficiency of optimised effort and automation utilisation.

A testing centre of excellence that focuses on the telecom market, and employs business flows as its primary method for testing, will

SUMMARY The use of a business flows as part of a comprehensive testing lifecycle consistently leads to better test detailed design, better coverage of the customers’ end‑to‑end business, and optimal test automation enabling a maximisation of testing work efficiency.

Testing performed using business flows is proven to be a methodology that provides a better overview on the complete path of customer experience.

This ensures improved quality, especially in high business focus areas, and in

Figure 3. Reporting testing status by business flows.

likely have a wealth of corresponding business knowledge, gathered from its experience across providers. Such insight would undoubtedly serve those many customers that are concerned with getting the highest quality deliverables, enabling them to better operate in their competitive business environment.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

ARE STANDARDS STIFLING INNOVATION? Taken from his keynote presentation at the National Software Testing Conference in May, Kieran Cornwall, Head of Testing, ITV, questions the merits of standardisation in software testing.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

31

S T A N D A R D I S A T I O N

S

tandards are ever‑present in our lives – from the screws and bolts in airplane wings to education curricula. However, when comparing the physical to the intangible we have a problem. Yes, I want the half‑inch screws in my house or car to be standardised (and be exactly half an inch!) and I want to trust that the car, which has been built from parts manufactured all over the world, will work exactly as specified.

But when it comes to education standards, it’s trickier. Are we all happy with the education we’ve received?

The UK alone is seriously behind in terms of teaching IT in schools. This is because the curricula set by school boards are planned years in advance and are slow to change; meaning learners suffer are never be up to date.

This is more evident in some subjects than others.

What is a standard?

ISO defines a standard as: “A standard is a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose.”

The wording in ISO’s definition poses a major problem, as the definition can be interpreted to mean essentially whatever you want it to. A standard is either a document that provides requirements and specifications that must be used consistently. Or alternatively, a standard outlines guidelines and characteristics. We’re now faced with a dilemma – is the standard meant to be

a specification that must be followed or is it a guideline? Can you have flexibility and conformance within a standard?

There is a danger that the wording/definition either becomes so generic that it bears no reflection on the standards that it is meant to be, or it becomes so overbearing that it is unusable.

ABOUT THE ISO29119 STANDARD The ISO29119 standard, designed to cover software testing, has been a work in progress for many years. There are five parts to it: • Concept and vocab.

• Processes.

• Documentation.

• Testing techniques.

• Keyword driven testing.

There is a danger that the wording/definition either becomes so generic that it bears no reflection on the standards that it is meant to be, or it becomes so overbearing that it is unusable.

ISO29119 exist to fill a gap that seemed to be missing from the more industrial focused standards applied to date.

It is important to note here that the ISO29119 working group have to gain consensus through ISO/IEEE processes, something I will go into in more detail later.

OPEN VERSUS CLOSED STANDARDS ISO29119 is by definition of a closed standard. Closed standards are managed by a governing body, so the ISO and IEEE are closed standards making organisations. The working group opens itself up for industry participations and commentary, but this is only during designated ‘open sessions’ and in a specified format.

You need to be part of the group or at least be involved in the sessions to have an impact. To have a direct impact you need to be part of the central working group. This is in direct contrast to initiatives such as HTML, which is an open standard and welcomes development and participation.

KIERAN CORNWALL HEAD OF TESTING ITV

Kieran’s test management experience spans eight years, primarily in the finance industry, and including central banks, asset management and private wealth. Now working in an agile continuous integration delivery environment, Kieran has increased the rollout of pragmatic test functions within ITV.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

32

Concepts & Vocab

S T A N D A R D I S A T I O N

BS 7935-1 Constant amplitude dynamic force calibration.

Calibration and verification of non-resonant uniaxial dynamic testing systems

Processes

ISO/IEC 829

Documentation

IEEE 829

Testing Techniques

BS 7925-2

Process assessment model for software testing

Standard for software test documentation

Software component testing standard

Keyword Driven Testing

Figure 1. If you break ISO29119 down, you can see how it has been built on previous standards.

POSITIVES AND NEGATIVES It’s worth outlining the positives and negatives of open and closed standards for comparison.

OPEN STANDARDS Positives include: • Free: Only your time needs be applied. Although open standards still have their procedures and governance to an extent they are not bound to one delivery method.

• Open to all: There are numerous ones out there, only one Google search away. W3C (the world wide web consortium) are an example and provide open public forums as well as regular online forums and discussions. ££ Promotes interoperability, communication, etc. and does not tie you to a product, area or industry.

• Quick: They are often quick to change and can be more iterative due to their open nature.

Negatives include: • Chaos: If you’re used to the closed standard community,

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

then open standards can appear chaotic. However, I think some of the similar rules from closed standards are followed within reason in open standards management – otherwise nothing would ever happen.

• Complex: Open standards are often not prescribed; instead they may need a layer of technical understanding in the software world to implement. Bear in mind that using the command line for many is still an unknown.

CLOSED STANDARDS Positives include: • Control: The standard making organisations have control over what gets in and what does not.

The working group will decide on all feedback and agree as a group what is in and what is out.

• Mapped out: Standards prescribe the process for you and provide you with clear steps to (at least in this instance) infer quality.

• Easy replication: By providing clear‑cut steps, a closed standard means you can replicate it easily in different areas. This undoubtedly increases trade essentially in a similar way to

manufacturing standards.

• Comfort: Standards and aligning with them gives comfort and security that you’re doing the right thing – or at least using the distilled wisdom of those who have trodden the same path.

Negatives include: • Cost: These standards are not free.

They are not necessarily overly expensive but not free like with open standards. There are also the additional financial and time costs to consider if you wish to have a significant impact, since then you will need to be part of the working group, which entails: ££ You need to become a member of your national standards body. ££ There is a required attendance six days every six months at various locations worldwide. ££ They do not provide funding to members. ££ You will need to invest your time and your money, as it may involve regular membership and travel.

• Elitism: Considering that not all companies, startups or individuals can afford the time and cost investment in ISO and IEEE, there is a danger that the standards body only reflects the interest and opinions of wealthier stakeholders.

• Slow: A final decision for ISO29119 will be reached once all points have been taken into consideration, vetted and reviewed by the core body. Then they have to go through the standards bodies they align to such as the IEEE and ISO. This process takes years!

• Comfort: Getting too comfortable could have disastrous consequences. It should not be enough just to say ‘this is how we’ve done it before’ or ‘this is how the standard told me to do it’. Test managers need to own the responsibility for their work and think outside of the box.

So in summary, closed standards are more grounded and guided, whereas with open standards you will need

SMART PERFORMANCE TESTING STARTS WITH THE RIGHT TOOLS

High-volume load testing services built for the modern enterprise Test like a pro with the most powerful SaaS platform, tools and expertise available anywhere. Easily schedule load tests over time, perform highvolume tests from a mix of many international locations, and compare your results in the powerful reporting dashboard.

Start your Free Account www.apicasystem.com Apica LoadTest - Professional Load Testing platform from Apica

34

S T A N D A R D I S A T I O N

ISO began work on parts 1-3 of ISO29119 Netflix switched from providing hard copy DVDs in the post to online streaming.

Apple launched the first iPhone.

2007

Parts 4-5 were due for publication in 2014/15, but there is no clear evidence of their publication as of yet.

Parts 1-3 became recognised standards.

2008

The iPhone App Store opened

to be a bit more involved and learn from the beginning. However, there is so much information out there that this shouldn’t take too long. In terms of speed, closed standards are bound to strict hierarchies that promote drawn out discussion, which might not be perfectly suited to the fast‑paced world we’re now living in.

ISO29119 TIMELINE Before answering the question if ISO29119 is a good thing, it is prudent to consider the market it is serving.

The above timeline is quite eye opening.

If we consider these two technology companies, Netflix and Apple, they’ve managed to achieve extraordinary growth and innovation – all whilst only parts 1‑3 of ISO29119 were in development. They’ve clearly managed. So considering the market, one should ask ‘who would use this standard?’ Startups are unlikely to benefit, as they deliver in a continuous integration software engineering fashion, utilising test driven development, behaviour driven development, API targeted, microserviced, and cloud delivered systems. The tests are in the code from day one, there’s no shifting left as they started right.

Older legacy systems may benefit if they are not covered by rafts of automated tests and are extremely manual.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

iPhone 5s launched.

2015

2013

SUMMARY In my opinion the ISO29119, or any other closed standard, is not the right fit for the software testing industry.

We work in continuous integrated development/deployment. We automate absolutely everything.

Software moves too fast, nothing stays the same long enough to get a grasp in regards to a standard due to its slow agreement process.

By all means, standards can be used a as a guideline for your own (iteratively generated) processes.

Since standards have to be generic, which is their inherent weakness, it is not enough to follow them blindly, check the boxes and retire at the end of the day comforted by the fact that you followed the procedure. You’re responsible for quality, so take responsibility.

So, do standards stifle innovation? Of course they don’t.

You do!

Standards try iPhone 1 to offer a golden ticket – but there’s no one person out there with all the answers. Instead recognise that you’re

Netflix is today one of the core streaming multi-media services in the world, capable of streaming full HD films to your phone.

The iPhone 6 is now widely available, and in June, Apple announced there have been over 100 billion apps downloaded from the App Store.

in the driving seat and take charge.

Communication in the software testing industry and sharing good practices is the way forward. Trust in your experience, and if you don’t have it, then there will always be someone else out there who has it.

iPhone 6

READY, SET AND BE PREPARED Barry Weston, Delivery Director, Sogeti UK, highlights how structured process improvement models can help testing programmes.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

T E S T

T

P R O C E S S

37

I M P R O V E M E N T

est process improvement is a term that can be used to describe any activity that takes an existing process related to testing and makes it either better, faster or cheaper. These sort of improvements, of course have their place. Anything that improves the efficiency or effectiveness of a process in a controlled way is to be encouraged.

However, if these tactical changes are to have sustainable and maximised impact they are best undertaken as part of a wider improvement programme that considers supporting processes and the willingness and ability of the organisation to support the change.

This is where structured process improvement models can help. They are based on research into what leads to effective, consistent test delivery and recommend an order or roadmap to improvement activity that considers the three aspects of people, process and technology – not only within the core testing activity but also the supporting processes of project management, stakeholder involvement and so on. This helps to ensure the organisation, as well as the test team, are ready and prepared for the improvement that is to be made.

Although many improvement models exist for testing specifically, only two have gained wide acceptance, both in the UK and worldwide. These are TMMi owned by the TMMi Institute and TPI NEXT® owned by Sogeti. Both deliver improved processes and consistency of delivery but achieve it in different ways. As an employee of Sogeti I’ll describe and discuss the TPI NEXT model and how to get the most out of implementing changes based on it.

The first thing to say is that TPI NEXT is a flexible model that not only allows you to implement staged improvements – being able to measure your processes as managed, efficient or optimising – it also allows improvements to be made to resolve focused issues such as poor or ineffective automation in

a continuous model, where named maturity levels are less important.

The improvement model ensures you have the right foundations in place whichever approach is right for you.

What this means is that the model ensures that foundation processes are always in place to support your improvement activity, maximising the chance of success.

TEST PROCESS ASSESSMENT Organisations interested in process maturity often require a formal assessment of current test practice across the whole test discipline to be performed. This provides a strong indication of strengths – to be protected and built on – as well as weak or absent processes that are candidates for change. The diagram in Figure 1 gives an overview of the model and shows its 16 key process areas.

TPI NEXT groups these key process areas into Test Management, Stakeholder Relationship and Test Profession. This grouping allows the correct focus to be applied when gathering objective maturity evidence. Guidance is given in the model as to which areas apply most to the different disciplines involved in test, which ensures that only relevant questions are asked of those involved in any assessment activity. The matrix shown in Figure 1 identifies the individual attributes that apply to the key process areas for each maturity level (staged model); each element represents a specific question that indicates whether the element is satisfied or not. These questions are detailed in the TPI NEXT book that describes the model.

Organisations interested in process maturity often require a formal assessment of current test practice across the whole test discipline to be performed.

BARRY WESTON DELIVERY DIRECTOR SOGETI UK

Barry Weston is a Delivery Director for Sogeti UK. He has a background of more than 25 years in the testing industry and has significant experience of test process improvement and

A ROADMAP FOR IMPROVEMENT In its simplest form, the process improvement activity is then a matter of fixing the individual areas of weakness as realised in

managing testing programs. Seen by peers as a thought leader and evangelist, Barry was recognised for his work in Test Transformation having won The European Software Testing Award for ‘Testing Innovator of the Year’ in 2013.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

38

T E S T

#

Area

P R O C E S S

Managed

I M P R O V E M E N T

Efficient

Optimising

1

Stakeholder Commitment

1

2

3

4

1

2

3

2

Degree of Involvement

1

2

3

4

1

2

3

1

2

3

Test Strategy

1

2

3

4

1

2

3

1

2

4

Test Organiz.

1

2

3

4

5

Communication

1

2

3

4

6

Reporting

7

Test Process Management

1

2

3

4

8

Estimating and Planning

1

2

3

4

9

Metrics

10

Defect Management

1

2

3

4

11

Testware Management

1

2

3

4

12

Methodology

13

Tester Prof.

14

Test Design

1

2

15

Test Tools

1

2

16

Test Environment

1

2

1

2

1 1

1

3

3

2 2

2

3 3

3

1

2

3

4

1

2

1

3

2

3

1

2

3

1

2

1

2

3

1

2

1

2

3

1

2

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

1

2 1

3 2

1

2

3

1

2

3

1

2

3

4

1

2

3

4

1

2

3

3

1

2

3

4

1

2

3

3

1

2

3

4

1

2

3

1

2

3

4

1

2

3

4

4

1

2

Figure 1. Test maturity matrix.

the assessment. TPI NEXT takes a slightly different view and allows areas to be grouped into clusters of priority to ensure the fundamentals of any testing process are implemented first.

The TPI NEXT model is considered to be ‘business driven’ as it includes the flexibility to prioritise certain key processes above others to satisfy business requirements.

For example, you may not be interested in test automation and therefore rank it of low importance; whereas you consider reporting to be fundamental and rank this as high importance. This ranking then reorders the default clusters

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

to reflect the relative importance of specific process areas, without breaking the fundamental essentials of the maturity model.

Using the maturity model in isolation is unlikely to create a sustainable improvement to test maturity as the improvements are best implemented as part of a structured change programme.

Deming’s Plan-Do-Check-Action model is one of the simplest change models to implement and includes the essential actions of checking on the improvements that have been implemented and making further changes if the anticipated benefits have not been realised. Sogeti prefers to work with customers to build robust change programmes to ensure that it’s not just the process

T E S T

P R O C E S S

I M P R O V E M E N T

that is prepared for launch, but also the organisation is ready to receive it and that appropriate stakeholders and evangelists have been identified to ensure improvements quickly become the new normal. Managers and consultants are often heard to talk about needing quick wins – something tangible to happen in a short time. This is of course important as it means focus is less likely to be lost on the process improvement activity. As a rule of thumb it is also important that improvement programmes are compartmentalised into chunks of activity of no longer than three months – this allows successes to be identified, benefits realised and both celebrated before starting on subsequent changes. Kotter’s 8-stage change model1 provides a good framework within which to implement such changes.

WHEN AND WHERE TO START Although the focus of TPI NEXT is often to determine how to fix actual, or perceived, quality problems (typically too many significant incidents in live system operation) it is a valuable tool for comparative benchmarking of new suppliers to determine those that are best able to deliver consistent quality of solution to you. This can be particularly useful for organisations delivering systems of systems using multiple suppliers to minimise the risk of poor quality being uncovered late in the test delivery lifecycle.

We also recommend periodic assessments of stable ‘business as usual’ test functions to confirm that those processes that have previously been identified as contributing to the delivery process continue to do so and are still relevant in what is likely to be a changing environment of technologies, development methods and of course tools available to support the work of the tester.

Whether you decide to perform an internal informal process assessment using the TPI NEXT model or employ an external assessor to provide an independent view of test process maturity the following checklist will ensure you are ready to start: • What is the scope of the assessment? Do you wish to consider your entire testing function, or is there a particular project, programme or group of projects that you wish to assess?

• Do you have senior sponsorship for your assessment? There is an investment of time and therefore money in any assessment, but if areas of improvement are identified there will be the cost of the improvement itself, but also the need for the change to be seen important enough to implement and become part of your business as usual activity.

This is often difficult to achieve if the change is seen as a test function only change.

• Have you communicated to all areas considered in scope for the assessment its purpose and their role in helping to make it successful? Without the buy-in and commitment of all involved it is unlikely that the assessment will deliver all the benefits that are possible.

• Have you identified a large enough sample size for your assessment to minimise the influence of a single response on the assessment outcome and also to minimise assessor bias if it is an internal assessor being conducted by the testing function?

39

As a rule of thumb it is also important that improvement programmes are compartmentalised into chunks of activity of no longer than three months – this allows successes to be identified, benefits realised and both celebrated before starting on subsequent changes.

And remember – it’s not just a case of changing test processes in isolation – supporting processes must also change and, as always with any type of change, buy-in from the wider stakeholder group is fundamental.

Reference 1.

Kotter J., ‘Our Iceberg is Melting: Changing and Succeeding Under Any Conditions’.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

BECOMING MORE REACTIVE TO CHANGE Huw Price VP Application Delivery, CA Technologies, presents the case for model-based ETL testing.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

E T L

41

T E S T I N G

M

ost testing is, frankly, too random, unstructured, and repetitive. It lacks measurability, is not thorough enough, and cannot keep up with the rate of change. A significant 70% of testing remains manual1, and even so‑called ‘automated’ testing requires the effort of writing scripts, maintaining them, and investigating test failures.

Such testing simply cannot deliver the speed and quality required of modern organisations, especially within continuous delivery frameworks or ‘agile’ environments.

This is especially problematic for ETL testing, as ETL routines evolve as fast as the data‑driven organisations which rely on them. That’s fast: the amount of data collected by the average organisation is doubling each year2, while 90% of data in existence was collected in the last two years alone.3 We are in the so‑called ‘information age’, where every organisation is a software organisation, which relies on the analysis of customer data to drive research and development, and whose business success is bound directly to technological success.

In order to handle the increased variety and volume of data collected by a business, ETL routines grow rapidly. As they evolve, they quickly become highly complex, where even a relatively simple system with 32 nodes and 62 edges could have 1,073,741,824 possible paths through its logic, if arranged in a linear manner. This grows exponentially with every added decision, and there are numerous factors which can increase the complexity of an ETL routine, from the addition of a data source or target, to a new look up or active union. So, how does this impact ETL testing?

TYPICAL CHALLENGES FACED IN ETL TESTING THE REQUIREMENTS DON’T HELP The impact of complexity on testability is made worse as ETL routines are

usually poorly documented in written documents or spreadsheets, or, worse, are confined to the developers’ minds.

This makes abstraction for the purpose of increasing testability practically impossible, while ghost ETL scripts or SQL then have to be derived haphazardly from such documentation in order to test the routines. In fact, as many as 56% of defects can be traced back to ambiguity in requirements4, as testers and developers are forced to fill in the blanks.

AD HOC, MANUAL DERIVATION

Probably the greatest threat to a continuous delivery framework is an inability to respond to change. ETL routines evolve faster than the ETL scripts can be maintained.

This time‑consuming manual derivation of ghost code is usually performed in an unsystematic, ad hoc manner, where more and more scripts pile up.

As a consequence, 10 ‑ 20% functional coverage is the norm, leaving the majority of an ETL routine exposed to defects, and risking system collapse at the hands of ‘bad data’. Testing is also typically ‘happy path’ focused, repetitively testing business‑as‑usual scenarios (four times over‑testing is the norm), when it is in fact critical that ETL routines are tested to make sure that they reject outliers, unexpected results and bad data.

THE DATA PROBLEM Even if ghost code is written for negative testing, it is unlikely that there will be available data to execute it. This is because most organisations rely on production data, collected during past operations. Such data is sanitised by its very nature to exclude ‘bad data’, and does not include future scenarios. It is unlikely, for example, to contain data which might threaten the integrity of a data warehouse – such as missing inter‑dependent or mandatory data – and will not contain the data needed for negative testing or for the testing of new business requirements.

MANUAL COMPARISONS Whatever the coverage provided of the tests and test data, running the ghost code will produce a huge amount

HUW PRICE, VP APPLICATION DELIVERY, CA TECHNOLOGIES

Huw joined CA Technologies in 2015 as Vice President of Test Data Management, when specialist testing vendor Grid-Tools was acquired into the CA DevOps portfolio. During his 30 year career, Huw has gained a deep understanding of the challenges faced by modern organisations, and, with an understanding of the science of testing, how to solve them.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

42

E T L

T E S T I N G

of highly complex data, which then has to be validated. This often involves a manual comparison of the data to the source data and expected results, which is further time‑consuming and error‑prone. The vast data must be validated on numerous levels (for completeness, consistency, integrity, etc.), while the source data will typically come from numerous different file formats.

TESTING CANNOT KEEP UP WITH CHANGE Probably the greatest threat to a continuous delivery framework is an inability to respond to change.

ETL routines evolve faster than the ETL scripts can be maintained.

When a change is made to the requirements, testers have no way to automatically identify its impact on the scripts, which usually lack traceability back to the updated documentation. Scripts are often checked and updated by hand, before the data required to execute them is found or created. We once worked with a financial organisation where it took two testers two days to manually check the existing test cases when a change was made to a single requirement. Given that some organisations maintain tens of thousands of ETL scripts, this is clearly not a viable method.

THE SOLUTION: FULLY AUTOMATED ETL TESTING In order to deliver valuable applications which can keep up the continuous delivery of valuable software applications, ETL testing needs to become less manual, more rigorous, and more reactive to change. Below is an example, demonstrating how model‑based testing can be readily implemented within ETL testing, and how it can resolve the bugbears set out above.

Broadly, the approach consists of five (overlapping) steps: 1. Formally model the ETL rules,

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Figure 1. Flow chart 1.

overlaying the model with logic and data.

2. Automatically derive optimised test cases from the model, such that they provide 100% functional coverage.

3. Automatically generate the data needed to execute the tests, based on the model itself.

4. Execute the tests, and automatically compare the results to the expected results defined in the model.

5. Update the model, and automatically reflect changes in the data and test cases, by virtue of their traceability back to the model.

Put simply, the example is a transform involved in a payment plan system. It establishes, based on a customer’s current balance and risk code, whether they should receive a payment plan – or, whether they should be marked for inspection, or whether their risk code should be adjusted.

1. START WITH A FORMAL MODEL In the below example, the ETL transform has been mapped to

a flowchart model, which has been overlaid with functional logic and data. This offers the fundamental advantage that it ‘shifts left’ the effort of testing, as all subsequent test assets can be derived from the initial effort of designing the routine.

It also offers the more specific advantage that it helps combat ambiguity and incompleteness, reducing the transform into its core cause and effect logic. The steps of the flowchart can then be reliably converted into test steps. Abstraction for the purpose of managing ever‑growing complexity is also made possible, especially when using subflows, which embed lower‑level components within master‑flows. The master‑flow provides a single point of reference, while the tester can drill down into the subflows if more granularity is needed.

2. AUTO‑GENERATE THE TEST CASES Because the flowchart is a mathematically precise model of the ETL transform, test cases can be automatically and systematically derived. Automated mathematical algorithms can be applied to derive

E T L

43

T E S T I N G

5. AUTOMATICALLY IMPLEMENT CHANGE

Figure 2. Flow chart 2

As organisations collect and analyse more and more data, ETL routines are going to grow ever‑more important, and ever‑more complex. the possible paths through the flowchart model, which cover every possible combination of inputs and outputs – this can be carried out using cause and effect or homotopic analysis. The above image shows one possible path through the ETL transformation, with the logical steps set out on the right hand side.

These paths (which are equivalent to test cases) can then be optimised, so that 100% functional coverage is retained in the smallest possible number of tests. As a single piece of logic might feature in multiple paths, optimisation techniques such as All Edges, All Nodes, All In/Out Edges, and All Pairs can be applied, to eliminate over‑testing and shorten test cycles.

3. AUTOMATICALLY GENERATE THE DATA The data needed to run the ghost ETL transform can also be automatically derived from the model. As the flowchart is created,

output names, variables and default values can be specified for each node. As the test cases are created, data can then be created automatically from default values, or using a synthetic generation engine, or can be found from back‑end systems using automated data discovery.

4. EXECUTE THE TESTS In stages two and three, you have in effect created data packs, with expected results linked to them.

In the current example, these expected results include marking a customer for inspection, giving them a payment plan, and putting an account in arrears. The expected results can be seen specified in the example path given above. Once the data packs have been run through the test ETL environment, a data comparison tool can be used to validate the actual data against the expected results.

Changes made to the ETL transform can be implemented automatically, in a fraction of time required when testing manually. Again, this is because the flowchart is a mathematically precise model.

Automated path impact analysis can therefore be applied, to identify the exact impact of a change made to the model on the existing paths. Any affected test cases can be automatically removed or repaired, with any new tests needed to retain 100% coverage created. At the financial services company mentioned, this reduced the time to update test cases after a change was made from two days to five minutes.

SUMMARY As organisations collect and analyse more and more data, ETL routines are going to grow ever‑more important, and ever‑more complex.

To handle this, testing needs to be made more efficient, more effective, and more reactive to change. Model‑based testing offers one way to achieve this, tackling the complexity of ETL routines while introducing a far greater degree of automation into the testing process.

Reference 1.

2.

3.

4.

Howard, P., ‘Automated Test Case Generation’, p. 5. Retrieved 02/09/2015 from” http://www. agile-designer.com/resources/automatedtest-case-generation-report-bloor/ Becla, J. and Wang, D. L., ‘Lessons Learned from managing a Petabyte’, p. 4. Retrieved 19/02/2015 from: http://www.slac.stanford. edu/BFROOT/www/Public/Computing/ Databases/proceedings/ ‘What is big data?’, IBM e-book, http://www01.ibm.com/software/data/bigdata/what-isbig-data.html Bender RBT Inc., ‘Requirements Based Testing Process Overview’, Retrieved 21/08/2015 from: http://benderrbt.com/BenderRequirements%20Based%20Testing%20 Process%20Overview.pdf

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Welcome to the 2015 edition of our 20 Leading Testing Providers We hope that you’ll find this guide outlining different, selected software testing and quality assurance products and services useful. The software testing landscape changes rapidly and we find that an annual update on the marketplace is a good starting place as you consider purchase decisions going forward.

Sponsored by

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

45

Tata Consultancy Services (TCS) Since 1975, Tata Consultancy Services (TCS) has been helping UK and European businesses become more effective. TCS UK and Europe service over 350 customers across UK and the continent, including 44 of the FT Europe Top 100 companies. TCS is currently working with: • 4 of the top 6 Tier 1 global investment banks.

• 4 of the top 5 UK retail banks.

• 2 of the top 2 global life sciences companies.

• 7 of the top 10 life and pensions providers.

• 5 of the major telecom and media companies.

• 8 of the top 10 retailers in the UK and Europe.

• 4 of the big 6 electricity and gas companies.

• 7 of the major water utility companies.

• 3 of the largest energy and resources companies.

TCS is a major UK and Europe employer with over 50,000 people across 50 locations. Business heads from across Europe have appreciated TCS’ contribution to the continent: • 1,470 C‑level executives from Europe’s top companies across 13 countries gave TCS the highest general satisfaction rating (80%) in the industry, ahead of 22 other top IT services companies.

• TCS ranked number one in nine countries across Europe: UK, France, Germany, Switzerland and Austria and the Nordic region.

TCS’ services portfolio and delivery model TCS is an IT services, consulting and business solutions organisation that delivers real results to global businesses, ensuring a level of certainty no other firm can match. TCS offers a consulting-led, integrated portfolio of IT and IT-enabled infrastructure, engineering and assurance services. This is delivered through its unique Global Network Delivery Model™ (GNDM™), recognised as the benchmark of excellence in software development. GNDM™ allows clients to choose the sourcing strategy best suited to its business needs. TCS adopts a follow-the-sun approach, meaning that no matter where client’s business is located, TCS helps keep it running 24*7, while providing a seamless experience across all operations.

TCS Innovation Labs and Co-Innovation Network help clients achieve and maintain a competitive advantage, whilst TCS research-based solutions in advanced technologies support client’s business objectives.

With one of the most comprehensive portfolios of independent test capabilities on offer, TCS addresses both business and quality challenges for its global clients. It empowers organisations across domains to optimise overheads, realise first mover advantage and

improve customer satisfaction. TCS offers Assurance Services across the testing value cycle, including test consulting and advisory, test services implementation, and managed services for test environment and test data management.

TCS continually redefines testing and quality assurance paradigms to help its clients stay ahead of the curve. Its library of domain-based reusable business functions and proven engagement model founded on the twin pillars of product and process quality enable TCS to deliver certainty to its clients.

TCS’ Assurance Services Unit (ASU), accounts for ~8.5 percent of the company’s annual revenues, has 500-plus customers worldwide, over 65 mature Test Centres of Excellence (TCoEs) and 28,000-plus Assurance professionals (of whom over 24,000 are certified in assurance tools). The TCoEs have 23-plus cross-domain strategic and solution partners and, collectively, own over 100 accelerators. It has also filed over 28 patents in the software assurance space.

Analyst firms IDC, HfS Research, 451 Research and others commend TCS’ Assurance on achieving a strategic business position in the company and localising the global corporate brand effectively for country-based growth.

Awards and recognitions • TCS recognised as a Leader in Everest PEAK Matrix Leader for Independent Testing Services.

• Ovum recognised TCS as a leader in Outsourced Testing Services.

• Gartner recognised TCS as a Leader in the Magic Quadrant for Application Testing Services, Worldwide.

• NelsonHall recognised TCS as a Leader for Overall Software Testing Services.

• Ranked No.1 for customer satisfaction in UK IT Services – Whitelane research, 2015, 2014 and 2013.

• Ranked Britain’s Top Employer in 2015, 2014 and 2013 – Top Employers Institute.

• TCS adjudged as one of UK’s super brands for the year 2015.

• Awarded the Gold Award in ‘Innovation in Learning’ 2014 – Learning and Performance Institute.

• Achieved a score of 97% in ‘Business in the Community’s’ Corporate Responsibility Index in 2015.

• Largest software consultancy in the UK to qualify for Investors in People – Gold standard for the year 2014.

TCS’ social commitment TCS invests millions in corporate responsibility initiatives yearly and support initiatives of over

+44 (0)20 7245 1800 uk.info@tcs.com www.tcs.com 4th Floor, 33 Grosvenor Place, London SW1X 7HY UK

www.tcs.com 40 charities across Europe. TCS employees invest over 100,000 hours of their time each year, volunteering and supporting social causes and community projects.

TCS IT Futures – The community engagement work, conducted as part of global ‘Purpose4Life’ initiative, has won many awards. It aims to inspire a generation of young people to work at the forefront of technological change.

TCS Eco Futures – This initiative aims to reach and engage employees across TCS UK so that TCS and its customers can reduce adverse environmental impact. The focus is on energy efficiency, waste, recycling and business travel.

About TCS worldwide TCS is a part of the Tata Group, one of the largest and most innovative industrial conglomerates with 100 companies. Building on more than 40 years of experience, TCS adds real value to global organisations through domain expertise, proven solutions and world-class service.

TCS has 320,000 plus employees, representing 118 nationalities. TCS partners with clients across 44 countries. Repeat customers contribute to 99% of TCS’ revenue.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

46

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

ELEKS +44 (0)203 318 1274 eleksInfo@eleks.com UK Office

5 Harbour Exchange,

South Quay, London, E14 9GE UK

www.eleks.com/QA

ELEKS is a global organisation that provides end-to-end software product development.

Since 1991, the company’s innovative and award-winning solutions to include data science, mobility and wearables, as well as digital solutions have significantly contributed to the success of ELEKS’ customers, including Fortune 500® companies, and are recognised as a valuable part of international best practices.

Acknowledged among top 10 Quality Assurance Leaders according to Clutch’s research, ELEKS has been recognised for the company’s integrated approach to quality assurance, delivering secure, high quality, bug-free and cost-effective solutions to the customers worldwide.

ELEKS quality assurance advantages • Top 10 quality assurance leaders, according to Clutch.

• 20+ years excellence in QA. • 200+ qualified engineers.

• 40+ certified ISTQB specialists.

• Delivery for 100+ large and SMB clients.

• Quality assurance for the full-cycle product development.

• Manual, automated and innovative parallel

testing techniques.

• Comprehensive QA Audit.

• Standards applied: IEEE 829, ISO 9126, ISO 27001.

Similar to project management best practices, ELEKS applies quality management in order to facilitate the achievement of the quality objectives and to deliver the highly effective solutions. Quality management requires the involvement of the quality assurance engineers on all the stages of software development lifecycle.

“Software solutions today are expected to be perfect in their first release,” said Nazariy Popov, QA Program Manager at ELEKS. ”We are committed to delivering high quality solutions on accelerated schedules, building testing tools and platforms, improving profitability and increasing the end-user satisfaction. With 20+ years of experience employing quality management programs, we help our customers exceed expectations in their marketplace.”

Interested in test automation? Please visit: eleks.com/auto-test

Amdocs Amdocs Testing is the world’s leading testing service provider for the communications industry.

With over 30 years’ experience in the telecommunications industry, Amdocs knows how the industry works.

As experienced telco professionals, its professionalism is a product of a unique blend of knowledge, technology, and processes including expertise in multi‑vendor environments. Amdocs’ repository of best practices and know‑how are all wrapped‑up in the innovative Amdocs BEAT™ framework.

Amdocs’ holistic testing approach includes patented technology, skills and knowledge which ensure customers gain testing solutions at an optimised cost, superior speed, and top quality. It earned its position as the top testing vendor by taking its customers’ business applications to go‑live with industry‑low defect levels while reducing cost and minimising time to market.

Its innovative testing approach includes a patent‑approved framework,

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Amdocs BEAT™, which standardises and optimises testing processes. Amdocs uses a sophisticated analytical model to make recommendations so that every testing project is significantly more productive and cost effective. In this way, it helps customers achieve their business goals and deliver a superior customer experience.

Utilising unique proprietary tools, methodologies and best practices, Amdocs Testing delivers efficiencies and value throughout the testing lifecycle. Its market leadership is driven by professionalism, reliability and true partnerships with customers as well as a global presence of highly skilled resources to ensure seamless, ongoing support and “follow the sun” continuity.

Amdocs Testing is part of Amdocs, a global company (NASDAQ:DOX) with revenue of US$3.6 billion in fiscal year 2014.

Amdocs employs a workforce of more than 24,000 professionals serving customers in over 90 countries on six continents across the globe.

+1 314 212 7000 testing@amdocs.com Missouri 1390 Timberlake

Manor Parkway

Chesterfield, MO 63017 USA

www.amdocs.com

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

47

Apica +46 (8) 400 273 27 sales@apicasystem.com Apica AB, HQ

Garvargatan 9

112 21 Stockholm Sweden

www.apicasystem.com

Apica’s LoadTest Portal is an enterprise‑level load and stress testing SaaS and script tool trusted by many of the world’s largest organisations. Use Apica LoadTest Portal to accurately execute load tests of nearly unlimited scale and with a depth of result detail unparalleled in similar testing tools.

All of Apica’s testing services can be accessed from one convenient, affordable and easy‑to‑use portal. Whether you have a product launch on the horizon or are planning for seasonal traffic spikes, the SaaS‑based LoadTest Self‑Service Portal allows you to effortlessly perform any number of self‑diagnostic tests prior to anticipated peak load times.

Schedule regular website load tests over time, perform high‑volume tests from a mix of many international locations, and compare your results in the powerful reporting dashboard.

Easily create powerful custom scripts with no prior programming knowledge. And because Apica’s load test tool Zebratester is Java‑based, the tool is platform‑agnostic and can run on Windows, Mac OS, Solaris, and Linux systems.

Apica’s load test tool supports automated variable handling and offers Java Plugin support

for complex script functionality as well as script creation without programming. Supported Web Standards include HTTP and HTTPS, HTML forms, HTML hyperlinks, HTML redirects, CGI parameters, JSON data, XML and SOAP, and WebDAV for extracting and assigning dynamic session parameters at runtime and more.

Integrations: New Relic, AWS CodePipeline, AppDynamics, TeamCity, Jenkins, Windows Agent, Linux Agent.

Start your free trial www.apicasystem.com Apica – your partner in performance Apica provides the most effective technology for optimising the performance of cloud and mobile applications. Apica offers companies and developers alike easy‑to‑use, cloud‑based load testing and web performance monitoring tools to test applications for maximum capacity, daily performance, improved load times, and protection from peak loads.

Apica was founded in 2005 and is based in Stockholm, Sweden and Santa Monica, California with offices in London and New York.

Centre4 Testing Centre4 Testing is the largest, fastest growing privately‑held software testing pure‑play in the UK. Earlier this year, Livingbridge, the highly regarded private‑equity firm, invested £5.5 million after more than 100 clients and independent testing professionals rated Centre4 Testing more highly than its peers in every category they surveyed, from quality of people to pricing.

Centre4 Testing tailors services to meet the varied needs of clients, from full‑service consultancy (its OnTrack service line) through to fast and flexible resourcing solutions using a database of over 10,000 professional UK‑based testers. The British Computer Society awarded one of its client’s projects as a finalist in The European Software Testing Awards.

Clients benefit from managed services incorporating single or multiple service lines: • OnTrack – consultancy services providing clients with expert test consultancy for strategic work, test management, automation and performance testing. Clients enjoy competitive pricing and technical expertise as a result of Centre4 Testing’s close partnerships with HP, SOASTA, Smartbear and TestPlant.

The company is also an accredited Atlassian Partner for JIRA consultancy.

• OnSite – resourcing solutions with rapid access to the UK’s best professional contract testers.

• Cloud Testers – crowd‑sourced testing to some of the country’s biggest consumer brands for device, browser and operating system compatibility testing; plus accessibility and OnDemand functional test teams.

• Grad Testers – intelligent, personable and adaptable ISTQB‑qualified graduates who have passed rigorous selection processes and embarked on Centre4 Testing’s comprehensive and ongoing training programme including agile techniques, mobile testing & automation. A high calibre and compelling, low cost day‑rate alternative to offshore and nearshore resources.

Centre4 Testing invites you to join its popular client London City breakfast briefings and free‑of‑charge hands‑on automation ‘Test on Tuesday’ workshops (for clients and testers alike) at its London Bridge offices.

+44 (0)20 3697 1444 ryan.hannigan@ centre4testing.com The Hop Exchange,

Southwark Street

London SE1 1TY UK

www.centre4testing.com

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

48

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

HP

+44 (0)845 270 4567 angela.vorley@hp.com Amen Corner

Cain Road, Bracknell

Berkshire, RG12 1HN UK

www.hp.com/go/software

In the idea economy of today, business success is being more-and-more defined by the digital experience. Often a customer’s first interface to your brand and value will come through software. As a result, users across the globe are unrelenting in their demand for faster delivery of engaging applications and uncompromising in their expectation of quality and performance. As a result, our customers in IT and line-of-business application teams who deliver the software the powers business innovation and customer satisfaction demand proof of HP software’s investment in the Application Delivery capabilities they care about the most, coupled with investment for innovation and ease of use.

HP Software Application Delivery Management (or ADM) is a proven and integrated portfolio of software running over 8,000 customers’ application lifecycle and testing needs for over a decade. We pride ourselves on delivering the most innovative, open and proven software solutions for customers world-wide who build their business on delivering applications with

unprecedented velocity and uncompromised quality.

We come from a core competency of quality, beginning from an idea launched in a Palo Alto Garage with our founders focused on enabling the testing of the technologies that would be the underpinnings of a digital revolution.

HP software’s heritage is delivering integrated software solutions aimed at developing, testing and managing IT throughout the past several decades driving the innovations and capabilities vital as each new IT discontinuity has entered the landscape.

We are now on our 12th edition of this powerful set of solutions and HP ALM 12 represents the culmination of several years of R&D leadership and innovation to deliver the software that will enable our customers to deliver applications that embody the new style of IT, driving brand preference and delighting their internal employees and customers.

Maveric Systems Started in 2000, Maveric Systems is a leading provider of IT lifecycle assurance with services across the SDLC, from requirements to release.

With a strong focus on the BFSI and telecom sectors, Maveric’s services are led by deep domain expertise and platform-led solutions. Specifically in the UK, Maveric focusses on: 1. Providing platform led strategic offerings to Tier 1 banks & insurance corporations, and 2. Supporting the technological transformations of newly licensed banks, wealth management companies and operative unions.

In the UK and Europe, Maveric’s services have been enlisted by clients such as Citibank, Deutsche Bank, IFDS, RBS, Capita, Tesco, MCIS Zurich, Ericsson and Validis. Maveric has also worked with emerging leaders such as Metro, Kleinworth Benson , KBL and Rabo Bank.

In these markets, Maveric has twin focusses of: Maveric’s requirements assurance, application assurance and program assurance services are aimed at eliminating quality, cost and time-to-market risks associated with large IT transformation programs. Maveric continuously strives to up-hold its commitment to the principles of independence, vertical focus and innovation.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Maveric has been identified as a transformation specialist serving ‘Transformation Focused Clients’, by NelsonHall and was the recipient of Frost and Sullivan Product Innovation award, as well as the Banker Middle East Industry Awards for the ‘Best Banking Technology Partner’ in the years 2013 2014, and 2015. Further, Maveric has also been shortlisted as one of the finalists for the prestigious The European Software Testing Awards (TESTA), 2013 & 2014.

Maveric has to its credit multiple engagements around a large number of banking, insurance and telecom products, including Temenos T24, Flex Cube, Misys Equation, Finacle, Eximbills, Trade Innovation, BankTrade, Fidessa, Sungard, Murex, Ingenium, TIA, BSCS, Siebel, TIBCO, CS5, iMAL, Ethix and Pheonix.

Our expertise in products like FIS, Temenos T24, Oracle Flexcube, Finacle, TCS Bancs, and Misys has contributed to successful implementations for our clients.

With a workforce of 1,200 located in offices across London, Princeton, Dubai, Singapore and Chennai, Maveric delivers 100% successful IT implementation to clients.

+ 44 (0)20 7001 0555 info@maveric-systems.com 5 Laybourne House,

Admirals Way,

London, E14 9UH UK

www.maveric-systems.com

50

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

Odin Technology +44 (0)118 370 6759 info@odintech.com Atlantic House

Imperial Way

Reading RG2 0TD UK

• Increase automation across the enterprise up to 80%.

• Eliminate the overhead of script maintenance.

• Improve productivity rates in automation.

• Leverage existing tool investment or open source.

• Standardise automation practices across the enterprise.

The market leading enterprise automation platform Axe is trusted as a standard by over 100 organisations worldwide to unify their test automation approach and skills, reducing time to market without reducing quality. Axe provides a packaged automation solution, with proprietary execution capability, bundled open source tool integrations and support for the commercial market leading QA suites for testing ERP, mobile, web, desktop, SOA and legacy applications.

www.odintech.com

For traditional, agile, BDD, DevOps and bi‑modal IT By not relying on outdated capture‑replay methods, Axe uniquely supports multiple approaches to development across the modern

enterprise. Enabling true “in‑sprint” test automation in agile and continuous integration environments for any application type.

Focus on defining test scenarios not coding With a simple table based test definition interface and support for BDD and Gherkin, non‑technical participants can define test scenarios without requiring coding skills. Backed by an innovate, tool agnostic code generation technology, Axe generates fully functioning automated tests and documentation. Unlike other “scriptless” tools, this unique approach means Axe can be fully customised and extended by end users to support those well known automation hurdles and custom control issues.

Axe fully supports the following software quality tools and suites: • Selenium Webdriver (Web), Appium (iOS and Android), White Framework (Desktop), Axe SAP Certified SAP GUI Testing, SOA Service Testing.

• HP QTP/UFT, QC/ALM, Microsoft‑Coded UI, MTM/TFS, IBM‑RFT, RQM, Oracle OATS/OTM.

Performance Lab Performance Lab is a private company established in 2008 to address a growing void in the area of independent software testing and quality assurance for enterprise customers. Since 2014 the company has been the leading provider of these services within the Eastern European QA market. The company’s rapid growth is a result of proven success across multiple industries, such as finance, insurance, telecom, retail, e-commerce and software engineering, including companies listed on NYSE, LSE and MOEX. The company offers a full range of quality assurance services and has exceptional expertise in performance testing, test automation and mobile testing and automation on a real devices.

The company’s staff of over 500 employees is highly educated and motivated to fill our client’s needs for software quality with great technical background and experience. Performance Lab’s school of testing was founded in 2010, and has graduated 1000+ testing engineers that are currently employed by Fortune-500 companies and bringing value to the world QA practice.

Being the leader of Russian market, the company proudly produces the Russia Quality Report 2014.

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Why Performance Lab Performance Lab offers the following strengths and differentiators that extend our position as the industry leaders in software testing: • Proven experience of end-to-end testing services for various industries.

• Experience of successfully executing largescale testing engagements, some with sizes of 100+ test professionals.

• Availability of the right mix of local & offshore test professionals with worldwide presence.

• Proven framework to improve test maturity.

• Proven consulting capabilities, such as test consulting.

• Technology Centers of Excellence (CoEs) comprising of: automation CoE, performance CoE, security CoE, SAP testing CoE, and Siebel.

• Testing CoEs ensuring that the latest trends are being studied and industrialised within the Practice and with our customers.

Performance Lab’s specialties include software testing; quality assurance; test automation tools & services; performance testing tools & services; and mobile testing.

+1 844 446 7587 engage@performancelabus.com 4633 Old Ironsides Drive,

Suite 220, Santa Clara,

California, 95054 USA

www.performancelabus.com

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

51

PractiTest +972 8 637 6997 info@practitest.com 5 Oppenheimer St.

Rabin Science Park

Rehovot, Israel

www.practitest.com

Founded in 2008, PractiTest is an end-to-end QA and test management solution, designed to help you control your testing and development process, focusing on how to manage your project and its information, and how to communicate the outcomes of your testing to everyone in the organisation.

Imagine driving your car at night without turning on your headlights, guided only by your experience as a driver and by your knowledge of the road ahead. You would never think about doing this because it may lead to accidents. This is like managing a software development process without proper test management in place. The software industry demands regular testing to guarantee product’s quality, and to ensure the development process isn’t derailed by unforeseen factors. This, together with the trends of agile and DevOps, the growing number of platforms your applications need to support, and the increasing pace of market competition, make the need for testing more critical than ever. In the past, testers would take weeks to run tests and send reports to management. Today we need results almost

immediately, and so organisations rely on a combination of manual and automated tests, and live dashboards available to everyone in the organisation. With PractiTest, we aim to provide holistic platform that serves the demands for process and information management for the organisation.

The software allows you to organise your requirements, create and run tests, tracks bugs etc. Integrations are available with top bug management tools including: JIRA, Bugzilla, RedMine and Pivotal Tracker as well as automation tools such as Selenium, JUnit, SoapUI, QTP, Jenkins, TestComplete and many more.

PractiTest’s team of testing veterans maintains an open conversation channel with clients during the whole process of implementation, helping users with their questions and gathering feedback that is then integrated as features into the system. The methodological support from the experts at PractiTest adds to the technological capabilities leveraged by the client.

QASource QASource is the comprehensive leader in software QA, headquartered in the Silicon Valley and working with companies large and small, the world over. QASouce grows 20% ‑ 50% per year and has completed hundreds of successful engagements for clients such as IBM, Oracle, Cisco, Prudential, Mass Mutual, Luxotica, Ford, and a host of startup companies. As an independent quality assurance and testing company, QASource delivers effective, customised testing solutions at significant cost savings. QASource’s QA expertise includes designing test strategies, manual test execution, building maintainable automation, setting up test labs, documenting test processes, providing project management, security testing, mobile testing, and a host of other software testing services.

High‑quality talent QASource’s dynamic partnership of India’s technical talent and US management provides cost‑effective testing solutions while avoiding the major challenges with the effectiveness of testing as it is currently performed in most companies. QASource’s

staff of QA professionals can take your quality assurance from the initial design phase to completion, providing the necessary QA resources every step of the way toward success. QASource engineers know how to ‘go beyond the checklist’ to deploy high‑capability test practices resulting in high‑quality software releases, time after time. QASource engineers are formally and continually trained in the areas of new testing methodologies, effective and efficient communication and best‑practice processes. Turnover at QASource is an incredible 9%, due to hiring people who love working hard, appreciate the company’s ‘promote from within’ policy and enjoy constantly improving their domain knowledge.

Quality that creates value QASource delivers international‑standard quality, strict adherence to deadlines and total client satisfaction. QASource customers have had successful company exits exceeding US$15 billion in M&A transactions, leading to QASource’s slogan, ‘Quality That Creates Value.’

+1 (925) 271 5555 info@qasource.com 73 Ray St, Pleasanton

CA 94566 USA

www.qasource.com

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

52

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

QualiTest Group +44 (0)845 869 6070 qualitestuk@qualitest.co.uk European Headquarters

2440 The Quadrant

Aztec West Business Park

Bristol, BS32 4AQ UK

www.qualitestgroup.com

Recognised as a Visionary in Gartner’s Magic Quadrant for Application Testing Services, QualiTest Group is the world’s second largest pure play software testing and QA company with over 1500 software testing professionals located worldwide, offering a full spectrum of testing services.

We design and deliver contextualised solutions that leverage deep industry‑specific understanding with technology‑specific competencies and unique testing‑focused assets. QualiTest delivers results by combining customer‑centric business models and the ability to gain a profound comprehension of customers’ goals and challenges with critical thinking and independence.

Your partner in success We are always aiming to improve our clients’ businesses while providing more cost‑effective solutions. For example, our Results Based Testing engagement model bestows clients with contractual guarantees for quality improvement and cost reduction. QualiTest works closely with the client to ensure that the project is delivered on time and on

Ranorex Test automation for desktop, web and mobile applications What kind of application do you need to test? Is it installed on the Windows desktop? Does it run in a browser? Is it used on smart phones or tablets?

It does not matter which platform your software is developed for. Ranorex provides cost-effective and comprehensive test automation software to create reliable automated tests for any technology. Due to its ease-of-use, increased testing accuracy and low cost-per-seat, Ranorex is an excellent choice for software teams of virtually any size or level of sophistication, and is now used by 2,000+ software companies around the world.

Reduce test maintenance and become agile Not only software development teams move to agile methodologies. Also testing teams need to become agile too. Automated testing is one of the key factors to deliver in time and to ensure high software quality in today’s fast moving world.

In processes like this, robust and reliable tests scripts are absolutely essential. There is no time left to spend days for test script “reanimation”.

Ranorex tools assist you in being prepared. A modern, object based test automation approach

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

budget, providing advance test automation frameworks for rapid test automation implementations whenever possible.

Committed to quality The key to assuring high‑quality software is using an independant third party who can think outside of the box to locate defects that would be overlooked by in‑house IT teams. Through our All‑Shore services, delivering the best combination of onsite, onshore, offshore and crowdsourcing testing services, we provide a variety of services to Fortune 500 companies, as well as local small businesses. Providing QA solutions and outsourced testing services to a wide variety of industries, such as healthcare IT, aerospace, defense, finance, retail and telecommunications has enabled us to maintain a strong presence in the US, Europe, the Middle East, and Asia.

We are always looking to recruit strong candidates. If you think you would be a good fit for our organisation please contact the HR department in your locale.

For more information visit QualiTest Group at www.qualitestgroup.com

Ranorex

®

separates your logical test case structure from the technical identification layer. Moreover, addressing UI elements on your desktop, in the browser or on your mobile device is done by the powerful RanoreXPath, allowing your test scripts being robust against UI changes in the system under test.

Working in teams – not everyone has to be an expert!

Test automation is more than Capture & Replay, right? At same time, we know that not everyone in a test team has the skills to implement automated test scripts. For this reason, Ranorex offers different test automation approaches making it easier to work in teams. While domain testers might focus on which test cases have to be automated, automation experts concentrate on preparing reusable automation modules.

You don’t have any experts? No problem.

Ranorex Recorder is perfect to get started. It is a lot more than classic Capture & Replay. It allows users to simply creating flexible and reusable automation modules without hitting the ‘Record’ button.

Top benefits of using Ranorex test automation solution: www.ranorex.com/tour.html

+43 316 281328 info@ranorex.com Strassganger Strasse 289,

8053 Graz, Austria

www.ranorex.com

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

53

Seapine +44 (0)208 326 1840 salesuk@seapine.com Saracen House

Swan Street

Isleworth TW7 6RJ UK

www.seapine.com

Three tenets power Seapine Software in our quest to help you build better products: 1. The best process is yours - agile, waterfall, v-model, spiral, or any combination.

Chances are your development model is a hybrid, with some degree of agile. That’s where TestTrack shines, helping you capture, collaborate, and communicate using your process with one tool.

2. Traceability is essential to improving quality. It is the key to achieving early visibility of problems, staying on top of change (e.g., knowing which test cases are invalid when requirements change), and performing many types of analysis (impact, coverage, root cause, etc.).

3. Support customers as we’d like to be supported. From our award-winning product documentation to our incredibly knowledgeable and responsive sales, professional services, and technical support teams, we ensure your questions are answered and our solutions are meeting your needs.

Since 1995, Seapine Software has provided development and IT organisations with the tools, technologies, and best

practices needed to deliver quality software on time and on budget. Our application lifecycle management (ALM) solutions drive the development of recognised brands, life-saving medical devices, even games of the year.

Our product line includes TestTrack Pro for issue, defect, and task management, TestTrack TCM for test case management, TestTrack RM for requirements management, TestTrack Suite for hybrid agile project management, Surround SCM for version control, and QA Wizard Pro for automated functional testing and load testing.

Seapine Software is a multinational corporation with headquarters in Cincinnati, Ohio, and offices in Europe, Asia-Pacific, and Africa, and over 8,500 customers worldwide.

Soflab Soflab Technology is the biggest (200 people) and most experienced (>2000 projects delivered) Polish testing and QA company.

Our services include but are not limited to: • Functional software testing.

• Performance testing.

• Security testing.

• Mobile application testing.

• Test automation.

• Audits, advisory, quality management.

• Test tools, data & tests environments.

Soflab is an independent company with no affiliation with any IT system suppliers. Thanks to this our work is objective and concentrates 100% on our client’s requirements. By offering top-quality services, we support our clients in ensuring the reliability of implemented software and maintaining business continuity. We provide accurate reports on the quality of inspected IT solutions. This enables optimised relationships with suppliers with the introduction of KPIs and measures, which in turn can lead to reduced ICT expenditure. Our customers mainly consist of international telcos, media, banks, insurance companies, public entities and IT integrators. Soflab

is the first company in Poland that has won and implemented two large-scale full testing & QA SLAbased outsourcing projects (for an international telco group and major national public entity). Our employees hold over 150 ISTQB Certified Tester Foundation and Advanced Level certificates, as well as dozens of other IT certificated skill sets (including QAMP, Prince2, AgilePM, Professional Scrum Master, SAFe Consultant). Soflab’s experts take part systematically in main QA events in Europe to implement best practices in our projects in Poland and abroad.

Our strategy from 2015 is to expand significantly our operations in the EMEA region.

We are looking to connect with customers in EU countries who are interested in Nearshore testing and QA services delivered primarily from our offices in Poland, and/or are interested in the creation of your own Testing Center of Excellence (TCoE) with the support, experience and knowledge provided

by Soflab.

Our advantages are an extensive range of high quality of services, freedom of movement within the EU, a high level of technical & analytical skills, a customer focused approach, and highly competitive rates.

+48 22 211 26 02 soflab@soflab.pl 103 Ligocka,

Katowice 40-568 Poland

www.soflab.pl/en

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

54

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

Sogeti +44 (0)330 588 8200 info@uk.sogeti.com 85 London Wall

London EC2M 7AD UK

Sogeti UK part of the Capgemini group is a leading provider of local professional services, with over 13,800 testing professionals and 20 years of experience in delivering cutting-edge structured testing solutions that help customers achieve high quality, measurable and cost-effective results by increasing speed to market, enhancing software quality, mitigating risk and reducing costs.

A wide portfolio In the UK, Sogeti’s portolio comprises a range of traditional software testing and QA solutions, including: Test Process Improvement (TPI) assessments, functional and non-functional testing; performance testing; automation; and agile transformation and workshops, as well as cloud; web, mobile and security offerings.

All of our solutions can be delivered onshore locally, nearshore or offshore, and through our years of experience we have built state-of-the-art test processes, methodologies (TMap and TPI), reusable proprietary test frameworks and templates, test tools and test solutions.

www.uk.sogeti.com

Sogeti Studio Sogeti Studio is our UK-based web and mobile

testing lab, which offers clients the chance to test across a wide range of regularly refreshed devices, browsers and application configurations, almost on-demand and within a set time period – like a recording studio.

Flexible pricing We are able to offer a range of pricing models, from time and materials and fixed price, to outcome-based agreements.

Analyst recognition In the last 12 months, Sogeti and the Capgemini group have been recognised by a number of industry analysts as leader in our industry: • Ranked as a Leader in Ovum’s Decision Matrix: Selecting an Outsourced Testing Service Provider, 2014–2015.

• Positioned as a Leader in Transformation‑Focused Testing Services in the first NelsonHall Vendor Evaluation and Assessment Tool (NEAT) evaluation for Software Testing, 2014.

• Positioned as a Leader by Gartner Inc. in the inaugural Magic Quadrant for Application Testing Services, Worldwide, 2014.

Test Insane Test Insane Technologies Pvt. Ltd., based in Bengaluru, India, is a year old software testing firm, where we specialise in skill-based exploratory approaches to web and mobile app testing. Following the motto of Santhosh Tuppad, founder of Test Insane, we believe in being unique rather than different and so our services have not only been recognised locally but have a popular presence around the globe as well.

What we do?

We’re a team of 11 – 50 members with hands-on experience in software testing and a passion to give the best customer experience enabling their ventures to reach greater heights. At Test Insane we focus on various quality criteria like functional testing, usability & accessibility testing, security testing, compliance testing, performance testing, test automation and more.

That’s not all! We’ve always resented the distinction between software developers and testers in the community. That’s why; we decided to have a mixed breed of both at Test Insane, making possible a mutually benefitted learning experience as well.

Apart from this, we work towards giving back to our testing society for which we

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

continuously contribute to the Open Source community through development of test tools, micro-services, mind-maps repository to name a few.

Culture More than anything, we refuse to accept the fact that an organisation should only work in a hierarchical manner. As exploratory testers, we believe in talent more than experience or age and the fact that we call ourselves a team explains that every tester at Test Insane is crucial to every customer.

Value Addition is the mantra at Test Insane.

Whatever be the customer’s needs, we consider that our services will only be rendered as valuable if we take it a step further and add that value ourselves. For us, customer satisfaction means that a client never again has to come back with newer or recurring problems with their website/app. So far, we’ve only enjoyed delightful conversations with happy customers over coffee!

When we say that we’re passionate about what we do, we mean every word of it. But along with passion, testing drives us to the brink of insanity (it’s what we love) and that’s exactly why we’re called TEST INSANE!

+91 98 8095 2643 st@testinsane.com 184/15, 2nd Floor (Willy’s Nest), 5th Main, 30th Cross

Jayanagar 4th Block, Bengaluru India

www.testinsane.com

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

55

Testuff +972 3600 6015 contact@testuff.com 4 Shoham Ramat Gan

69641 Israel

Testuff is a SaaS test management tool for manual and automation software testing. It includes all a tester would expect from a test management solution, and then some more.

Over 5,500 testing teams who tried it, over 3,150,000 stored tests, 81 consecutive monthly upgrades, 10 data centres, and uptime of 99.98%.

8 years track record of excellence.

Developing together with users Servicing its community since 2007, and working closely and continuously with users, Testuff has been able to create a mature, fully‑featured solution which is designed directly to improve the tester’s work, and the testing projects management. Minimal bells and whistles and emphasise on providing an easy‑to‑use and intuitive GUI, is a strategy of Testuff since inception.

www.testuff.com

Testuff brings benefits to the testing community Testuff customers, who come from 5 continents, and are enjoying globally spread 10 data centres, work interchangeably with the two clients offered (desktop and web), and have control of how, when and where they use it. Being a SaaS tool helps

customers focus on their core business, and their testing. No need for local servers, installations, periodical upgrades efforts, maintenance, etc.

Testuff takes care of it all, and provides a clean and quick way to test. This, and the very short learning curve is a time‑and‑money saving point for companies. As an industry leading vendor, Testuff was the first to introduce a two‑ways integration process with any bug tracker, and a unique embedded video recorder for defect reporting.

By being a flexible solution, Testuff enables testing groups to implement any testing methodology, including agile, waterfall, blackbox and exploratory testing.

Experience and motivation Testuff team includes experienced industry veterans who have many years of experience in all aspects of software development and testing.

Working on both desktop applications and websites, Testuff has used tools and techniques including suites by all leading vendors and most agile methodologies. The goal is to create smarter software testing tools to help customers build more usable and stable software for the benefit of their clients.

56

2 0

L E A D I N G

T E S T I N G

P R O V I D E R S

2 0 1 5

Tricentis +44 (0)844 80 79 905 office@tricentis.com Tricentis UK Ltd.

1 Bedford Row

London WC1R 4BZ UK

www.tricentis.com

The continuous testing company Tricentis, the continuous testing company, provides leading enterprise software testing solutions for Global 2000 companies. Their testing solution, Tosca Testsuite, empowers enterprises to accelerate their business innovation via agile development methodologies. Tricentis’ model-based approach to software test automation enables enterprises to achieve unprecedented automation rates while maximising their business risk coverage.

Tricentis’ continuous testing solution leverages market-leading model based test design and planning, test data provisioning, and automation for end-to-end testing of modern applications across web, mobile, .Net, Java, and several legacy technologies. Tricentis delivers a 10X improvement over traditional script based testing solutions and is poised to disrupt the US$60 billion annual spend on software testing services that are primarily delivered via outsourced manual testing.

Tosca Testsuite Tosca Testsuite addresses the challenges in Endto-End (E2E) testing that result in high costs and delays in time-to-market by optimising, managing, and automation your testing. Tosca Testsuite begins with Risk Coverage Optimisation; minimising

your overall test portfolio while maximising the amount of business risk the tests cover. Next, the Test Data Management tool provides you with a fully integrated set of capabilities for the design, generation and provisioning of test cases. Finally, Model-Based Test Automation allows you to build robust, reusable automated test cases – all without requiring technical expertise.

In addition, Tosca Testsuite provides a wealth of other tools and resources, such as API testing, Mobile testing, Orchestrated Service Virtualisation, hands-on training, and much more.

Awards and customers Gartner recognises Tricentis as a Visionary in their 2014 Magic Quadrant for Integrated Software Quality Suites Report. In 2015 Forrester Research named Tricentis a “Strong Performer” in Functional Automation Tools while the CIO Review recently called Tricentis one of the “20 Most Promising DevOps Solutions Providers of 2015”.

Tricentis’ more than 400 customers include global brands such as AMEX, Allianz, BMW, BofA, ING, Deutsche Bank, HBO, Orange, Swiss Re, UBS, and Vodafone. Tricentis has offices in Austria, the US, Germany, Switzerland, the UK, Netherlands, and Australia.

Vornex Vornex is a provider of enterprise software testing solutions. Our flagship product, TimeShiftX is a date and time simulation testing software that lets you time travel software applications into the future or past for temporal testing or time shift testing to validate all date and time sensitive functionality and code such as year-end, daylight savings, leap year, billing, rates, policies, etc.

TimeShiftX enables time travel (inside Active Directory/Kerberos) without code changes, manual work, or server isolation in order to perform your forward date testing or back date testing with features including: • Instant time travel No code changes, no altering server clocks, no isolating servers. Just turn on TimeShiftX & begin temporal testing.

• Active directory compatibility Safely time travel inside Active Directory, Kerberos, LDAP, and other domain authentication protocols.

• Total app and DB compatibility TimeShiftX enables time travel for all applications & databases such as SAP, SQL,

T E S T M a g a z i n e | S e p t e m b e r 2 01 5

Oracle, WAS, .NET, and others.

• Cross platform and cloud time travel TimeShiftX is compatible on all platforms & operating systems and can be run in the cloud.

• Distributed environment time shifting TimeShiftX allows you to easily temporal test large, distributed software stacks and environments.

Vornex services worldwide clients with time travel who extend many industries including banking, finance, healthcare, state & local government, utility, and insurance. In addition, Vornex works with industry leading IT consultants, 3rd party integrators, and VARs such as Accenture, CapGemini, Wipro, IBM Global Services, etc.

All these industries and enterprises rely on TimeShiftX to validate their date sensitive applications and accelerate their time of delivery. Vornex enables organisations to achieve increased efficiency, reduced resources, and improved corporate bottom line by providing TimeShiftX as an affordable solution that can fit any company’s IT budget.

+1 408 713 1400 info@vornexinc.com 43575 Mission Blvd.

Suite 613

Fremont, CA 94539 USA

www.vornexinc.com

Unique Outsourcing Partner

We deliver tailored ROI-effective Test Automation solutions with Ranorex on top

Driving QA efficiency, we enable customers to significantly speed up the products’ time to market.

ranorex.eleks.com

ELEKS’ Automation Testing services are powered by seasoned experts and Ranorex-certified professionals who can tackle the toughest technology challenges, building an efficient Automated Testing environment for web, mobile and desktop products of any scale and complexity.

We have delivered significant value through Test Automation for our clients. We can do the same for you.

Talk to our expert!

Nazariy.popov@eleks.com Find ELEKS among top ten software testing firms at Clutch.co

For more information, please visit eleks.com/auto-test

SUCCESSFUL SOFTWARE DELIVERY

BUILD ON YOUR MANUAL TESTING QUALITY WITH AUTOMATION Manual testing is great, so far as it goes. The problem is, it can’t scale to keep pace with development and rapid Agile release cycles. So what if you extend the reach of human testing through automation?

With a hybrid environment from Borland, you integrate requirements, manual and automated testing. So when manual testing struggles to keep up, you get a helping hand to increase testing speed while maintaining quality – across all of your different devices, platforms and OS versions. It’s automation at its best, with Borland’s human touch.

‘The human side of test automation’– download your FREE White Paper at www.borland.com/testautomation

CONNECT YOUR PEOPLE FOR SUCCESSFUL SOFTWARE DELIVERY Copyright© 2015 Micro Focus. All Rights Reserved.

Portions Copyright © 1994-2009 Borland Software Corporation (a Micro Focus company).