INNOVATION FOR SOFTWARE QUALITY VOLUME 5: ISSUE 5 OCTOBER 2013 THE EUROPEAN SOFTWARE TESTER
INSIDE: VIEWPOINTS: HOW BENEFICIAL IS COLLABORATIVE TESTING? TESTING IN PRACTICE FOCUS ON: BUSINESS
www.testmagazine.co.uk
“FACE VALUE” TAKES ON NEW MEANING
CONTENTS
INSIDE THIS ISSUE
NEWS 6
Current tools can’t meet mobile apps demand
WORLD NEWS 8
I s Apple already testing a new operating system?
VIEWPOINTS 10
10. HOW BENEFICIAL IS COLLABORATIVE TESTING?
How beneficial is collaborative testing? Vijay Balasubramaniam and Hardeep Garewal explain how the rise of collaborative technologies and social media tools overcome barriers to allow software testers around the world to successfully work together…
18.
INTERVIEW 12
AVOID OUTSOURCING PITFALLS TO DELIVER SOFTWARE APPLICATION INNOVATION
The changing face of software testing
Rajesh Sundararajan shares his thoughts on the changing face of software testing…
TESTA 14 TESTA finalists announced!
After months of anticipation, The European Software Testing Awards is delighted to announce the finalists…
THOUGHT LEADERSHIP
18
Avoid outsourcing pitfalls to deliver software application innovation Chris Livesey discusses the importance of business requirements when outsourcing software application projects and the need for requirement management tools…
COVER STORY 22
22. "FACE VALUE” TAKES ON A NEW MEANING
"Face value” takes on a new meaning
Sophie-Marie Odum investigates the testing of a revolutionary new mobile app that allows customers to pay for goods with their face…
TESTEXPO PREVIEW 24
An interview with Stefan Gerstner
Vice President of Global Service Line Testing at Sogeti, hosts of TestExpo 2013, shares his visions and expectations for this upcoming event…
OCTOBER 2013 | www.testmagazine.co.uk
PAGE 1
CONTENTS
MOBILE APP TESTING
26
28.
The diary of a mobile load tester
MOBILE APP TESTING: THINK YOU’VE GOT IT COVERED?
Henrik Rexed shares a diary, describing how to test a networked mobile. He provides a detailed insight into real-world load and performance testing, the methodology, what is actually tested, how it is tested, and the pitfalls inherent in this type of project…
TESTING IN PRACTICE
30
The challenges of testing self-service technology
38.
Paul Gardiner addresses the challenges of testing self-service technology…
AGILE DEVELOPMENT
31
Avoid the “pull of the past”
EXPLORING FOR THE DEEPEST, DARKEST DEFECTS
Sophie-Marie Odum speaks with Phil Knight to find out why many companies are yet to adopt an agile approach to development…
FOCUS ON: BUSINESS
40
The power of small wins
Ros Taylor explains how managers and team leaders can ensure excellent team performance, and how the smallest changes can differentiate a successful team from a not so successful team…
45. THE 20 LEADING SOFTWARE TESTING PROVIDERS
the 20 LEADING software TESTING PROVIDERS
201 3
OCTOBER 2013 | www.testmagazine.co.uk
FOCUS ON: BUSINESS
42
How much do you know about tax?
Many contractors working in the software testing industry lack awareness of complex tax rules and how best to stay compliant. Adrian Learer explains the issues that freelancers need to be aware of…
LAST WORD 43
The evolution of the “Super Tester”
Dave Whalen explains why it’s soon time that you get fitted for a cape…
THE 20 LEADING 45 SOFTWARE TESTING PROVIDERS TEST presents the leading providers in the software testing industry...
PAGE 3
Novus
the Young g in or s n po S e r a We ear Award Tester of the Y
Flexible, cost effective resources to meet the peaks and troughs of your projects Capita Novus trains new test analysts in the latest technologies and supplies them to organisations looking for:
Cost Effective Resource v Low risk fixed cost v Cheaper than contractor market
Flexibility v Benefits of permanent employees with the flexibility of contractors
Reduced Risk v UK based staff v Employment responsibilities lie with Capita v Option to employ at the end of a fixed term
more? Want to find out Call: +44 (0) 8456077466 Email: Info.at@capita.co.uk
www.capita-at.co.uk/novus
LEADER
SPOT THE DIFFERENCE… Since joining the software testing sphere, I have noticed many similarities between the role of a software tester and that of an editor.
N
ot only is it essential that we are both passionate about our roles, but we also both have the responsibility to develop strategies for the future direction of our projects. Not forgetting that strict attention to detail is paramount as is the rigorousness and the ability to break stuff! However, in my case, it’s the ability to dissect editorial and extract quality content to ensure readers attain an objective, interesting and thought-provoking article or news story, with an aim to strike debates. Whilst we have similarities in the requirements needed for a successful outcome, the expertise and knowledge necessary for software testing professionals is complex yet equally unique. With this being said, we recognise that there is a demand for an allencompassing website that can help software testing professionals keep up-to-date with what's going on in the industry, including the latest testing tools that could help enhance performance. This is why we are excited to announce that our fresh, new Software Testing Hub is now live at www.testmagazine.co.uk. From the Hub, you can visit one of our three, newly-designed, revamped websites: you can choose to read the latest software testing news and features; learn more about TEST magazine and take part in a monthly poll; or find out more about The European Software Testing Awards (TESTA). Whilst on the subject of TESTA, we are also delighted to reveal the finalists (please see page 14). Following hours of deliberation, the judges had a tough job choosing their finalists for each category. The winners will each be announced on the 20th November at a black-tie ceremony at the Marriott Hotel Grosvenor Square, London. If you would like to attend this event, please visit www.softwaretestingawards.com for more information.
Do you want to write for TEST magazine? Please email sophie. odum@31media.co.uk
I hope you enjoy this issue!
Sophie-Marie Odum Editor
© 2013 31 Media Limited. All rights reserved. TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited. ISSN 2040-01-60 T H I R T YO N E
OCTOBER 2013 | www.testmagazine.co.uk
EDITOR Sophie-Marie Odum sophie.odum@31media.co.uk Tel: +44 (0)203 056 4599 TO ADVERTISE CONTACT: Sarah Walsh sarah.walsh@31media.co.uk Tel: +44(0)203 668 6945 PRODUCTION & DESIGN Tina Harris tina.harris@31media.co.uk
EDITORIAL & ADVERTISING ENQUIRIES 31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PD Tel: +44 (0) 870 863 6930 Email: info@31media.co.uk Web: www.testmagazine.co.uk PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA
PAGE 5
NEWS
CURRENT TOOLS CAN’T MEET MOBILE APPS DEMAND A global survey found that while a majority of Windows developers are experiencing great demand for mobile development, the actual delivery of these apps is being compromised due to the cost and complexity associated with supporting multiple platforms. The survey, sponsored by Embarcadero Technologies and conducted by Dimensional Research, comprised of 1,337 Windows developers from around the world, revealed that 85% of Windows developers receive requests for mobile apps, whilst 99% say existing apps must continue to be supported. According to the company, this
demonstrates a disconnection between the growing interest in mobile apps and the lack of tools available to actually develop them. Tony de la Lama, senior Vice President of marketing and R&D at Embarcadero Technologies, said, “This independent, global survey of application developers confirms the brutal challenges they’re facing as the shift from desktop apps to connected mobile devices unfolds. Developers have been telling us they must support existing desktop apps and at the same time make them available on mobile devices.” Key findings from the report also found that Android is the most requested
QA AND TESTING NOW ACCOUNTS FOR ALMOST A QUARTER OF IT SPENDING Application testing and quality assurance now stands for 23% of total IT spend, according to a global study, which examined the state of application quality and testing practices across multiple industries and geographies. In addition, Capgemini and Sogeti’s 5th World Quality Report found that UK companies continue to embrace outsourcing on a large scale and at a much higher rate than the rest of the world. Respondents indicate that 26% of testing practices is outsourced to external vendors, compared to an average of 18% among Western European countries and 22% within the Nordic region. UK organisations embrace outsourcing on a large scale at a higher rate than the rest of the world (20% worldwide average as a proportion of the engagement model). Also, it was discovered that the UK is among the leaders in adopting cloud technologies. On average, an organisation has 22% of its applications migrated or hosted in the cloud. Larger corporations are also taking advantage of Testing as a Service (TaaS), and are at the forefront of using the cloud for application testing. Overall, UK respondents report that nearly a
PAGE 6
quarter (23%) of application testing occurs in the cloud today – down from 25% reported last year – representing businesses without existing extensive infrastructure investments. More than half (56%) of UK businesses indicate that they test mobile applications and devices. UK firms report that the biggest obstacle to mobile testing is a lack of adequate processes and methodologies (63%), followed by device availability (57%) and insufficient time to test (49%). Brian Shea, CEO of Sogeti UK, claims that spending on QA has increased in the UK as companies are increasingly reliant on IT systems and applications to support their core business functions without disruption, so are taking a more strategic, centralised and business-led approach to QA. He said that organisations are realising the value QA brings by capturing metrics related to wider business ROI, such as reduced time to market (45%) or cost savings by preventing defects (39%). This shift has seen the decision to invest in QA move from IT managers to wider areas of business, such as marketing and board-level decision makers
platform among Windows developers (83%); HTML5 and JavaScript are not the solution of choice for mobile apps as 85% of respondents say native apps are best for end-users, and 74% of those who used HTML5 and JavaScript have had challenges. Diane Hagglund, senior researcher at Dimensional Research, added, “Windows developers clearly see the need to bring their deep experience with application development to mobile applications. Today’s development options either limit the end-user or result in costly and complex native development across multiple platforms. These Windows developers clearly need better options.”
NEW WHITE PAPER FOR SOFTWARE DEVELOPERS Best Practices in Automated Software Testing: Four Key Steps to Success, a complimentary White Paper for software developers, is now available.
IT PROVIDES A SERIES OF SPECIFIC, “BEST PRACTICE” APPROACHES AND DEVELOPER TIPS
In this white paper, VIAcode Consulting author and senior developer, Sergey Mikhalev, provides compelling reasons for software developers to outline a testing approach for their new business applications at the conceptual stage of the application – rather than at the end. He then offers a thorough discussion of how to design and build an effective software testing strategy and plan that will ensure a stable, bugfree application. The paper focuses on four key software testing areas: manual vs. automated approaches; testing API’s vs. UIs – when and why; the value and extent of unit tests; and why the testing strategy needs to evolve from one application to the next. Within each of these topics, Mikhalev is said to provide a series of specific, “best practice” approaches and developer tips. “Most software programmers do not even think about testing what they have built until the final stage of development is complete,” explained Mikhalev. “Unfortunately, this approach does not always ensure that the resulting application is flexible, maintainable and stable.”
OCTOBER 2013 | www.testmagazine.co.uk
NEWS/WORLD NEWS CROWDSOURCING SHARES THE LATEST CYBER THREATS It would seem that crowdsourcing and open source are being accepted as ways of getting the information security industry to share the latest cyber threat intelligence This follows a recent announcement that CrowdStrike and AlienVault are the two latest recipients of venture funding to help make corporate networks less “open” to data thieves and cyberspies, and HP’s new Threat Central service, AlienVault claim to have pioneered this concept as open source is something that it has heralded for years - its Open Threat Exchange (OTX) platform was built on the principle that “if I help others, I further my own goals.” Barmak Meftah, President and CEO of AlienVault, and former VP of HP’s Software Security Products division, commented, “We welcome this week’s announcement from HP about their Threat Central service. We see this as further validation of what we’ve known for a long time: Crowdsourcing or open source threat intelligence is the
only way organisations have any hope of combating the ‘bad guys’. “With over 8,000 contributors from more than 140 countries sharing threats every day through our Open Threat Exchange, we’ve learned first-hand that being open and collaborative are the essential requirements to sharing and disseminating the comprehensive threat intelligence that no one company could ever collect in isolation. The era of closed systems and proprietary enterprise solutions to address the security concerns of organisations around the world is over.”
SUCCESSFULLY FINDING AND FIXING DEFECTS More than 20 code defects were caught within three weeks of testing the open source Java project ANTLR (ANother Tool for Language Recognition). Coverity analysed the ANTLR open source Java project, including defect density as compared to the industry average defect density for good quality software and types of defects identified.
DO YOU THINK OUTSOURCED OFFSHORE TESTING IS DECLINING? Take part in this month's poll online at www.testingmagazine.com
Although the ANTLR project only started using the company’s scan service in late August 2013, it has already found and fixed 20 previously undiscovered, high- and medium-risk defects, including a resource leak and copy-paste error that could have caused a significant software crash in production. “ANTLR is one of a growing number of Java open source projects that have joined the Scan service to help enhance code quality,” said Jennifer Johnson, chief marketing officer for Coverity. “The ANTLR team has done an excellent job of addressing key defects in their code in the short time that they have been participating in the service, and we look forward to continuing to work with them to ensure that their Java code is of the highest quality.” ANTLR is a Java-based parser generator for reading, processing, executing or translating structured text or binary files. The software, which is used to build languages, tools and frameworks, is downloaded more than 5,000 times per month and is used by a host of companies, including Apple, Oracle, Salesforce.com and Twitter.
PLANS TO DOUBLE R&D TEAM The global provider of information, analytical tools and marketing services, Experian, plans to double its R&D team in Bulgaria. At present there are 270 specialists in 10 different departments, 60 of them developers, working in the Bulgarian office.
OCTOBER 2013 | www.testmagazine.co.uk
After expanding the team of developers, the Bulgarian R&D centre will become the biggest in the Decision Analytics division of the company, according to the company. The hiring of the new software developers and QA specialists will be done next year.
PAGE 7
WORLD NEWS IS APPLE ALREADY TESTING A NEW OPERATING SYSTEM? Apple has just released the latest version of the iPhone, which comes with the latest operating system software iOS 7. However, a new report suggests Apple employees already began testing iOS 7.0.1 and iOS 7.0.2 ahead of the iOS 7 release date. Smaller iOS updates - iOS 7.0.1 and iOS 7.0.2 – which are typically much smaller of an update may just be a couple of minor bug fixes or security updates for the handsets, following alleged reports that iOS 7 GM is crashing more than expected. In addition, it was also recently exposed that Germany’s Chaos Computer Club
A NEW REPORT SUGGESTS APPLE EMPLOYEES ALREADY BEGAN TESTING IOS 7.0.1 AND IOS 7.0.2 AHEAD OF THE IOS 7 RELEASE DATE
has already successfully bypassed the biometric security of Apple’s Touch ID (fingerprint recognition system). The Club claimed that by photographing a fingerprint left on a glass surface and creating a fake finger they were able to unlock the phone. However, Apple maintains Touch ID is secure. It says there is a one in 50,000 chance of two separate fingerprints being alike and the technology provides “a very high level of security”.
GLOBAL SOFTWARE TESTING SERVICES IN BFSI MARKET WILL SEE GROWTH The global software testing services market in BFSI (Banking, Financial services and Insurance) will grow at a CAGR (compound annual growth rate) of 6.91% over the period 2012-2016, according to The Global Software Testing Services Market in BFSI Sector 2012-2016 market report. It’s thought that one of the key factors contributing to this market growth is the need to reduce operational time and cost as this sector has been witnessing a shift from on-premise software testing to cloud-based software testing. However, it’s been noted that a lack of data control in outsourced services could pose a challenge to
the growth of this market. Global Software Testing Services Market in BFSI sector 20122016 has been prepared based on an in-depth market analysis with inputs from industry experts. The report covers the market in the Americas and the EMEA and APAC regions; it also covers the landscape of the global software testing services market in BFSI sector and its growth prospects in the coming years.
DEMAND FOR TESTING SERVICES CREATES 75 JOBS 75 specialist roles are to be created in Dublin over the next two years as SQS sees further demand for its testing services. The German company already employs 200 people in Ireland with offices in Dublin and Belfast since 2002, and the new jobs will see 50 employed in Dublin and 25 based in the Belfast office. The company is looking to recruit up to 30 positions before the end of this year. The jobs being created are high-end, specialist roles encompassing software quality engineers, software testing managers, business analysts, performance engineers and software test analysts. CEO of SQS Group, Dik Vos, said the company was expanding due to the development of the overall IT industry in Ireland, and acknowledged that sourcing talent in the competitive IT field remains a problem in Ireland, but said the company has taken steps to ensure that they are developing talent to meet their demands. “There is no doubt that recruitment of such specialist roles remains a challenge,” said Vos. “However working closely with Ireland’s third-level colleges and initiatives such as the SQS Graduate In-Take Programme, have proved very successful and will remain a key part of our recruitment strategy, providing valuable training and investment in developing key skillsets to support businesses in the Irish economy to the required SQS quality standards.”
Q-LAB TMMI CERTIFIED NEW OFFICE CREATES NEW JOBS Expecting to create 40 new, full-time jobs in the next year, a software testing company from California will open a new office in Iowa City, Shasta QA, which tests software, websites and smartphones for clients in various industries, will be hiring software testers around eastern Iowa during the upcoming months. Director, Jeff Lord, said, “There has always been amazing talent here, but with new opportunities in innovation, collaboration and entrepreneurship, these professionals are staying and developing their ideas here. We want to be part of that.”
For the latest news visit softwaretestingnews.co.uk and follow us @testmagazine
PAGE 8
The Malaysian Q-Lab has received TMMi (Test Maturity Model Integration) Level 5 assessment, the highest certification level, and it is said to be the third testing facility in the world to have achieved this level. Malaysian Software Testing Board's (MSTB) President Mastura Abu Samah, said, “MSTB’s Q-Laboratory has passed its TMMi Level 5 assessment in August 2013. This makes Q-Lab the third testing facility in the world to have achieved this level, which is the highest in the TMMi certification model.”
OCTOBER 2013 | www.testmagazine.co.uk
www.softwaretestingawards.com
THE EUROPEAN SOFTWARE TESTING AWARDS
CELEBRATING TECHNICAL EXCELLENCE
Headline Sponsor
BOOK YOUR TABLE TODAY!
THE EUROPEAN SOFTWARE TESTING AWARDS
CELEBRATING TECHNICAL EXCELLENCE
An independent awards programme designed to celebrate and promote excellence, best practice and innovation in the software testing and QA community.
Will you be joining Lloyds Banking Group, Wincor Nixdorf, Brickendon Consulting, A1QA and many others at The European Software Testing Awards? Come and celebrate the best of the best in your profession with some of the biggest names in the software testing industry. Book your table today! Please contact Swati Bali, event sales manager on +44(0) 203 668 6946 or email swati.bali@31media.co.uk
VIEWPOINTS VIJAY BALASUBRAMANIAM GLOBAL HEAD OF TESTING PRACTICE ITC INFOTECH
HARDEEP GAREWAL PRESIDENT, EMEA OPERATIONS ITC INFOTECH
HOW BENEFICIAL IS COLLABORATIVE TESTING? Vijay Balasubramaniam and Hardeep Garewal of ITC Infotech explain how the rise of collaborative technologies and social media tools shatte language, geographical and time zone barriers to allow software testers to successfully work together and share information… n the wider world, social media is revolutionising everything from our personal relationships to the way products and services are marketed. Software engineering in general and testing in particular have always been highly collaborative activities and now social media is providing new ways for testers to collaborate and work across geographical, temporal and cultural boundaries, democratising processes that were previously in the control of just a few stakeholders.
I
Social media tools – be they public forums like Facebook and Twitter or more restricted internal collaboration tools like Sharepoint – provide an “architecture of participation”. These tools offer a collaborative framework suited to any distributed network of testers, but they are especially useful in a crowdsourcing context. Crowdsourcing is becoming a dominant force as one of the key practices in the testing of mobile apps. This offers testers with expertise in the necessary range of platforms operating systems and hardware that the mobile apps market requires. There are of course other benefits to be gained from collaboration techniques. When used in the software development process, they can help organisations to innovate faster; improve productivity through a community architecture and the reuse of shared software assets; and cut costs by reducing the number of applications, servers and administrators and eliminating shadow IT. When we use the term “social media”, in the context of crowdsourcing and testing, we are not necessarily referring to the more high-profile products like Facebook and Twitter because they would be much too “open” and public for discussion of bug fixes and business issues. However, when using our internal crowdsourcing model for mobile testing, social media techniques are an excellent method for onproject collaboration. In our testing centre of excellence (TCoE) for mobility where we develop and test the mobile applications for use in our parent company, we use a range of collaborative social media techniques to share information and best practice. The apps need to be tested on many different devices and operating systems so we run a real-time testing phase
PAGE 10
where we involve all our testing groups around the globe to try out mobile apps on various devices in various time zones and languages, all collaborating though platforms like Sharepoint.
THE POTENTIAL FOR COLLABORATION USING TOOLS LIKE SHAREPOINT IN REAL-TIME IS HUGE
The potential for collaboration using tools like Sharepoint in real-time is huge. Open source tools can help testers across various geographical locations collaborate on projects, logging defects and keeping track of the quality of the applications under test. We use Sharepoint for sharing everything from the planning of tests to execution and discussing defects and corrections. We use it as a platform to store our information and make it accessible to all the people working on a problem at the same time. Information can be updated in real-time so that the most current issues and fixes are shared as and when they are developed. As well as Sharepoint, there is a range of other internal mechanisms for collaboration. We use forums and a Wiki-style resource for sharing research information and tool-based innovations which allow us to create forums of experts to tackle any issues as they occur. From our personal experiences, and from the way we use social media to help make buying decisions or to get feedback on products and services, social media is having a major impact on the way customers perceive companies. This includes software and IT service companies too of course. Inside testing organisations, collaboration tools make working across traditional boundaries much easier, and the next generation of testers are set to be far more collaborative on their mobile devices. As ‘Generation Y’ enters the workforce it is demanding the kinds of networking and collaboration tools that they have grown up with. Collaboration across a range of social media is going to be critical to the future of software testing.
OCTOBER 2013 | www.testmagazine.co.uk
INTERVIEW RAJESH SUNDARARAJAN PRACTICE HEAD, TESTING SERVICES MARLABS SOFTWARE
THE CHANGING FACE OF SOFTWARE TESTING Sophie-Marie Odum interviews Rajesh Sundararajan, practice head, Testing services, Marlabs Software, to find out his thoughts on the changing face of software testing… How do you think software is changing, in comparison to five years ago? Software has evolved and impacted businesses and everyday lives to such an extent that today, it is everywhere and has put computing power in the hands of almost everyone. Software applications have been evolving from the older, monolithic structure to the current scenario where most enterprise software TESTERS NEED constitutes distributed TO MAKE THE
TRANSITION FROM APPLICATION FUNCTIONAL EXPERTISE TO BUSINESS DOMAIN EXPERTISE
components involving heterogeneous technologies with scalable architectures. While technology has been a constant change, the recent trend has seen lot of new open- source technologies, application development frameworks, use of pre-packaged solutions, etc. Some of the recent trends which we observe across the various dimensions of software are: • Software is increasingly accessible from multiple
platforms and devices. The same web application is now available on a variety of desktop, mobile and smart devices.
• Localisation and internationalisation of applications to
cater to diverse markets has become a necessity and needs applications to be appropriately designed and coded for it.
• Software applications are increasingly becoming
available as a service. They are hosted on the cloud, on virtual machines and available on a pay-per-use model.
• Software systems – especially in some of the
newer industry segments – are much more interactive, collaborative, and provide rich and personalised content.
• Software increasingly uses and generate huge
amounts of data. Handling huge volumes of data, making sense of the data and using it for intelligent decision has led to the evolution of new technologies like analytics and Big Data.
Why do you think such changes have occurred? Ultimately, technology is driven by business. The rise of new business opportunities, access to global markets and a competitive business environment has led one set of changes where software applications have to meet the needs of diverse markets with their different languages and changing compliance regulations, and also engage fast growing new customer segments like social media for their unique needs. Technology innovations and related consumer behaviour, like the massive adoption of a variety of mobile and smart devices, means that software has to adapt and seamlessly PAGE 12
OCTOBER 2013 | www.testmagazine.co.uk
INTERVIEW
integrate across these devices and platforms, providing end-users a consistent experience across platforms.
What can organisations do to help software testers meet such demands and adjust to changes?
The growth of the Internet has led to a huge user base of applications, which need to be scalable and efficient. This has also led to extremely large volumes of data being used and created, which businesses need to be able to analyse and interpret for intelligent decision making.
The first is continuous training. Apart from the testing techniques, tools and processes, testing teams need training on technology and business topics and software development methodologies.
One of the key challenges for CIOs is to rationalise costs and maximise return investment. Cloud-based on-demand infrastructure and SaaS models have been effective options for CIOs to achieve this.
How have these changes affected the industry? We see the development of many new streams of technology – Enterprise Mobility, Cloud, Analytics, Big Data, HTML5, RWD(Responsive Web Design) to name a few. While these are enablers in solving business problems, they also come with their own challenges including a learning curve. Ensuring quality has meant the development of corresponding testing strategies for each with its own tools, methodology, and best practices. There is a lot more focus on the non-functional side of software; right from requirements, design, development and testing. This includes performance, usability and security, etc. to name a few.
The second relates to involvement of testing teams in the early stages of the project rather than at a later time. This facilitates the process of building quality into the application right from the early stages. The third is regarding project governance. There needs to be a clear definition of a stage-gating process (at each software development phase) and stringent adherence to it. This will drive the importance of testing and provides much needed support to the testing team.
Do you think software testing gets the recognition it deserves? The recognition software testing gets, or not, is a function of: • How critical is quality in the particular industry or
domain? In certain areas of avionics, financial services and healthcare, the cost of failure is extremely high and testing teams are highly valued compared to their counterparts in “less critical” environments.
The need for faster time to market and the ability to respond quickly to business needs has led to the greater adoption of agile development methodologies like “Scrum” where developers and testers work in an environment of collaborative, continuous and iterative development.
• What is the level of “process-orientation” and usage
Cost, skills and team scalability factors have led to the phenomena of distributed engineering teams and outsourcing, including offshoring. Software teams are getting more diverse and a younger workforce is also making its presence felt, especially in new technology areas.
• In a lot of project situations, testing starts off as a
How can software testers meet the demands of today’s processes? Testers need to make the transition from application functional expertise to business domain expertise. This helps them understand the business context and “big picture”; gain an integrated view of systems and their interdependencies; and better appreciate the facets of quality. A significant portion of software testing today is technology specific and technology intensive. Here, testers need to have a fairly good understanding of the relevant technology, such as in the case of performance testing, data warehouse testing, etc. Tools are a valuable component of testing and we see a wide range of options – commercial and open source – for different types of testing. Testers need to have good knowledge and experience of tools in their testing area in order to be able to make informed choices and leverage their capabilities. Finally, testers have to learn to adapt to work in an increasingly agile environment and in close interaction and collaboration with developers and the rest of the software project teams.
OCTOBER 2013 | www.testmagazine.co.uk
of metrics? Process-oriented teams often value the testing team’s contributions; and metrics provides a way of better understanding and articulating the benefits of testing.
secondary activity compared to development. When the tester is happy with such a situation, this is driven by the development or project team. Therefore, in my opinion, he/she is not valued as much as someone who demonstrates expertise (functional and technical) and functions as an independent QA.
So, while some aspects are beyond one’s control, in a lot of cases, the value of testing is down to the performance of the testing team itself.
Opportunities vs Challenges The current situation of business complexity and technology change has created a lot of opportunities as well as challenges for testers. Some of the things testers will have to do well include: 1. Clearly understand the value of testing and critical aspects of quality in their application context. Focus on the successful delivery of these to create value for the application and project team. 2. Demonstrate expertise – functional, technical and process – to be able to work as valued partners on projects. 3. The value of testing is not always obvious. Use metrics, such as quality, productivity, operations and RoI effectively to clearly state the value of the testing team.
PAGE 13
TESTA
TESTA FINALISTS ANNOUNCED! THE EUROPEAN SOFTWARE TESTING AWARDS
CELEBRATING TECHNICAL EXCELLENCE
After months of anticipation, The European Software Testing Awards is delighted to announce the finalists... Headline Sponsor
After sifting through numerous entries, and hours of deliberation, the judges have chosen their finalists for each category, and the winners will be announced on the 20th November at a black-tie ceremony at the Marriott Hotel Grosvenor Square, London.
This promises to be a great event and provides a wonderful way to thank your team and celebrate all that is best in a profession that rarely gets the full credit it deserves. Will you be there to witness support your colleagues and be a part of the first ever TESTA?
Sophie-Marie Odum, Chair of the judging panel, said, “The judges had a tough choice in choosing their winner for each category, and in some categories it was so close, illustrating the high calibre within the software testing industry. The judging day was TESTING a successAWARDS and we look THE EUROPEAN SOFTWARE forward to announcing the winners in November.�
Come and celebrate with peers, like-minded individuals and key opinion leaders who are keen to make a difference in the industry, as well as see what the industry has to offer.
CELEBRATING TECHNICAL EXCELLENCE
To book a table, please contact Swati Bali, event sales manager on +44(0)203 668 6946 or email swati.bali@31media.co.uk
Please turn page for finalists...
PAGE 14
OCTOBER 2013 | www.testmagazine.co.uk
Software quality tools from design to production
Easy-to-use tools which empower designers, developers, testers and managers to reduce time to market improve quality save money
from mainframe to mobile
built by THE QUEEN’S AWARDS FOR ENTERPRISE INTERNATIONAL TRADE 2013
TESTA
The finalists are as follows:
The TechExcel Best Agile Project • Sopra Group & Student Loans Company • Datacastle • Lloyds Banking Group in partnership with Cognizant The eggPlant Best Mobile Project • Lloyds Banking Group in partnership with Cognizant • PokerStars • The Financial Times Limited • DMG Media The Sogeti Best Automation Project • Maveric Systems • Brickendon Consulting • Tata Consultancy Services • Cigniti Technologies Best Overall Testing Project – Public Sector • Knowit • Sopra Group & Student Loans Company Best Overall Testing Project – Finance • MagenTys • LMAX Exchange • Exeter Family Friendly • A1QA The Capita Best Overall Testing Project – Retail • Tata Consultancy Services • John Lewis IT • Wincor Nixdorf
Green Testing Team of the Year • Cigniti Technologies • Cognizant
Young Tester of the Year • Andrew Thompson, Cognizant • William Gibbons, Sogeti
The UKTB Testing Manager of the Year • Chris Comey, Testing Solutions
• Debbie Woelfell, Sopra Group • Tina Kelly, EGUK Testing Innovator of the Year • useMango • Barry Weston, Sogeti • Brickendon Consulting
Test Champion of the Year • Datacastle • Craig Ian Thomas, Cognizant
The BCS Best Overall Project • Tech Mahindra • Chickenwings • Norton Rose Fullbright • Aditi Technologies • Tricentis Best Overall Use of Technology • Gamesys • Jaspersoft
Best Use of Tools • Brickendon Consulting • Tech Mahindra • Tata Consultancy Services • Capita IT Professional Services
• Eleks • Amdocs • Brandt Technologies
Most Innovative Project • The Test People
The Thinksoft Testing Team of the Year
• ValidSoft UK
• Proxama
• Amdocs
• Lloyds Banking Group in partnership with Cognizant
• Gamesys
• Waitrose • MGM Advantage • Home Office IT Test Design and Consultancy Services
Testing Management Team of the Year • A1QA • Maveric Systems • Close Premium Finance
PAGE 16
We are looking forward to announcing The Borland European Software Testing Award winner and honouring an individual for their Lifetime Achievement on the night, as well as Best Newcomer and the Best Crowd Project.
OCTOBER 2013 | www.testmagazine.co.uk
THOUGHT LEADERSHIP
AVOID OUTSOURCING PITFALLS TO DELIVER SOFTWARE APPLICATION INNOVATION Chris Livesey, Vice President of Worldwide Sales and Operations, Borland, Micro Focus, discusses the importance of business requirements when outsourcing software application projects and the need for requirement management tools‌
PAGE 18
OCTOBER 2013 | www.testmagazine.co.uk
THOUGHT LEADERSHIP CHRIS LIVESEY VICE PRESIDENT OF WORLDWIDE SALES AND OPERATIONS BORLAND, MICROFOCUS
W
UNFORTUNATELY, THE BENEFITS OF OUTSOURCING SOFTWARE DEVELOPMENT PROJECTS CANNOT BE REALISED WITHOUT SOME INTERFERENCE FROM THE CHALLENGES AND SO A BALANCE MUST BE manual testing to drive up billable STRUCK hours. However, both of these views
hilst purse strings still remain tight, businesses are under increasing pressure to deliver innovative software applications that enable them to gain competitive advantage and meet customer demand in a cost efficient way. However, most software and systems development projects run into problems such as schedule delays, unsatisfied users, or additional costs. This can sometimes be attributable to skills, experience and processes, and consequently, outsourcing software development, testing and maintenance has become an increasingly popular option. However in the majority of cases, the issues are due to a lack of precision and control on the product requirements. Regardless of who actually carries out the work, without sufficient support to properly elicit, validate and manage software requirements, rework, delays and spiralling costs are still common.
THE PRO AND CONS OF OUTSOURCING According to independent research commissioned by Micro Focus in August 2013, almost half of the 590 global CIOs and IT managers polled confirmed they outsourced development (49%) and testing (47%) projects to external providers. This is an increase from two years ago when an average of 43% of development and 42% of testing was outsourced. It’s clear to see why outsourcing is attractive. It can help reduce internal costs as businesses no longer need to hire, train or nurture a variety of in-house skilled staff. It can also reduce risk as the outsourced partner is held accountable by contract, as well as improve the overall speed of application delivery to customers as outsourced staff already have the specialist skills, tools and process experience in place. These benefits free an organisation to focus on its core competencies. In equal measure, there are challenges to outsourcing that have proved a deterrent for businesses. Many industry commentators believe outsourcing software application projects can hamper an organisation’s ability to achieve business agility because there isn’t the same incentive to drive efficiencies into the development process as there would be in-house. In addition, critics contend that outsourcing vendors deliberately use
OCTOBER 2013 | www.testmagazine.co.uk
are considered out-dated. Outsourcing can actually provide businesses with far greater agility because it removes the need for additional resources, training and specialist knowledge. Manual testing is often favoured when there is no other option or the nature of the system under test doesn’t lend itself to automation. Other known challenges to software development and test outsourcing include time zone and geography issues, communication and clarity issues, managing change control and differences in culture, behaviour, languages and expectations.
FINDING A BALANCE Unfortunately, the benefits of outsourcing software development projects cannot be realised without some interference from the challenges and so a balance must be struck. It is therefore vital that businesses have visibility into the possible challenges they may face ahead of a project in order to minimise risk. This can be achieved by increasing the precision and control of the processes around the specifications of the project, such as defining the business requirements and in turn, the specification of the test cases, which need to be executed to attain sufficient confidence that the business requirements have been met. The lack of rigorous requirements that outline what the organisation wishes to achieve, in a clear and transparent way for the external development teams, is often the common denominator between most failed outsourcing projects. According to our research, more than four in five (81%) organisations are not totally confident in their ability to clearly document and communicate project requirements to outsourcing vendors. This ability can be further weakened when changes in requirements set off a chain reaction of delays, revisions and rework. For example, a change in a requirement, can lead to a change request, which can lead to additional costs and changes to the scope of work. So how do businesses ensure that outsourced projects deliver the right software on time and on budget, and limit any hidden costs that may occur when requirements change? With any development or testing outsourcing
PAGE 19
THOUGHT LEADERSHIP project, a business requirements management tool plays a vital role in ensuring a project is delivered back to the business exactly as it was planned.
USING TECHNOLOGY TO MANAGE BUSINESS REQUIREMENTS The current processes for establishing business requirements may be considered ad-hoc and inefficient, leading to miscommunication and insufficiently defined requirements. According to our research, the vast majority of businesses are currently opting for low technology solutions such as written documents (73%) and spread sheets (76%), which can limit the amount of information that can be inputted and be open to multiple interpretations by an outsourcer. In order to be able to deliver strong and clear requirements, the processes used in a typical outsourced software application model require regular checkpoints with clear benchmarks, expectations and statements about each requirement. This could include the activities in the plan to deliver those requirements and how any changes that will inevitably take place during the project could affect the requirements and overall plan. Such changes are not linear by nature and have an exponentially detrimental effect on the cost and schedule of the project, the later they are discovered and addressed. Therefore, it is essential for businesses to firstly implement a strategy for precision and control which will enable changes to happen, but in a manageable way. Not enabling change is an impractical alternative. Aside from these considerations about how any project is managed, organisations need to be aware of and consider the control points that exist on the boundaries of each of the major phases, which are pivotal to the smooth flow of the project. These control points, where specifications or final results are delivered, require active collaboration between stakeholders, such that either party can be proactive, clear and explicit about the progress and condition of the work. Many businesses should ensure they have taken the four-step process to define their requirements with their key stakeholders. These could include:
• Elicitation As the first step, all parties involved in defining the requirements in the first instance must outline their needs in a visual way, such as a work flow diagram to determine which business flows the application will deliver; who the users will be and how they will utilise the eventual software.
• Analysis Once the business flows are collected, a reality-check needs to be carried out by the development teams to ensure that they understand what is needed and can deliver it based on the resources at their disposal. This helps verify the feasibility of the plan and catches any serious problems or inconsistencies early on. Additionally, by conducting the appropriate levels of analysis and prioritisation, the in-house development team are able to save time and money. Analysis enables smoother adjustments later in the cycle and prioritisation enables the ranking of various functions and features of the application in order of importance. This way, high-priority tasks are built first, with mid-tier and lower-priority features added as time and funds allow. If there is a change and the
PAGE 20
MANY BUSINESSES SHOULD ENSURE THEY HAVE TAKEN THE FOUR-STEP PROCESS TO DEFINE THEIR REQUIREMENTS WITH THEIR KEY STAKEHOLDERS
full list of requirements can’t be met immediately, the application can still carry out its core functions and serve the prioritised end-user’s needs.
• Specification As the requirements begin to take shape, stakeholders are able to add detail through expanded use cases, business rules, business models and prototypes. This step involves documenting the requirements, establishing the management protocol to be followed and determining the way this project integrates with existing applications and processes.
• Validation Once all the requirements are specified, each stakeholder should validate that their initial vision is reflected in the business flows, and that the details are accurate and complete. At this stage, business analysts interactively review the scenarios to ensure they deliver the desired results, and the business will ensure that the flows will be most effective in meeting their needs and integrating with their work environments. Test cases and release criteria are established based on these requirements.
OCTOBER 2013 | www.testmagazine.co.uk
THOUGHT LEADERSHIP
By taking these four steps, businesses can create an established and defined approach to how a project will be managed and controlled. The in-house team should then create a comprehensive document that outlines all of the development requirements in full detail. To further improve management and control of business requirements, businesses should also look to use a software requirements management tool to help visualise, define and manage these requirements. By doing so, the needs of the business will be met and the project outcome will be delivered quickly and efficiently.
THE BENEFITS OF SOFTWARE REQUIREMENTS MANAGEMENT TOOLS A software requirements management tool, such as Micro Focus Borland’s Caliber, is designed to help businesses understand the scope of work and support the design, management and measurement of any outsourced software project to enable better and more precise communication with outsourced partners. For example, a software requirements management tool enables outsourced development teams and
OCTOBER 2013 | www.testmagazine.co.uk
stakeholders that may be involved in the project to effectively collaborate by providing a single system of record to manage requirements throughout the project lifecycle. It also allows a company to share requirements with its outsourced quality teams and developers and if the requirements evolve during the process, the tool enables the business to track any changes and ensure all affected parties are kept in the loop through automated notifications. With a single central view – both outsourced development teams and in-house staff will have stronger visibility and control of the project, which will eliminate unnecessary levels of risk and variance to the software application project. There are multiple benefits to outsourcing software application development and delivery, and ensuring that the business requirements of the project are managed closely can help to eliminate the risks involved in such projects. By using a business requirements management tool, businesses and development teams can help to better communicate to produce a software application that meets the needs of the business in a streamlined, quick and efficient way.
PAGE 21
COVER STORY
“FACE VALUE” TAKES ON A NEW MEANING Global e-commerce business, PayPal has released a new mobile app, which allows customers to use their face and smartphone to purchase goods. Sophie-Marie Odum investigates the testing of this revolutionary software that could see high streets become “wallet-less” by 2016…
P
ayPal is leading the UK in the quest for a “walletless” high street as it releases a new payment system that allows consumers to use their face and smartphone to make payments.
In a first for the UK, consumers can now make payments on their mobile phone and be recognised by their first name and profile picture. Currently, a dozen businesses in a high street in Richmond, London, including cafes, restaurants, shops and a hotel are testing this new app.
PAY BY FACE Once users have downloaded the PayPal app for iOS, Windows OS and Android phones, it highlights nearby
PAGE 22
shops and restaurants that accept PayPal; the customer then checks in to the shop, and their name and photo appears on the shop’s payment system. After the customer agrees the amount to be paid, the cashier charges them by clicking on this image. The customer receives an alert on their phone, letting them know how much they’ve paid, as well as PayPal’s usual email receipt. Explaining why PayPal decided to introduce this new app, Rob Skinner, PayPal’s director of PR, UK & Ireland, said, “It’s part of our mobile app. Paypal was one of the first major financial services WE business to introduce BELIEVE THAT an app if we go back IN 2016, YOU WON’T to 2008 when Apple
NEED TO TAKE A WALLET/ PURSE SHOPPING WITH YOU – YOU’LL BE CONFIDENT THAT YOU’LL FIND ENOUGH PLACES TO SHOP OR HAVE A MEAL THAT WILL TAKE MOBILE PHONE PAYMENTS AND THAT’S A KEY DEVELOPMENT FOR US
OCTOBER 2013 | www.testmagazine.co.uk
COVER STORY
launched their first App Store, so this is something that our consumers are very used to and we’ve been adding features to it ever since. “The latest version which allows consumers to check-in to a restaurant, café or a retailer, for example, is designed to recognise the smartphone as it plays a major role in our everyday lives. We take the view that the mobile phone is where the action is. “People don’t want to carry a wallet or purse around when they're just popping out for a coffee, for example. And a smartphone is something that many people carry around with them all the time, so we have recognised this and have made it easier for consumers everywhere. “However, the key point for PayPal is that we don’t mind how the consumer connects with us because, whether it’s on a mobile phone or a desktop, or even a games console, they are using their PayPal account. Naturally, when you are talking about a face-to-face environment, like the high street, inevitably smartphones and tablets are the obvious ways for people to make those payments. So we are very much focussing on mobile at the moment.”
TESTING THE NEW APP Discussing the testing of the new app to ensure it was ready for public release, Rob said, “We are a technology company with huge expertise in payment so making sure everything works smoothly is hugely important to us. “Our philosophy is that technology for the sake of technology is pointless. Technology only has a purpose if it solves real-life problems or makes life easier. Creating that lovely, smooth customer experience is very important to us and that is something that we have tested significantly both in the US and the UK.” Rob was unable to comment on the testing time scale, but explained that the development and testing teams go into great detail to ensure that the app works correctly and that it's highly secure, which is very important. “The Richmond project shows testing of the app in real environments. We looked at how it is for businesses and consumers. Businesses were very pleased at how easy it was to use, as it offers their customers a new way to pay but without ripping out their existing systems. “Cash will still have a long life, and people will still choose to pay with a card on the high street, but the key point is that it's about choice. We believe that in 2016, you won’t need to take a wallet/purse shopping with you – you’ll be confident that you’ll find enough places to shop or have a meal that will take mobile phone payments and that’s a key development for us.”
OCTOBER 2013 | www.testmagazine.co.uk
The need for software development cycles to be completed quicker to ensure products are delivered to market in a timely fashion to meet customer expectations was no different for PayPal in the development of this app, as Rob explained: “Consumer expectations are increasing, and rightly so. They always want to have great experiences. So we need to make sure that they do – we’ll continue to evolve the services we offer because things are moving so quickly. That’s the great advantage that PayPal has because, being a technology company with roots in Silicon Valley, California, and some very smart people around the world, including in the UK, we can make sure that what we offer meets the local needs of people in UK and US. “Security is our number one priority. It’s fundamentally important. We believe that the mobile phone is an incredibly good way to offer services, but we need to make sure, and we do make sure, that PayPal is secure on a mobile phone as it is on any other type of device.” “We’ve built our name on security and it is always the first thing we check first whenever we introduce a new service. Our check-in facility is absolutely in line with that philosophy as no data or information is stored on the phone – users have to log in to the app as normal.”
FACIAL RECOGNITION TECHNOLOGY On the release of this updated app, there was confusion that a facial recognition feature was included. Rob explained, “When you say ‘pay by face’, the technology can easily be confused with facial recognition software, but this isn’t facial recognition in that a phone recognises your face. This is a picture popping up on the store’s iPad, or till system, so the clerk can see it’s the right person and confirm a payment in this way.” But is facial recognition the future for PayPal? “I couldn’t speculate about what the future holds,” answered Rob. “What I would say is that PayPal is very flexible and we are offering customers lots of different ways to pay. But also we have been pioneering this move from the high street to the online world for several years. “Even recently, eBay UK and Argos have partnered so that consumers can buy goods from selected merchants on eBay UK and collect from an Argos store. We are similarly playing a big part in helping break down the barriers between the online world and the high street, but also creating new opportunities for the high street. “We think mobile technology in particular really does give a real boost to the high street and we are very pleased to be playing a part in that.”
PAGE 23
TestExpo PREVIEW
AN INTERVIEW WITH STEFAN GERSTNER Stefan Gerstner is Vice President of Global Service Line Testing at Sogeti, hosts of TestExpo 2013. As well as chairing this year’s TestExpo, Stefan will also be sharing insights and recommendations from the latest annual World Quality Report during the event. The Report examines the state of application quality and testing practices across 25 countries the globe, comparing perceptions and trends in application quality, methodologies, tools and processes over the past 12 months with visions for the future… 1. Let’s begin by hearing a little bit about you, Stefan. I am an international testing evangelist for Capgemini and Sogeti. International partly because I am a German guy living in the Netherlands, and also because for the last 10 years I have been responsible for coordinating our group’s global testing services; initially for all Sogeti entities, and then for the Capgemini Group as a whole for the last three years. I am time and time again thrilled by the challenges which testing services are facing, be it from a technology, organisational or economic perspective. It is rewarding to have an active role in helping to understand client requirements and highlighting important market trends. TestExpo is a great platform for doing just that. 2. Tell us a little bit about TestExpo? For those that haven’t attended before, TestExpo was launched initially back in 1996 and has since become one of the biggest and most renowned software testing events in the UK! It is a one-day conference that allows those in our industry to share information and guidance on the most relevant hot topics in testing and quality assurance that impact today’s businesses. It has been great to watch the event transform over the years into a fantastic forum for debate, discussion and networking for the industry. We’re also proud to say that TestExpo became an international event in 2013 following successful launches in both Denmark and Norway earlier this year.
Our 2013-2014 World Quality Report, released on 12th September, highlights a growing strategic importance of testing and QA, despite many organisations still struggling to demonstrate the true value of their testing function to the business. We have structured this year’s event around a series of themes based on findings from this latest World Quality Report, which, in our experience, are current key areas of pain or interest – from agile and mobile to environments and budgets – all delivered via keynote presentations and round-table sessions where all participants can join in and discuss their own unique testing issues, as well as broader industry level concerns. We want to help our visitors find feasible solutions that suit their own, specific requirements, and will work to help them in their quests to improve quality and speed up testing activities, as well as providing them with realistic ways to reduce their costs in both the short- and long-term. 4. TestExpo 2013 is packed full of presentations, knowledge sharing and networking sessions, what do you hope attendees will take away from this year’s event? By its nature, TextExpo aims to give delegates the opportunity to take a fresh look at their testing problems, discuss and debate them, and gain invaluable insight. Ultimately, this year’s event will provide visitors with the solutions, tools and knowledge to effectively transform their testing function and increase test maturity.
This year, we have specialist conference organisers, UNICOM, on-board to help create what we expect to be the best TestExpo event to date – in my opinion, definitely one not to be missed by anyone in testing!
To put it into a nutshell: attendees should be inspired with ideas that matter!
3. The theme for TestExpo 2013 is ‘Transforming Testing – Identifying Solutions that Work’. Why was this theme chosen and how is it reflected in the conference program?
Obviously all of the sessions will be valuable. We have some great topics, including The Mobile Application Lifecycle, Application of Model-Based Test Design and Automation to Transform Testing Effectiveness, Why can’t we get the Reports that we want?, and of course my own session on this year’s World Quality Report findings!
We’re very aware as a group that improved quality, reduced time to market and lowering the cost of testing sit firmly at the top of all test teams’ wish lists! However, we also see that the increasing numbers of tools, methods and solutions available in the market today provide a confusing view of how these aims should and can be achieved. Of course, not all solutions are right for every organisation.
PAGE 24
5. Are there any sessions that you are particularly looking forward to?
However, I think the two round-table sessions will provide everyone – delegates and sponsors alike – with a fantastic insight into where the testing industry is today, and it will be interesting to see how these discussions compare with the Report. I’m interested to hear the main pain points across the board and help our delegates with suggestions
OCTOBER 2013 | www.testmagazine.co.uk
TestExpo PREVIEW
for best-practice or ideas to help them to improve and mature their test functions. 6. What, in your opinion, have been the key impact areas for testing over the last 12 months? From a technological perspective, a key topic that has been high on the agenda and where major steps forward were achieved over the last year is mobile application testing. Findings from this year’s Report show an increasing number of companies with fully operational Testing Centers of Excellence. Not just on a UK level – where we saw an increase from only 7% last year to a staggering 27% of organisations now reporting that they have a working TCoE in place – but also on a global scale. Rewind 12 months to the launch of last year’s World Quality Report (2012-13) where we highlighted the fact that only a minority of companies were applying structural testing in this area. This has changed considerably, with a jump from 19% in 2012 to now more than half (56%) of UK companies carrying out mobile device testing. We are discovering that companies are now better equipped to tackle this issue, which is crucial given the potential high business impact of quality issues with mobile applications.
anticipation surrounding this year’s event and reinforces its value for all involved.
More and more companies are convinced that this approach allows them to “square the circle” – achieving reduced cost, increased quality and improved time to market at the same time through centralisation.
We realise that, by involving sponsors we are able to offer a much greater experience, insight and thoughtleadership to our delegates, and therefore can provide a richer mix of topics with a more holistic view of practical solutions for every need or budget.
7. What new challenges, key trends and technologies do you expect the testing industry will come across during the next 12 months ?
2013 sponsors include IBM, Hewlett-Packard, Microsoft, Tricentis, Seapine Software, TestPlant, Zephyr, Soflab Technology, Neotys and Capacitas, with Test Magazine and Professional Tester and as key media supporters.
More than enough! Let me name just a few. We are still in the early stages of integrating cloud-based solutions in to our daily practices. Cloud-based services still offer a great potential for cost savings and increased organisational flexibility. The “shift left” paradigm also still offers great opportunities beyond where we have already seen benefits. Think, for example of model-based services and service virtualisation.
9. Why should test professionals attend events such as TestExpo? TestExpo is a great platform to exchange experience with peers, and get inspired with ideas that really matter and will have an impact on the way you test tomorrow. I am looking forward to seeing everyone there!
A demanding subject is also the changing role of the tester. Not only from a “shift left” perspective and in the increased need for early, proactive involvement, but now also in areas like agile, DevOps and model driven development. These areas already have great impact on the skill sets required from a testing professional and his role in software development as we see the market requiring more “technical testers”. 8. Which sponsors are involved and how do you feel this affects the dynamic of TestExpo 2013? We have a good number and range of sponsors supporting TestExpo 2013, which really highlights the
To download your copy of the 2013-14 World Quality Report, please visit: www. uk.sogeti.com/2013-14WQR
TestExpo 2013 will take place at the Millennium Gloucester Hotel London Kensington on Thursday 24th October and will start at 8.30am. To guarantee your place, please visit www.testexpo.co.uk OCTOBER 2013 | www.testmagazine.co.uk
PAGE 25
MOBILE APP TESTING
The diary of a mobile load tester
Henrik Rexed, senior performance engineer at Neotys, shares a diary that describes how to test a networked mobile application. He looks at a specific mobile application to give readers a detailed insight into realworld load and performance testing, the methodology, what is actually tested, how it is tested, and the pitfalls inherent in this type of project‌
A
s a load and performance tester, how do you meet the performance expectations of mobile application users? How do you meet expectations for rapid response times? The answer is through thorough, yet rapid mobile application performance testing. Yet, it is not as simple as just applying performance tests from the desktop world to your mobile applications. Mobile applications are diverse and have their own set of performance challenges making them different from web applications – therefore they also have their own set of requirements. The mobile application I tested is an enterprise content management platform designed for intranets. For the context of our test, this application typically has 200 concurrent users, and connection spikes around 500 concurrent users (for example when a new document is released). The objective of the test is to ramp up the user load to 600 virtual users and to analyse the behaviour of the application in a lab environment, before going to production. Often, mobile application development projects tend to have shorter development cycles compared to traditional desktop browser-based applications because these apps are often light versions of the full applications with limited features. In our case, the application I NEEDED TO was developed in TEST ONE OS, three weeks and we FIVE NETWORK were given no more CONDITIONS AND than three days to 10 DIFFERENT test the performance of the application GEOGRAPHIES IN under load. This THREE DAYS! situation is very challenging since testing
PAGE 26
mobile applications are more complex than testing desktop browser-based applications. This is because you need to integrate two additional dimensions, i.e different network conditions and different devices used.
DAY 1: PREPARING FOR THE TEST- STUDY PHASE Devices used to connect to the application The first task of the study phase is to identify what devices are used to connect to the application, and if they generate different transactions on the application server. If all devices use the same framework and generate the same transactions, then you are lucky and you should only test on one single OS instead of testing on multiple ones. In our test case, we have the same transactions for all devices. But I have worked on projects where the mobile application development was outsourced to third parties that specialised in iOS and Android. The iPhone app was developed by one team, and the Android app was developed by another team. In this case, we needed to have two separate testing tracks. The information on the devices used to connect to the application is provided by the development team. Therefore, it is important that testers integrate the time required to get feedback from the development team in their project time frame. Network conditions and geographies The second dimension that makes a big difference when testing the performance of mobile browser-based applications is the network conditions. Network characteristics, including bandwidth, latency and packet loss, have a huge impact on client response times and on the way the server is loaded. So with my test case, I needed
OCTOBER 2013 | www.testmagazine.co.uk
MOBILE APP TESTING
to simulate different network conditions to forecast the effects of changes in the network infrastructure on the application’s performance. In order to get realistic metrics of the different network conditions. I used the SpeedTest application. That enabled me to identify the main network characteristics like latency, and performed that task for each different network that I had to simulate. Location is also an important dimension that impacts the application performance. In the case of my application, users would connect from all parts of the world so it was required to simulate users from different geographies. Information of the networks used to connect to the application, and the main geographies, can also be provided by development teams. In my test case, I had to simulate five different network conditions (Hedge good network, Hedge poor, 3G good, 3G poor, Wi-Fi) and 10 different geographies.
DAY 2: TEST EXECUTION Once you have integrated the network constraints and taken into account device diversity, running a load test on a mobile application is very similar to running a load test on a desktop browser-based application. The main difference between the two is timing. In my example, I needed to test one OS, five network conditions and 10 different geographies in three days! This represents five times more cases than testing a web application, where I would not have integrated the network conditions. Therefore, it is crucial to rely on a testing tool that can rapidly design test scenarios and integrate the different parameters such as latency, packet loss and simulate connections from different parts of the world. In my test case, the application connects directly to the server, regardless of the proxy settings of the operating system. Therefore, I needed to perform the recording based on network capture and tunneling because proxy-based recording was not possible. Another parameter I had to take into account (which will certainly become more the case in mobile applications) is that the application was based on HTTPS. All HTTPS recording methods (even when tunnel based), are seen as man-inthe middle attacks by the device, which leads to an outright connection refusal in a native application. This makes it impossible to record the secured traffic. In order to actually perform the recording, I needed to provide a root certificate installed on the device that authorises the connection with the tunnel.
DAY 3: ANALYSIS OF THE RESULTS Like the actual test itself, analysing the results of a mobile application test is very similar to that of a desktop browser-based application.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
Again, my tests were focused on the application backend itself (on the server side). My tests did not include the analysis of the device hardware characteristics such as CPU, memory, battery capacity or screen size or the impact of contextual behavior like localisation/GPS, gyroscope and interruption from SMS, voice calls. While these characteristics are important in assessing the end-user experience, they were not taken into account when running my load and performance test.
CONCLUSION The most important thing to remember is that the main difference between testing desktop and mobile applications can be found in the study phase. So focus on this phase and gather the information you need to integrate the network constraints and device diversity. Be prepared to report on what networks are used and what devices are used by the application. The key to success in my test case was to rely on a tool that would provide the required features to handle all my mobile testing specifics and, more importantly, manage my three-day time constraint. I also needed to be sure that my contact would be available when required, and be able to execute all test scenarios within the short amount of time available. Today, users are accessing applications from an assortment of mobile and PC devices, across a variety of networks from different geographies. As a load tester, how can you validate the performance of your mobile applications and confirm they will meet the expectations of mobile users? Mobile users are connecting through a diverse range of devices that typically have little bandwidth and computing power, yet they still want fast response times. Performance testing for mobile applications is a necessity in today’s world. I hope this diary provides you with a better understanding of how you can actually test the performance of a mobile application under load. Now, it’s your turn!
While the case study described in this article is specific to one application, it provides a good example on what you might expect in a typical mobile testing project. It relates to load and performance testing of a networked application as seen from the server side. This means my tests did not analyse the performance of the application on the device. While the application performance on the device, related to CPU, memory and other device characteristics are also key to get the full picture of the end user experience, I did not address this aspect in this article.
PAGE 27
MOBILE APP TESTING ANTONY EDWARDS CTO TESTPLANT
MOBILE APP TESTING: THINK YOU’VE GOT IT COVERED? TestPlant discusses the rise in mobile app testing; the need for automated testing to meet such demands; and the emergence of cloud-based mobile app testing…
M
obile applications are now well beyond the initial boom and have become the established mainstream. Mobile devices now outsell traditional computers by two to one and most people spend more time with their mobile than any other possession. This year 70 billion apps will be downloaded worldwide and people in the Western world will, on average, download 40 apps each in 2013! With this hyper-growth comes the challenge of releasing apps that are stable on multiple mobile devices and platforms. Quality assurance of mobile apps can be a complex, time-consuming and costly task, especially if companies are reliant on manual testing or traditional testing tools. Significant investment is required to develop an app and then promote it to new end-users.
conversation often goes something like this: Manager: “Okay guys, we’ve got to test this app… what should we do first?” Tester: “How about iOS?” Manager: “Okay, great. Let’s do it.” Sometime later… Tester: “Okay, iOS done. That was fun!” Manager: “Right, what’s next?” Tester: “Android?” Manager: “Go for it.” Some more time later…
As a rule, 85% of app downloads aren’t used after the third day – if your user experience is bad then what are the chances you’re going to be in the 15% that manage to engage consumers for longer? This has implications for both developers and the senior managers who are responsible for commercial growth and customer satisfaction.
PRODUCT EXPERTS ARE KEY Antony Edwards, CTO at TestPlant, said, “Mobile apps have really given test departments something to think about because testing mobile apps is much better suited to product experts than technical experts. “There is a real difference between testing technical parameters and testing user scenarios. The visual-based nature of some products means that they actually lead the tester to test real user scenarios, and we have seen this shift occur in environments where eggPlant has been implemented in a previously ‘technical’ test environment – they soon start thinking about things from a user standpoint.” The growth in mobile apps has caught everyone on the hop. Companies are now catching up, but are often jumping into things quickly, without much in the way of strategy. When a company first develops a mobile app, the
PAGE 28
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
MOBILE APP TESTING
Tester: “Android done… I’m bored now.” Manager: “Right – same again for Windows, Blackberry, iPad, other tablets, other devices etc etc.” Tester: “*$%*@*%!”
AUTOMATION LEADS THE WAY The growing number of devices and operating systems means that effective manual testing of mobile apps is becoming increasingly more difficult to perform. The use of test automation for mobile apps is fast becoming a necessity. “Let’s face it, test departments’ workloads are increasing all the time and, while other areas of software development can miss a deadline, testers often provide the contingency for the rest of the development lifecycle, as their release date is always set in stone,” says Edwards. “The pain of testing the same app across multiple devices and operating systems only adds to this workload.” But as companies consider making the shift from a manual testing environment to complex automation tools, the resulting skills gap often requires that companies hire new automation engineers to replace manual testers.
only way to guarantee the security of your data is to have a private cloud.”
THE GROWING NUMBER OF DEVICES AND OPERATING SYSTEMS MEANS THAT EFFECTIVE MANUAL TESTING OF MOBILE APPS IS BECOMING INCREASINGLY MORE DIFFICULT TO PERFORM
To satisfy the quality requirements of mobile apps, a testing strategy has to take into account different devices, multiple operating systems, screen sizes and resolutions, as well as the responsiveness of an application. Some of this can be checked on an emulator but for a realistic test it needs to be performed natively on physical devices and behind the firewall.
“It’s ideal to be able to test on actual devices that have not been jail-broken, which ensures that the app is being tested on exactly the same device that a real user will be using,” explains Antony. “In addition, having the device at hand allows you to test environmental factors such as switching from Wi-Fi to the network, which cannot be checked on a cloud-based solution.”
“Testers are often the people with the most product experience, and it seems crazy to think that a company would choose to lose that expertise just because they have adopted a new way of testing, but unfortunately some automated tools are actually that complex,” says Antony.
Furthermore, the cost of cloud-based services cannot be underestimated. For one-off testing, they may be fine, but for an on-going test requirement such as regression testing, they quickly become cost-prohibitive.
Antony is quick to point out the connection between the need for real user testing and the failure of outsourced testing.
“While eggPlant is primarily an on premise-solution, we do see the value in some aspects of offering a hosted solution for testing mobile devices, and we will soon be providing our customers with the option to set up their own private device clouds, within their own firewalls, of course.”
“It made no sense to have people testing your desktop software who had no idea what the final purpose of the product was,” he adds. “With mobile apps, the focus on how the app actually works from a user perspective is even more pronounced, so the need to be able to test like a user is absolutely vital. To be honest, developers can test code… testers should be testing products.”
UP IN THE CLOUDS? The recent emergence of so-called “cloud-based mobile app testing services” appears to offer a great solution for companies looking to leverage test automation across mobile apps, allowing access to a huge number of devices, networks and operating systems, including very niche platforms. “Being able to test on every single device sounds great, but is there any test team that really has the time to test an app on every single handset variant, when they can easily address 99% of the market by focusing on the main devices?” queries Antony. And cloud-based solutions come with their own set of challenges. “Imagine a situation where you find yourself in need of a short-notice test on a specific device and then you find out it’s currently in use and not available until later – that could be critical,” says Antony. “In addition, there is the security question – can you really be sure that your app data is secure? We’ve even seen one of the leading cloud-based solutions stating on their website that the
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
Although, Antony does feel that the cloud has its part to play in functional app testing.
SOPHISTICATION INCREASING While mobile app testing is still in its relative infancy, increasingly more information is being made available to testers to allow them to do their jobs better. Today’s testers are often now responsible for suggesting solutions rather than just finding problems, and technology developments are helping them achieve that. “We know that developers like it when testers can provide real information about what worked, what didn’t, how they tried to resolve the issue and so on,” says Antony. “Providing testers with all kinds of information about what is happening during a test, for example CPU spiking or increased battery usage, can provide very valuable contextual information when it comes to analysing a problem. Essentially, we are looking to bring server-level diagnostics to the device level.” The concepts of behaviour-driven and test-driven development are also becoming more widely adopted by testers, especially mobile testers. “BDD and TDD are great techniques for ensuring that testing and development are focussed on what will really impact users,” says Antony. “They are the practical tools that have made requirements-based testing and riskbased testing a reality.”
PAGE 29
TESTING IN PRACTICE PAUL GARDINER HEAD OF TEST PRACTICE EXCEPTION
THE CHALLENGES OF SELF-SERVICE TECHNOLOGY Paul Gardiner, Head of Test Practice, Exception, addresses the challenges testing selfservice technology presents… any of you will now be accustomed to the frustration of being told by an unattended check-out that an item should be “removed from the bagging area”. Travellers will be used to repeatedly scanning their passports before being informed to visit a manned check-in desk. Self-service technologies play an integral part of our day-to-day lives.
M
One of the oldest and most recognisable self-service devices is the automated teller machine (ATM) or cash machine, which was first deployed in 1967 and has much in common with the new breed of self-service terminals. However, the ATM and new self-service technologies differ in the perception of quality of service. The ATM is deemed to be highly reliable and rarely fails to complete a transaction. The ATM’s relative maturity is one reason for its quality, as well as: • The testability of the ATM • The availability of bespoke test tooling • The level of quality assurance that financial services
organisations apply compared to their peers in retail and travel sectors.
MANUAL TESTING AND AUTOMATION Functional testing of ATMs is a repetitive and costly process. To provide sufficient coverage of test scenarios, a test lab must acquire and maintain a range of expensive terminals and associated supplies of cards and consumables. Each manual tester requires access to suitable, physical ATMs for an extended period of time. In addition, traditional automation tools do not lend themselves to ATM testing due to the reliance on information generated from specific self-service technologies, such as personal identification number (PIN) entry devices and chip cards, which cannot easily be interrogated by off-the-shelf tools. The industry standard ATM middleware layer (European Committee for Standardisation Extensions for Financial Services, or CEN/XFS) dictates how ATM software talks to all models of ATM hardware, regardless of vendor. This, combined with the fact that an ATM is essentially a Windowsbased PC connected to a number of devices, has allowed test tool vendors to leverage virtualisation technology to create labs of virtual ATMs, supported by scriptable test automation engines. A combination of tools and test best practice facilitates significant test coverage at a relatively low cost. This is one key aspect which supports quality in the ATM compared with other self-service technologies.
PAGE 30
DEALING WITH ENCRYPTION One of the biggest challenges presented to the ATM tester is encryption. ATMs use highly secure cryptographic techniques to generate encrypted data to secure transaction elements, such as ATM identifiers, PINs and chip card data. Failure to successfully mimic encryption results in operational failure of automated testing. For functional testing – where encryption is not needed – encryption may be disabled for simplicity. Whereas, for non-functional testing, particularly performance testing, the encryption steps are a key aspect of the system which must be exercised to identify potential performance bottlenecks. Traditional load generation tools do not support the encryption steps required to perform ATM interactions with host systems. Testers leverage bespoke tools to allow full, endto-end, simulation of the ATM start-up and transaction steps. This is another reason for the increased quality at the ATM.
CARDS AND CHIPS In the early days of ATM testing, test data requirements for cardholders were very basic. A magnetic stripe card contained no more than a few hundred unencrypted ASCII characters, which could be easily stored in a text file and used repeatedly across test cases. Chip & PIN, the associated Europay, MasterCard and Visa (EMV) chip card standard, made the ATM tester’s challenge significantly more complex. The chip on an EMV card is a computer in itself and can embed functionality to verify the integrity of the card and maintain transaction counters. This requires ATM testers to either create and maintain a large number of chip cards during test lifecycles or leverage card simulation tools, which allow probes to be inserted in ATMs simulating card behaviour without the need to maintain multiple cards. These tools are equally viable to all self-service devices and are not ATM specific.
SELF-SERVICE TESTING ELSEWHERE Unfortunately, retail- and airline-based self-service technologies do not have similar middleware standards, meaning that reusable, automated test tools cannot easily be created. Banks and other financial services organisations place significant quality assurance budgets on ATM testing to ensure that 24/7 availability is maintained and industry regulations are met. However, self-service terminals in other environments do not have the same 24/7 uptime requirements, and regulatory compliance is almost nonexistent beyond EMV and PCI-DSS. It seems evident that significant advances in testability will be required if new self-service technologies are to be as successful as the ATM.
OCTOBER 2013 | www.testmagazine.co.uk
AGILE DEVELOPMENT
PHIL KNIGHT RALLY SOFTWARE
AVOID THE “PULL OF THE PAST” Adopting an agile approach in the development cycle helps organisations improve time-to-market, quality and customer satisfaction. So why haven't all companies jumped on the agile bandwagon? Sophie-Marie Odum speaks with Phil Knight from Rally Software to find out why… Sophie: When did you begin to see agile moving into development? Phil: In the 1970s, the waterfall method, gained favour as the preferred way to manage large software development projects. This approach, which can take months or years to complete, relies on rigid sequential execution of these phases. Organisations employing waterfall often structure internal departments around each development stage and use different legacy software tools for each phase and department, leading to siloed and disparate information; limited transparency and collaboration between teams; and heightened risk of misalignment between software development and business initiatives. Agile software development evolved in the mid-1990s as a reaction to these heavyweight waterfall methods. S: How did this trend come about? P. In February 2001, 17 software developers met at the Snowbird ski resort in Utah to discuss how to create software in a lighter, faster and more people-centric way. They published the Manifesto for Agile Software Development to outline methods of iterative and incremental development, where requirements and solutions evolve through collaboration between self-organising, cross-functional teams. S: Why do you think so many have adopted this trend? P: Agile represents a new methodology for software creation and delivery designed to reduce costs and significantly improve time-to-market, quality and customer satisfaction. Agile projects build software incrementally, in small batches, using short iterations of one to four weeks that help keep development aligned with changing business needs. Simply stated, agile works. According to the Standish Group, software applications developed using agile have three times the success rate of applications developed using the traditional waterfall method. Forrester Research says over 40% of companies have adopted agile, but less than 20% have scaled it across the enterprise. That’s important because agile is not just for development anymore. It’s changing the way businesses fund and develop projects and it’s aligning people across global organisations so everyone knows at any moment where projects stand and what teams need. Strategic planning, product and portfolio management leaders are infusing agile practices across their businesses to closely align development and business objectives. 2 S: Why then do you think not all software teams have adopted this process and what are the disadvantages of not adopting this process? P: As we all know, change is hard and moving to agile is no different. Many organisations suffer from what renowned Silicon Valley author and business advisor, Geoffrey Moore,
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
calls the “pull of the past.” As he discusses in his book, Escape Velocity, companies are often trapped by a legacy in which their growth came primarily from penetrating mature markets with lower-cost products. In today’s global economy, organisations are looking to agile to spur innovation and next-generation growth. Those who do not evolve and embrace business agility face the stark reality of faster, more nimble competitors innovating in fast, agile cycles, gaining market share, and putting oncepowerful market leaders at risk. S: How does agile benefit the software testing process? P: In agile, the software testing process is brought into play much earlier than in traditional development. Testers begin to validate requirements before they are built; they work with engineers to plan automated and manual testing before coding starts, and they work with engineers to maintain an automated regression test suite. As a result, manual and exploratory testing can focus on high-value corner and edge cases rather than constantly keeping up with happy-path testing. And the dramatic increase in automated test coverage means companies can release more often with more confidence and fewer defects. Frequent agile releases require very high discipline around quality. Companies that adopt agile find that they need to increase their maturity around test practices, and bring test teams and test managers upstream. Having testers embedded in the development teams, rather than forming separate test teams, reduces defect churn. Having developers walk through their proposed additions/ changes with testers – prior to check-in – benefits the team, as it reduces the cost of context-switching: it is far less expensive to have a developer correct what they are currently working on, than interrupt work they have moved on to. S: Is agile for you? P: With the dizzying rate of change in today’s global business environment, especially software and technology, leaders are adapting businesses in every industry to respond to change faster. Agile is for companies that want to change and improve the way new products are built and launched in today’s complex, ultra-competitive marketplace. A quick search of Merriam-Webster’s online dictionary uncovers two definitions of agile: 1) marked by ready ability to move with quick easy grace (an agile dancer); and 2) having a quick resourceful and adaptable character (an agile mind). Those are exactly the qualities that help companies spur real growth, expansion and innovation.
PAGE 31
AGILE DEVELOPMENT DR TIEREN ZHOU FOUNDER, CEO AND CHIEF SOFTWARE ARCHITECT TECHEXCEL
PRESENTING THE CASE FOR SPECIFICATION-DRIVEN DEVELOPMENT Founder, CEO and chief software architect of TechExcel, Dr Tieren Zhou provides a detailed look at specification-driven development (SpecDD), a robust and scalable agile-derivative… INSTEAD OF IMPLEMENTATION of the application) set at the OCCURRING ISOLATED beginning of the effort were the same set of features to be FROM CUSTOMERS AND delivered at the end. Hundreds DESIGNERS, SPECIFICATIONof hours were spent in the DRIVEN DEVELOPMENT design and development (SPECDD) PROVIDES THE cycles for these specs. At NEEDED PROCESS, WITHOUT the end of the road, though, customers too often found ADDING OVERHEAD TO that they hadn’t completely THE PROJECT TIMELINE represented their needs with the specifications.
Agile development methodologies came about to promote evolutionary changes within software engineering. There are some basic agile principles which are followed:
A
gile development is sparking the interest of everyone: from small development teams to multinational corporations. It is one of the hottest trends in software development. The ideas of iterative development of applications, evolving requirements, better channels of communication, and projects finishing on time prove to be instantly popular with the development community. This popularity was born out of the failures of previous large projects: it is very easy to read the headlines of millions of dollars in overruns. A fundamental flaw in these projects was the belief that the specifications (the “to-be delivered” functionality
PAGE 32
• Customers list and prioritise their requirements. • Development teams deliver increments of the software to meet these requirements. • Customers can then add, remove, or re-prioritise their requirements based on the delivery of the software. • At the end of each iteration, which is usually in less than a month, both the customer and the developer can evaluate the project and start a new cycle if needed. This dramatically shortens the time required for the project, since functionality designs are loose. Developers are free to implement their understanding of the requirements, and the customer is free to request that the functionality is changed to better suit their needs. However, this more open development environment also introduces risks to the project.
RISKS OF UNDISCIPLINED AGILE DEVELOPMENT The lack of formal design processes, one of the key benefits of agile methods, means that it is hard to get meaningful metrics for implementation teams. This lack of measurement is compounded by the global nature of modern teams, who have geographic, language, and cultural barriers.
OCTOBER 2013 | www.testmagazine.co.uk
AGILE DEVELOPMENT
SPECIFICATIONS CREATE A WAY FOR DESIGNERS TO COMMUNICATE A CUSTOMER’S INTENTIONS TO DEVELOPERS. THE FRAMEWORK managers, product managers and These barriers can lead to longer ENABLES AGILE DEVELOPMENT customers are always aware of the projects if programmers are BY INTRODUCING PROCESS potential impact these changes disconnected from the business logic AND TRACEABILITY can have. of the application and do not correctly TO CHANGES IN implement customer requirements. The constant communication that is required to REQUIREMENTS CHANGE MANAGEMENT keep the pace of agile projects is also more difficult to maintain for these dispersed teams. The entire evolutionary development model can quickly breakdown into chaos.
BALANCING AGILE WITH BUSINESS NEEDS In order to mitigate these risks, a design process needs to be added on top of the development process. This conceptual framework allows customers, product managers and implementation teams to interact in a measurable way. Instead of implementation occurring isolated from customers and designers, specificationdriven development (SpecDD) provides the needed process, without adding overhead to the project timeline. SpecDD states: • Business logic always dictates the correct way for software to function. • Requirements must be formally understood and translated into specifications. • These specifications, along with their related requirements and other collateral (called knowledge) form a ‘conceptual product’. • The conceptual product guides engineering and QA testing. • A conceptual product, as the complete blueprint of an application, should be compared to the implemented product. • Any deviations between the implemented and conceptual products must be corrected, since the conceptual product is an accurate representation of the application.
BENEFITS OF SPECDD The conceptual framework provided by SpecDD provides immediate benefits. First, it enables distributed development teams to interact with product managers and design teams. A core team can take a customer’s requirements and reinterpret them into developer-friendly language as a specification. Developers only need to worry about implementing the specification. If the customer’s requirements change, this can trigger a change to the specification. However, unlike our previous example where this could dramatically alter the course of the project, the impact of a change can be measured. Since the specification works as a parent item to all development tasks, project
OCTOBER 2013 | www.testmagazine.co.uk
SpecDD also makes it possible to quantify these changes. For changes to specifications or requirements that the implementation team is currently working on, an approval process can be used to analyse the change request. This analysis can look at each specification’s linked development tasks or test plans and flag items that are related to outdated specs and requirements. This allows for a living, parallel design process to take place even during development – providing a great degree of agility while still maintaining a consistent vision for the application. Change requests in SpecDD can be used to perform cost and impact analysis.
TRACEABILITY From a specification, all relevant data should be communicated. This information includes related requirements, development work items, QA test plans and customer support issues. Viewing all these elements together gives a full matrix of traceability for each specification. This empowers decision-makers by presenting all relevant information from a single viewpoint.
SCALABILITY AND COLLABORATION THROUGH SPECIFICATIONS The specification also provides a way for product managers to collaborate with developers. Each specification can allow team members to voice their opinion of the specification through a voting concept. The votes for a particular specification’s ease of implementation can be compared to its cost and revenue estimates. This allows product managers to forecast the success of a specification, and alter it if needed. Specifications can also be broken into multiple pieces and assigned to different teams if needed.
SPECIFICATION-DRIVEN SUCCESS Specifications create a way for designers to communicate a customer’s intentions to developers. The framework enables agile development by introducing process and traceability to changes in requirements. This in turn allows the developers to respond quickly to design changes and deliver better increments of the software. The project moves forward at all times, even while exact details of specifications are evolving. SpecDD maintains the efficiency of agile development, while also maintaining the quality and vision of the business behind the project. It also provides a way for businesses to adopt a scalable agile process without adding additional risk to their process.
PAGE 33
ADVERTORIAL
OPPORTUNITY KNOCKS: IT’S TIME FOR A NEW BREED OF TESTERS TO FACE THE FUTURE Stephen Allott, test analyst trainer, Capita IT Professional Services, discusses the “new breed” of software testers and the wealth of opportunities that are now available to this new generation…
PAGE 34
OCTOBER 2013 | www.testmagazine.co.uk
ADVERTORIAL STEPHEN ALLOTT TEST ANALYST TRAINER CAPITA IT PROFESSIONAL SERVICES
THE CHALLENGES PRESENTED TO TESTERS OFFER A WEALTH OF OPPORTUNITIES FOR YOUNG PEOPLE WITH AMBITIONS TO WORK IN THE FAST-MOVING AND HIGHLY PAID IT INDUSTRY
S
oftware testing, just like talent shows, has come a long way since the 1980s. Today’s testers absolutely must have the X-Factor if they are going to succeed in the modern competitive arenas such as cloud computing, Big Data and mobile apps/mobile device testing.
to flourish and helping to grow the profession. However, whilst ISTQB delivers a useful basic grounding in testing theory, one of the criticisms of these certifications is that they are out-of-touch with the practicalities of today’s large, complex, multi-dimensional, integrated systems. So, to survive, they must adapt and modernise to the new agile way of thinking about software testing.
Apologies, but John, Paul, George, and (what was the other guy’s name?) are simply not equipped for testing apps developed by the Facebook generation; their green screens, batch runs and client/server protocols are best consigned to history, in our opinion.
Also, there are still some organisations and IT departments who view testers as a commodity, and don’t really see testing as a professional occupation. However, many more are beginning to see the wider view of the professional tester and their important contribution to a project. Testers are increasingly becoming an integral part of the whole software development lifecycle. They are involved in early reviews of requirements and use realistic (sometimes live) data, production equivalent test environments and sophisticated tools to carry out their tasks that are nowadays vital to the success of the project, not just a “nice to have if there’s time left in the schedule”.
A new breed of tester now sits in the wings; these new testers are able to deliver a technical, holistic, and career-focused approach to what was once a mundane nine to five job. Gone are the tediously repetitive tasks. Gone are the fake, unrealistic data sets. Gone are overnight batch runs where printed output was meticulously checked to ensure there were no “bugs” in the program. The new testers add considerable value in a fast-paced, complex world and experience a breathtaking journey through the protocol stacks of cyberspace, requiring up-to-the-minute technical know-how, dedication and a problem-solving ability that is second-to-none. The complexity and scale of modern computer systems is potentially beyond many people’s imagination and whereas the developer tends to be focused on one or two specialised aspects, the new tester usually has to comprehend the bigger picture. The challenges presented to testers offer a wealth of opportunities for young people with ambitions to work in the fast-moving and highly paid IT industry.
START WITH THE BASIC SKILLS For over 15 years ISTQB (formerly ISEB1) has provided software testing certifications at Foundation, Intermediate and advanced levels, allowing individuals
OCTOBER 2013 | www.testmagazine.co.uk
THE NOVUS TRAINING PROGRAMME To address the perception that testers are unskilled, not technical, unable to communicate, not valued as true IT professionals, Capita developed an innovative, modular, and practical training programme for aspiring software testers, called Novus. The thinking behind its evolution stemmed from the need to create a new breed of tester, one that is more in tune with today’s business world, one that has the ability to communicate with peers, managers and clients, avoiding technical jargon and using business language to get their point across and their opinions heard. The Capita Novus training programme transforms eager, confident, ambitious IT graduates, into multi-skilled, technically savvy, inquisitive, problem-solving, software testing practitioners. An intensive three months of realistic, practical training at our purpose-built training centre in London allows our Novus to transition from university life
PAGE 35
ADVERTORIAL SOFTWARE TESTING PROMISES AN ENORMOUSLY VARIED CAREER AND IS ONE WHICH OFFERS A RICH SET OF REWARDING OPPORTUNITIES. BUT LIKE MOST THINGS IN LIFE, IT DOES NOT GET HANDED TO YOU ON A SILVER PLATTER
to corporate life so that they are ready to face the realworld challenges presented to them by our many private and public sector clients. Our Novus programme is intense and challenging to both the candidates who gain a place, and to those who run the course. Some of the issues we face when training are that many of the graduates appear to have been spoonfed by their university teaching, and although excellently equipped to pass exams, do not yet have the experience to communicate with peers, build commercial acumen, or solve problems within a corporate environment. This is when our soft skills/life skills module becomes rewarding, as it teaches the Novus to develop their interpersonal skills, and draw them out of their comfortzone in order to encourage confidence and assertiveness. Working on simulated projects, based on realistic specifications, the Novus programme experiences the technical challenges, the politics, the deadlines, and the hard, fast-paced realities of a large commercial operation. The ability to communicate effectively and openly within their peer group and to senior managers, managing a project from start to completion, and developing commercial awareness about the business; are areas that we focus on with Novus. We want to ensure that when they are deployed on site for the first time, they are perceived and behave as professional working people. We understand that being technically amazing is important, but that it is much more effective when these technical skills are communicated clearly to colleagues and clients. We are very selective with the people we choose for our Novus. The attributes we favour are motivation, determination, passion, and a real hunger for a career in IT. We look for those who can think on their own and within a group environment. Communication is vital and although we teach this as part of the programme, we want to see
PAGE 36
evidence of this; whether from university days or from some other work experience. One thing we don’t like, however, is arrogance, so we avoid bringing on board those candidates who think they know it all. Having a humble approach to learning, we believe, is the key to success. At the time of writing this article, our Novus programme is now on its 17th group of enthusiastic and ambitious graduates. It is a successful programme, and the team that runs it works hard to select the very best candidates, and to deliver the training thoroughly and methodically.
TAKE OWNERSHIP OF YOUR CAREER Software testing promises an enormously varied career and is one which offers a rich set of rewarding opportunities. But like most things in life, it does not get handed to you on a silver platter. We encourage our testers to get more involved in the industry, read around the subject, make tough decisions and “take ownership of their career” from the very beginning. Our recommendations include becoming members of both offline and online testing communities, as these give a wealth of insight into the world of software testing and some good tips and ideas can be picked up. It’s worthwhile for testers to follow what’s trending in the IT industry as a whole, and more specifically testers must keep a close eye on future topics such as Big Data, cloud, and mobile testing.
KEEP UP-TO-DATE WITH TECHNOLOGY... A career as a software tester is a continuous learning process. The rate at which our technology is developing is creating a need for this new breed of software tester. It’s not just the Internet and desktop applications that we must consider for testing purposes. Mobile phones, tablets and other handheld devices have now become an essential part of the increasingly complex network
OCTOBER 2013 | www.testmagazine.co.uk
ADVERTORIAL
THERE ARE STILL SOME ORGANISATIONS AND IT DEPARTMENTS WHO VIEW TESTERS AS A COMMODITY, AND DON’T REALLY SEE TESTING AS A PROFESSIONAL OCCUPATION of hardware and infrastructure that must be professionally tested. This new breed of tester will be able to solve the trickiest technical problems that will help discover the fiendish bugs that lurk behind the pages of our websites and apps.
solving skills, attention to detail, technical competence and above all an ability to communicate complex ideas in a clear, concise and meaningful way that appeals to all stakeholders on the project.
Customer expectations have risen. Very soon, project managers, test managers and team leaders will quite rightfully demand testers that can do much more than write a decent test script. They will also expect problem
Capita IT Professional Services is the sponsor of the Best Overall Testing Project - Retail award at the European Software Testing Awards in November.
References 1. I SEB – information systems examination board, now replaced by the international software testing qualifications board (ISTQB) who offer certifications in the UK through the BCS (The Chartered Institute for IT).
OCTOBER 2013 | www.testmagazine.co.uk
PAGE 37
THOUGHT LEADERSHIP DAVID BORCHERDING MARKETING COMMS SPECIALIST SEAPINE
EXPLORING FOR THE DEEPEST, DARKEST DEFECTS David Borcherding presents the case for explanatory testers and explains how such testing goes that one step further to ensure a successful outcome…
M
any testing teams rely solely on scripted testing – both manual and automated – to decrease the risk of defects in the product release. However, scripted testing is not meant to identify error conditions in scenarios that significantly deviate from the design or requirements. To find these hidden or divergent risks, testers need to go off script, and that’s where adding exploratory testing can help. It’s almost impossible to plan tests that cover every variation in data, configuration, interaction, sequence, timing, and so on. Scripted tests are designed to ensure that the application meets the requirements (using new-feature test cases) and to mitigate the risk of new features breaking existing functionality (via regression test cases).
PAGE 38
Experienced testers can anticipate issues that might occur, but it may be too costly or time-consuming to write a test case for every scenario that comes to mind.
PUSH THE ENVELOPE Exploratory testing allows testers to investigate anticipated issues. It puts the thinking back in the testers’ hands; they design the test, execute it immediately, observe the results, and use what they learn to design the next test. Exploratory testers push the software to its limits, learning about the application and then applying what they’ve learned as they continue testing. This allows testers to gain a better understanding of how the application works and where it breaks. Ultimately, testers using an exploratory approach have a more complete view and in-depth knowledge of the application, including its weaknesses and hidden risks.
OCTOBER 2013 | www.testmagazine.co.uk
THOUGHT LEADERSHIP
EXPLORATORY TESTERS PUSH THE SOFTWARE TO ITS that exploratory testing can find in a lessDISCOVER MORE DEFECTS LIMITS, LEARNING ABOUT structured, but longer, test session. THE APPLICATION AND On average, 11% more overall defects are discovered through THEN APPLYING WHAT FINDING THE MOST IMPORTANT exploratory testing vs. scripted THEY’VE LEARNED AS DEFECT QUICKLY testing. For defects that should be THEY CONTINUE immediately obvious, such as a missing Because testers are not bound by the test case TESTING button in the UI, exploratory testing steps, exploratory testing makes finding critical discovers 29% more vs. scripted. When it comes to “complex” bugs (bugs requiring three or more user actions to cause an error or failure), it jumps to 33% more defects found.1
But, why does exploratory testing find more defects? It gives testers more latitude to try different types of tests, using their past experience and knowledge of the application. Scripted testing limits testers to test case steps, which in turn limits their ability to consider other test scenarios. There are numerous reasons why test cases don’t always lead to finding bugs, such as how well the test case was written (did the analyst understand the requirement?); who wrote the test case (is the analyst writing the test case knowledgeable about how the product works?; how well the requirements document described new functionality; and so on. Even if testers had perfect test cases, exploratory testing would still find more defects over the course of a release, for several reasons: • Testers tend to find a good number of defects when
“testing around” functional areas while verifying defects. Fixing an issue often breaks something else. • If a defect exists and is not found while executing the initial test run (following the test cases steps), it is unlikely that the next tester, running the same test, will find the defect. However, exploratory testing in the same functional area may reveal the bug. • Exploratory testing allows testers to think outside the box and come up with use cases that might not be covered in a test case. For example, a tester might perform one test and then ask, “What if I tried this? What if I didn’t do that?” • Some defects, typically the hard ones to find, depend on a sequence of events. If you don’t have really deep test cases, testers can miss finding defects
risks faster by allowing testers to explore more of an application and focus on testing the “what ifs.”
For example, let’s say a tester is assigned to test the “Edit Bookmark” functional area in a product. There are two possible test scenarios: 1) Execute a test run 2) Perform exploratory testing of the “Edit Bookmark” functional area. In the first scenario, the tester would perform the steps outlined in the test case. When the test is complete, the tester will either have verified that each step works, or reported the step where a bug occurred. But what if the most important bug – the one that crashes the application and deletes the user’s data, say – doesn’t occur during these steps? The tester would not have found it. In the second scenario, the tester would explore or “test around” editing bookmarks. At first, they might perform the same steps as at the beginning of the test case. As testing progresses, however, they might ask, “What happens when I click ‘Edit’, delete the name, and try to save the bookmark with a blank field?” And boom, the application crashes and deletes all of the user data. The tester just found the most important bug by exploring something that wasn’t a step in the test case.
FILL THE GAP As corporate belts tighten, and testing budgets and staff shrink, many succumb to a temptation to settle for an incomplete test approach because they know it will satisfy the standards. That just opens the door to a buggy release. Combining exploratory testing with mandated scripted testing can help fill the gap left by shrinking resources, therefore strengthening your test effort and reducing the risk of defects in the release.
References Defect Detection Efficiency: Test Case Based vs. Exploratory Testing: Link available on request
OCTOBER 2013 | www.testmagazine.co.uk
PAGE 39
FOCUS ON: BUSINESS ROS TAYLOR PSYCHOLOGIST ROS TAYLOR COMPANY
THE POWER OF SMALL WINS Psychologist, Ros Taylor explains how managers and team leaders can ensure excellent team performance, and how the smallest changes can differentiate a successful team from a not so successful team…
A
huge influencer in life is reward. Psychologists learned this early on as a result of animal experiments. You could shock, speak sternly, or even, heaven forbid, hit a small furry animal but it would not move an inch towards a goal. Quite the reverse. It would freeze. However the piece of cheese at the goal mouth worked a treat. Humans are not dissimilar. Whether we want to admit it, or not, we all like to be rewarded and not just with money – though every little helps – but praise and encouragement are as important to us as the air we breathe. Giving feedback in the workplace to colleagues and staff is essential to help them feel comfortable with what they are doing and, if necessary, help them change. Reward at a more fundamental level is a greeting in the morning or a smile in passing. Deprivation of this positive social interaction leads to an uncaring attitude and a “psychopathic” organisation. Diminishing people by ignoring them, putting them or their ideas down or talking over people at meetings are common workplace practices and are not conducive to success. Criticism can be very positive and helpful if delivered in a friendly and self-developing way. But sarcasm simply makes people feel inadequate. It erodes self-confidence, leads to resentment and over-cautious behaviour.
Support from their leaders was an essential component in good workplace performance. Where there was conflict and competition at the top, performance was reduced.
REWARD AT A MORE FUNDAMENTAL LEVEL IS A GREETING IN THE MORNING OR A SMILE IN PASSING. DEPRIVATION OF THIS POSITIVE SOCIAL INTERACTION LEADS TO AN UNCARING ATTITUDE AND A “PSYCHOPATHIC” ORGANISATION.
Internal or ‘intrinsic’ motivation was a major factor too with those who really enjoyed the challenge of their work being more productive than those who simply saw it as a job. Where there were promises of rewards, fear of poor evaluations, or competitiveness then the opposite was true. They were less productive, less happy and were less creative.
Yet, we know that reward works. For example, it was Bill Clinton at a speech in the Albert Hall a few years ago who said, “call me old-fashioned, but I believe dropping food parcels will influence those in Iraq more than dropping bombs”.
Teresa deduced that “worker of the month” schemes actually undermined good performance. It is the instant nature of immediate feedback that works in terms of learning and behaviour. If you wait until the end of the month, or the end of the year to be rewarded, you have forgotten what you are being thanked for and so lose the pairing of what you did with the "being noticed".
WHAT MOTIVATES TEAMS?
IT’S THE SMALL THINGS THAT COUNT
I met Teresa Amabile, Harvard Professor, in Atlanta where she was discussing her latest book The Progress Principle. She had asked over 200 workers from seven companies to keep a daily diary of events, feelings and actions over four months. The results were remarkable. Workplace happiness was mentioned in diaries when a leader provided positive feedback on their progress and there was even a carryover effect lasting up to two days. A good mood has a lasting effect, generating a greater variety of thinking and this led to new ideas at work.
PAGE 40
So let’s be clear, it is the small reward; the “thank you”, the hand-written card on the desk, the everyday progress feedback that really works. Did the leaders in Teresa’s study understand the power of providing feedback on progress or as she calls it, "the power of small wins"? They did not. When asked what leadership skills made a difference to workforce performance, they rated progress feedback at the bottom of the list. Small things really do make a big difference.
OCTOBER 2013 | www.testmagazine.co.uk
FOCUS ON: BUSINESS ADRIAN LEARER MANAGING DIRECTOR PLAN.IT
HOW MUCH DO YOU KNOW ABOUT TAX? Adrian Learer, MD of Chartered Accountants plan.it explains there are certain tax issues that IT freelancers need to be aware of…
W
ith zero hour contracts recently making the news, it’s easy to forget some of the other important issues that can significantly affect freelancers’ pay. One of the most pressing concerns for contractors working in the software industry is ensuring that they pay the right amount of tax on their earnings, which by the very nature of their employment, can vary widely. As an industry that often uses contractors to fill the skills gap, the area of tax compliancy has risen to prominence as HMRC comes under increasing pressure from the Government to raise additional funds via tax revenue. Tax avoidance – and the grey areas which surround it – has been highlighted by a number of recent HMRC actions and the resulting claims. These include closing a major offshore umbrella provider who avoided paying millions of pounds of employers’ national insurance contributions (NI). Obviously it is important to stay on the right side of the law, but the complexity of many areas of tax and employment legislation can make it difficult for contractors to fully understand the subject.
WORKING THROUGH AN OFFSHORE ENTITY There is no reason why UK-based IT contractors cannot work through an offshore entity: the key is why this artificial arrangement is created, especially where the contractor is both from and working in the UK. Offshore employment intermediaries have invariably been established to avoid tax and NI. In some instances, the intermediary “employer” accounts for UK PAYE but avoids NI as it has no permanent presence in the UK. In some instances, the position is compounded by UK tax being reduced by unapproved travel and subsistence schemes; whilst in others, UK PAYE is not accounted for at all, claiming that the contractor is “self-employed”. Subject to a current review announced in the 2013 Budget, the employer’s NI loophole is highly likely to be closed in future. Furthermore, it is unlikely that any PAYE saving arrangements will bear close scrutiny or confer any advantage over a UK-based solution. However, the most important point that engagers need to understand is that complex existing UK tax legislation can make end-users fully liable for the PAYE and NI liabilities of contractors engaged directly or indirectly via an offshore entity. This is a significant risk, and if interest and penalties are added to the tax bill, the result can be catastrophic.
WORKING THROUGH YOUR OWN UK LIMITED COMPANY
For contractors working in the UK, the main question is one of IR35, the intermediaries’ tax legislation. In a nutshell, this legislation aims to counter “disguised employment” and states that if not for the existence of the intermediary, there would be an employment of the worker by the end-user. The worker will then be fully taxed as an employee regardless of how they operate. The subject of IR35 is too complex to fully explain here; however, so far, carefully formulated contracts and working practices have facilitated many freelancers successfully operating in this way. For those whose contracts are not caught by IR35, there are substantial tax advantages that can be gained. As well as tax relief on a wide range of expenses, the ability to extract funds from the company in the form of dividends as opposed to salary can save on both employees’ and employers’ national insurance, totalling approximately 25% of income drawn.
UK UMBRELLA COMPANIES Working through an umbrella company is a good viable alternative to working under PAYE directly for an agency or end-user. The key advantage is derived from having an overarching employment contract with the umbrella company. This in turn provides continuity of employment, flexibility; employment rights and the opportunity to increase take home pay via the deduction of expenses incurred wholly, exclusively and necessarily in performing the duties of paid employment. However, there has been much debate over what expenses can be claimed, especially with regard to travel, but the professional umbrella company provider should be able to maximise these claims, subject to individual circumstances.
WORKING OVERSEAS Both UK limited companies and umbrella companies can theoretically be used when working overseas, but are rarely ideal. Where work is performed overseas, then a UK limited company needs to be registered for tax in the local territory and it is invariably easier to work through a locally-based limited or umbrella company. Most foreign countries permit a UK umbrella company to send a worker overseas for up to 183 days per tax year and be exempt from local taxes, but this may not apply to local social security/national insurance liabilities. Ultimately, the best way to stay compliant and make the most of your earnings is to get the right advice. Not only does this offer the reassurance of compliancy, but you can also ensure that you are taking home exactly what you are entitled to.
There is no restriction on a contractor setting up such an arrangement, but does it offer any significant advantage? PAGE 42
OCTOBER 2013 | www.testmagazine.co.uk
LAST WORD DAVE WHALEN PRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST WHALEN TECHNOLOGIES HTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM
THE EVOLUTION OF THE “SUPER TESTER” By Dave Whalen
L
ike all species, testers too must evolve. I’m not saying we’re monkeys (most of us anyway), but if you’re going to do this job for a while, you have to move up the evolutionary ladder. You need to learn new skills and enhance whatever skills you already have, which holds true for any profession. If you remain stagnant you begin to stink up the joint, so before they breakout the disinfectant, move up the ladder… Become indispensable. What to do? Thankfully, in this day and age there are a lot of options. If you watch the industry trends as I do (because I apparently need a hobby), you will see that there seems to be an increasing demand for testers with automation skills. Look even further and you may find a gap in specific automation skills or tools. In my case, the tool in dire need is an open source tool, which has two pieces – a record/playback tool and a programming tool. Record and playback is not really an advanced skill so it’s usually a good first step if you have never done test automation. Almost anyone with basic computer skills can record and play back a test, so if you really want to become a valuable part of the team, dare I say it, you’re going to have to learn to write some code. I realise many of you may be trembling with fear at the very thought of having to write even a single line of code. Take a deep breath and count to 10, I’m not asking you to be a developer. Developers are my friends, and I love them all (OK, most of them) but no offence, I don’t want to be one. So, how can you become a pseudo-developer? Determine what language you need to learn. My recommendation is to become fluent in Java and C#. Especially Java. Luckily, C# is very “Java-like”. Both are object-oriented languages and the structures are very similar. If you know you are going to be using one of the common vendor-supplied tools, you may want to look at Visual Basic (VB). What’s more, the development tools for both Java and C# may be downloaded for free! So you’ve identified a language and downloaded the development tool. Now what? Turn on some Beach Boys music, fire up that search engine and let’s go surfing. Look for anything associated with learning both the automation tool and the language(s) they use. A hidden gem that I recently discovered is YouTube, which I stumbled across by accident. I found an entire course on the tool I selected. Free training but, be careful as many will offer the first hour free as a teaser, but then you have to pay for the rest. But I’ve learned quite a bit from free training. Also, if your
OCTOBER 2013 | www.testmagazine.co.uk
company is willing to foot the bill for more formal training, take advantage of it. Never turn down free training!
YOU NEED TO LEARN NEW SKILLS AND ENHANCE WHATEVER SKILLS YOU ALREADY HAVE... BECOME INDISPENSABLE
You can also look for industry conferences. I teach some of these tools at software testing conferences locally and across the country. Don’t be afraid to look to co-workers for help too. One of the developers on my team noticed that I was researching Java training online. He recommended a good book and offered to help me learn, and I learnt how real world code is written. He reviewed my code and showed me how to do things better, and more efficiently. This was probably the most valuable training that I received. It only cost me lunch at the local Mexican restaurant. Lastly, don’t overlook the most formal training of all – colleges and universities. These ones will cost you; both in tuition and books. Fortunately, there may be financial aid available so use it! By far the most valuable tool is experience. Roll up your sleeves, and start writing code. It can actually be fun. You’re now on your way to becoming Super Tester so you may want to get fitted for a cape.
PAGE 43
Burn Your Spreadsheets
Manage product testing with TestTrack TCM TestTrack TCM tracks all details of product testing for you, including test cases, suites, runs, and results. Now you can have real-time visibility over your testing effort and more time to plan and manage. • Achieve real-time visibility into testing progress. • Reduce risk and achieve greater confidence in the release. • Gain more time to plan projects and manage your team.
Download a free, 30-day trial of TestTrack TCM at www.seapine.com/testuk © 2013 Seapine Software, Inc. All rights reserved.
the 20 LEADING software TESTING PROVIDERS
2013
20 LEADING TESTING PROVIDERS
FOREWORD... Welcome to the 2013 20 Leading Software Testing Providers guide. It’s been exactly 12 months since we last published this guide, and a lot’s changed in that time; not only the team behind TEST Magazine, but also in regards to the software testing industry. With technological developments moving at a fast pace, a lot can happen in the space of a year. Many companies have enhanced or adapted their product portfolios to meet the demands of today’s software testing industry and recognised the tools and services that will appropriately compliment a professional’s skillset. This section of the magazine is aimed to help you understand the key differences between the various products and services on offer from the leading providers. With the insight gained, we hope this will help with future purchases. I hope you find this guide useful and equally helpful.
Sophie-Marie Odum Editor
PAGE 46
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
20 LEADING TESTING PROVIDERS
BORLAND Borland: delivering better quality software, faster The Borland suite of delivery tools meet testing challenges with software solutions. Borland – a Micro Focus company – has created products that evolve to meet business needs both today and in the future. The Borland roadmap shows the way through a shifting landscape and offers a suite of products that work throughout the entire development journey, embedding quality assurance from requirements definition to "go live". Borland: Define. Manage. Test Borland delivers world-class open and agile requirement, test and change management solutions that realise genuine cost savings for our clients. Our software development products transform good software into great software right across the application development lifecycle. The future will be open and agile so Borland has developed powerful products that work the way you, your team and your users want to work and will need to work in the future. Why remove defects at the end of development when you can identify and eliminate them at the beginning of the process? Borland Solutions focus on identifying and eliminating defects and delivers across three key areas: Requirements: Caliber, the requirements
definition and management tool, uniquely combines requirements definition, visualisation, and management into a single "Three Dimensional" solution. With it, managers, analysts and developers have the detail they need to engineer great software. To remove ambiguity in the requirements definition and management process and clarify the direction of the development and QA teams. Without it, they risk poor business outcomes. Change: StarTeam creates the single "source of truth" that empowers development teams to regain control in a constantly-shifting world. It’s a reference point to help testers – even global teams with complex environments and methods – prioritise and collaborate on defects, tasks, requirements, test plans and other in-flux artefacts.
A: The Lawn, 22-30 Old Bath Road, Newbury, RG14 1QN T: 00 800 581 02130 W: www.borland.com
Quality: The Silk Portfolio automates the quality process, from inception through to software delivery. Some solutions emphasise "back-end" testing. Silk tests are planned early and synchronised with business goals, even as requirements and realities change. Borland solutions provide an open framework, integrating diverse toolsets, teams and environments. Managers enjoy continuous control and visibility over the development process to ensure that quality output is delivered on time. Borland. Better software, faster.
BRANDT TECHNOLOGIES Brandt was established in 2002 and is an international business headquartered in Ireland, with offices in Germany and China. Brandt enables international software publishers in the information technology, antivirus, e-learning and mobile applications sectors to enter global markets. Brandt specialises in the translation, localisation and testing of software projects. Projects completed to date include: Internet portals, anti-virus software, CRM, games, e-learning, bespoke software solutions and consumer products. Through Brandt’s extensive experience of delivering software to global markets, it have developed processes and proprietary technology to support its clients’ needs. The challenges faced include faster turnaround time, reduced costs, growing language requirements and consistent quality across multiple platforms. Brandt’s patented Shadow technology has been developed to specifically meet these challenges. Products Shadow Test WorkBench provides a test platform which enables a test engineer to simultaneously test software and hardware applications in parallel. Brandt’s innovative approach was designed to conduct tests
across platform, browsers, languages and mobile devices. Shadow has been successfully applied to a broad range of test applications which include exploratory, functional, localisation, internationalisation and language testing. The parallel testing approach has proven to reduce manual testing effort, reduce bug counts and provide higher test coverage. SORT The Shadow online review technology (SORT) portal in combination with the Shadow Test WorkBench enables the product expert to share screen data, recorded scripts and create demo videos. This content can be published for use across an organisation for language review, documentation, training and marketing materials.
Number of sites globally: Three A: 3 Jocelyn Mall, Jocelyn St., Dundalk, Co. Louth, Ireland T: +353 42 942 0700 E: info@brandttechnologies.com W: www.brandttechnologies.com
Benefits of adopting Shadow • Cost reduction • Consistency of testing • Scalability • Traceability • Adaptive control
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
PAGE 47
20 LEADING TESTING PROVIDERS
BRICKENDON CONSULTING Brickendon Consulting provide management and technology consultancy services to investment banking and energy trading firms. They offer innovative solutions to their clients through consulting divisions in the following areas of expertise; e-trading, banking applications, energy trading, regulatory reform, trade monitoring, business change, architecture and test and quality assurance.
and Knowledge Management teams, consultants create and transform innovative ideas into strategic solutions which form the basis of a Brickendon approach. Brickendon test experts are encouraged to proactively evaluate challenges within a project environment and consider innovative methods and process improvements to advance their Test and QA offerings.
Brickendon Consultants are market experts within their respective fields and work within the Management Consulting or Technology Consulting divisions:
Examples of Brickendon Approach’s in Test and QA include:
A: 107 Cheapside, London EC2V 6DN T:
+44 (0) 207 397 3760
E: W:
info@brickendon.com www.brickendon.com
• Brickendon TCPS – Time check point system • Brickendon ITTA – Integrated test tools approach • Brickendon TMS – Test metrics solution • Brickendon ELMO – Electronic latency monitoring • Brickendon FTAS – Functional automation solutions • Brickendon CTF – Continuous testing framework
Management Consulting services include: • Business strategy formulation and alignment • Programme and change management • Multidisciplinary management • Industry and market analysis
The Brickendon approach brings a number of tangible benefits to a client’s organisation including:
Technology Consulting services include: • IT project management and analysis • IT architecture • IT development • IT testing and QA • IT managed services The expertise of Brickendon’s individual consultants is gathered together, organised and shared internally across the organisation, providing the basis for continuous learning and professional development for their consultants. Through collaboration between Brickendon’s Research
• Increased consistency in test project management • Enhanced transparency in test project management • Evidence-based performance management • Improved accountability of the testing team • Improved communication, allowing more proactive stakeholder engagement Contact Brickendon Consulting to find out more about what they offer and how they could benefit your business.
BUGFINDERS Web and mobile testing – rapid, high quality and massive coverage from large enterprise to small start-ups, companies trust BugFinders to deliver or compliment their testing team and produce the best quality software. Over 50,000 testers, 700+ web and mobile platforms, 80+ countries, BugFinders can help deliver the areas of testing that you couldn’t viably do before. • Web or mobile coverage: Our massive scope of devices means we can complement your own testing by covering a range of devices that it is not viable to purchase and maintain internally. • Localisation: Need testing or user analytics in remote locations. BugFinders can deliver testing when and where you need it.
PAGE 48
• Regression testing: Multiple cycles can be boring and time consuming for an internal team. Using our scalable team, we deliver fresh-eyed regression delivering weeks of works in just a few days. If you would be interested in a conversation about how we can support your testing needs, get in touch with us. We look forward to speaking to you soon.
Contact: Martin Mudge Technical Director A: 3 Rockfield Business Park Old Station Drive Cheltenham, GL53 0AN T: +44 (0) 844 870 8710 E: enquiries@bugfinders.com W: www.bugfinders.com
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
20 LEADING TESTING PROVIDERS
COVERITY processes to enable developers to build quality and security into the development process at the earliest possible stage, for fast, efficient delivery of high-quality software.
Coverity enables organisations to build quality and security testing into the software development process at the earliest possible stage, for fast, efficient delivery of high-quality software.
The Coverity Development Testing Platform automatically tests software source code for defects that could lead to product crashes, unexpected behaviours, security breaches or catastrophic failures. This proactive, development testing solution provides companies with accelerated time-to-market; enhanced customer satisfaction; increased revenues; reduced costs; and lower risk of costly; brand damaging software failures and security breaches. The platform is built on the Coverity SAVE static analysis verification engine, which incorporates multiple patented issue detection techniques and is based on the analysis of more than five billion lines of proprietary and open source code.
In this day and age, every company is a software company – from hundred-year-old Fortune 500 manufacturing companies to up-and-coming technology startups. It is no longer possible to be a “traditional” company without also being a software company. Organisations in all types of industries are now developing and/or using software to remain competitive – and relevant – in their respective markets. However these companies often struggle to maintain software quality and security in the face of increasing software complexity and intense time-to-market pressure. The worldwide cost due to poor software quality is estimated at $500 billion per year and software failures in the field can have a disastrous impact to brand reputations. Conventional approaches to managing quality and security are no longer sufficient; they are too reactive, cannot find all of the issues and occur late in the lifecycle, which can result in product delays and increased costs.
A: Unit 2, Archipelago Lyon Way Frimley Surrey GU16 7ER T: +44 (0)127 640 9500 F: +44 (0)127 640 9528
Coverity also offers its development testing technology as part of its free Coverity Scan service, which was initiated in 2006 with the U.S. Department of Homeland Security as the largest public-private sector research project in the world. The free service helps the open source community build quality and security into their software development process.
Coverity pioneered development testing, which combines specialised software and
EXPERIMENTUS Experimentus is an IT solutions and services company who, working in partnership with our clients, understand their business and technical objectives, and develop solutions designed to reap the benefits of a "shift left" mentality, these include:
TMMi and iTM Licensing – Licensing our accredited Test Maturity Model (TMMi) assessment method and our intelligent Test Method (iTM) to help companies achieve better/ more efficient software management processes throughout their organisation.
• Provide confidence in outcomes.
All of our services are underpinned by our award-winning intelligent Test Method (iTM), a complete test management framework. Incorporating comprehensive process maps, procedures, templates, checklists and guidelines and comprehensive training material, this based upon TMMi, International testing standards and ISTQB. iTM can be used to help improve and/or supplement an existing framework and is suitable for small to global test programmes.
• Prevent software defects rather than detect them. • Accelerate time to market. • Meet stakeholder expectations. • Provide independent advice and specialist knowledge. Our solutions address four key areas aimed at optimising our clients approach to software quality management. Transformation – Moving an organisation from where they are to where they should be – by managing the change of process, people and technology. Advisory and delivery – Delivery of test management consultancy and managed services. TMMi Assessment and Certification – Assessing and certifying companies against the industry standard Test Maturity Model (TMMi) and where required providing a roadmap of improvements and return on investment.
A: 17a Dorset Square, London, NW1 6QB T: +44(0)207 871 2300 E: Info@experimentus.com W: www.experimentus.com
Based on our experience of conducting TMMi assessments worldwide, we were the first company to offer the TMMi Professional training course and exam. For any further information on our solutions/ course details and dates please use the QR code or visit our website.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
PAGE 49
20 LEADING TESTING PROVIDERS
GRID-TOOLS Grid-Tools are the leading test data management vendor, specialising in end-to-end test data solutions for both agile and traditional development and testing environments, including those in the cloud and virtualised services. Grid-Tools enables companies to provision high quality test data for their non-production environments and outsourced operations by offering innovative solutions for synthetic test data creation, data masking, test case design, intelligent data subsetting and SOA virtualisation. Datamaker is the complete, end-to-end test data management suite from Grid-Tools, containing the world’s most powerful test data generation engine, and enabling users to provision high quality, compliant test data that is "fit for purpose". Datamaker, contains powerful functionality and components, including: Data security, intelligent data subsetting, synthetic data generation, data profiling, coverage techniques, test case design Enterprise Data Masking is the complete data masking suite from Grid-Tools, powered by Datamaker. Containing four industrial-strength masking engines, which can be used in isolation or as part of an enterprise-wide masking solution, Enterprise Data Masking provides support for major database types, as well as mainframe, flat files, and service message requests and
responses; enabling the tool to provide the flexibility and functionality required to meet any organisations data privacy challenges. Grid-Tools most recent data design solution, Visual Test Flow, is a tool which allows users to map any process to a flow chart, providing a clear visual path through logic gates in your requirements, eliminating ambiguities that arise within written specifications. The tool provides users with multiple methods of optimal test path design, allowing users to cover all "happy" and "unhappy" paths in the minimum number of test cases. When Visual Test Flow is linked with GridTools flagship product, Datamaker, users will be able to find and make the "right" data to satisfy their test cases, and link it directly to the logic gates and processes defined in the flow chart.
A: 11 Oasis Business Park, Eynsham, Oxford, OX29 4TP T: +44 (0)186 588 4600 E: sales@grid-tools.com W: www.grid-tools.com
Intelligent Virtual Services is the virtualisation solution from Grid-Tools, which provides a stable and isolated environment in which teams can work with test data that is structured and compliant, to maximise he quality of their SOA testing. Grid-Tools has been working with some of the largest government agencies, telecoms, financial intuitions and insurance providers around the globe. Grid-Tools’ focus is to ensure that better processes and best practice methods are in place for provisioning compliant test data that is "fit for purpose".
MAVERIC SYSTEMS Maveric Systems Limited is a preferred “IT lifecycle assurance” partner to leading corporations in the banking, insurance and Telecom verticals in UK, Europe, APAC and Middle East. The company’s Requirements Assurance, Application Assurance and Program Assurance services are aimed at eliminating quality, cost and time-to-market risks associated with large IT transformation programs. At the heart of Maveric’s value proposition is its ‘Assurance-only’ business model aided by domain and IP led integrated solutions that cater to diverse assurance requirements from inception to go-live. Maveric’s best-in-class methods in requirements definition, requirements validation, project management, leading product configuration, ITIL process consulting and functional and nonfunctional testing have been widely recognised. Maveric was the recipient of Frost and Sullivan Product Innovation award, as well as the Banker Middle East Industry Awards 2013. Maveric has been recently identified as a transformation specialist serving ‘Transformation Focused Clients’, by NelsonHall, a leading analyst and advisory firm. In UK and Europe, Maveric’s services have been enlisted by clients such as Citibank, Deutsche Bank, RBS, Capita, Tesco, MCIS
PAGE 50
Zurich and Ericsson. Maveric has also been working with emerging leaders such as Metro, Kleinworth Benson , KBL and Rabo Bank. Maveric focusses on product implementation assurance and has to its credit multiple engagements around a large number of banking , insurance and telecom products, including Temenos T24, Flex Cube, Finacle, Eximbills, Trade Innovation, BankTrade, Fidessa, Sungard, Murex, Ingenium, TIA, BSCS, Siebel, TIBCO, CS5. Maveric continuously strives to up-hold its commitment to the principles of independence, vertical focus and innovation. Further, its “assurance only” focus which favors depth over width has resulted in many leading players such as CLS, Deutsche Bank and IAG selecting it as a niche vendor in the last 12 months.
A: Lords Tower, Block 1, 2nd Floor, Plot No. 1 & 2 NP, Jawaharlal Nehru Road, Thiru Vi Ka Industrial Estate, Ekkaduthangal, Chennai 600 032, India T: +91 44 4344 2500 E: info@maveric-systems.com W: www.maveric-systems.com
Maveric has turned its insights across requirements definition, product configuration and acceptance testing into a licensable innovation called “AssureHawk”, which accelerates upgrade initiatives around a number of applications including Temenos T24 , Flexcube, VisionPlus and Tsys. With a workforce of 1,000 located in offices across London, Princeton, Dubai, Singapore and Chennai, Maveric delivers 100% successful IT implementation to clients.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
20 LEADING TESTING PROVIDERS
MICROSOFT Founded in 1975, Microsoft (Nasdaq “MSFT”) is the worldwide leader in software, services and solutions that help people and businesses realise their full potential. The UK subsidiary established in 1982, currently employs over 2,400 people. The main functions are service, support, and sales and marketing for our UK customers and partners. ALM is core to the success of development teams throughout the world. Microsoft Visual Studio provides a family of tools for roles across the entire application lifecycle. From requirements capture and traceability via project management tools that enable developers and testers to work collaboratively in role specific environments to deliver and test high quality solutions. Microsoft Visual Studio Ultimate, Premium and Test Professional include:
Continuous feedback loops – Rich stakeholder engagement and feedback gathering tools to enable continuous feedback loops in envisioning and building the right products. Visual studio team foundation server – for full lifecycle compatibility and visibility. Test lab management simplified – Simple, powerful and flexible test lab management services to enable configuring and managing environments for executing tests. To find out more visit: www.microsoft.com/ visualstudio Download a trial of Visual Studio: www. microsoft.com/visualstudio/eng/downloads Find out about upcoming events and latest news by following our twitter @VisualStudioUK
A: Microsoft Campus Thames Valley Park Reading RG6 1WG T: 011 890 94601 E: ukvs@microsoft.com W: www.microsoft.com/visualstudio
Comprehensive testing tools – Broad testing tools to enable manual, exploratory, and automated testing of software, with work item planning and tracking interoperability.
PARASOFT Service virtualization Test earlier, faster, more completely with service virtualisation. Service virtualisation delivers a simulated test environment that allows you to test earlier, faster and more completely. The result: • OpEx reduction by cutting wait time, configuration time and access fees. • CapEx reduction by reducing the need for hardware, licenses and lab infrastructure. • Risk reduction by enabling teams to test earlier, faster and more completely. • Incremental top-line revenue by accelerating innovation and time to market. Service virtualisation solutions are commonly applied in the following situations: • Agile and parallel development: To avoid the process bottlenecks that commonly occur when teams are developing/testing interconnected system components in parallel. • Performance testing: To rapidly configure realistic and consistent transaction performance from dependent system components as well as avoid the oftenexorbitant access fees and wait times associated with performance testing. • End-to-end functional testing: To provide access to all the distributed system components involved in the end-to-end
transaction – with the necessary configurations – whenever the team is ready to test SOA quality as a continuous process Parasoft empowers organizations to deliver better business applications faster. We achieve this by enabling quality as a continuous process across the SDLC – not just QA. Our solutions promote strong code foundations, solid functional components, and robust business processes. Parasoft's SOA solution provides an automated infrastructure that enables SOA quality as a continuous process, allowing you to reap the full benefits of your SOA initiative.
T: +44 (0) 208 263 6005 E: pcollett@parasoft.com W: www.parasoft.com
Continuous functional regression testing Parasoft's SOA solution assists you in managing the complex and distributed nature of SOA. Given that your SOA is more than likely to span multiple applications, departments, organizations and business partners, this requires a component-based testing strategy. With the Parasoft solution set, you can execute a component-based testing strategy that ultimately allows you to focus on the impact of change. Parasoft's continuous regression tests are applied to the multiple layers throughout your system. These tests will then immediately alert you when modifications impact application behaviour providing a safety net that reduces the risk of change and enables rapid and agile responses to business demands.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
PAGE 51
20 LEADING TESTING PROVIDERS
RANOREX Test Automation for Desktop, Web and Mobile Applications What kind of application do you need to test? Is it installed on the Windows Desktop? Does it run in a browser? Is it used on Smart Phones or Tablets? It does not matter which platform your software is developed for. Ranorex provides a cost-effective and comprehensive test automation tool to create reliable automated tests for any kind of application technology. Due to its ease-of-use, increased testing accuracy and low cost-per-seat, Ranorex is an excellent choice for software teams of virtually any size or level of sophistication, and is now used by hundreds of enterprise and commercial software companies around the world. Reduce Test Maintenance and become agile Not only software development teams move to agile methodologies. Also testing teams need to become agile too. Automated testing is one of the key factors to deliver in time and to ensure high software quality in today’s fast moving world. In processes like this, robust and reliable tests scripts are absolutely essential. There is no time left to spend days for test script “reanimation”. Ranorex tools assist you in being prepared. A modern, object based test automation approach separates your logical test case structure from the technical identification layer. Moreover, addressing UI elements on your desktop, in the browser or on your mobile device
is done by the powerful RanoreXPath, allowing your test scripts being robust against UI changes in the system under test. Working in Teams – Not Everyone has to be an Expert! Seriously, test automation is more than Capture&Replay, right? – You are right! At same time, we know that not everyone in a test team has the skills to implement automated test scripts. For this reason, the Ranorex tool set offers different test automation approaches making it easier to work in teams. Ranorex Studio assists testing teams working together on a test automation project. While domain testers might focus on which test cases have to be automated, automation experts concentrate on preparing reusable automation modules (key words).
Ranorex
®
Entry Level Price: € 1,480.00 A: Strassganger Strasse 289, 8053 Graz, Austria T: +43 316 281328 E: info@ranorex.com W: www.ranorex.com
You don’t have any experts? No problem. Ranorex Recorder is perfect to get started with Ranorex. Integrated in Ranorex Studio Ranorex Recorder is a lot more than classic Capture&Replay. It allows users to simply creating flexible and reusable automation modules without hitting the ‘Record’ button.
SEAPINE SOFTWARE Seapine Software is the leading provider of process-centric product development solutions. Our tools and services ensure the consistent release of high quality products, while providing traceability, metrics and reporting, and compliance. Seapine’s solutions are especially suitable for regulated, safety-critical, or mission-critical product development where traceability, auditability, and repeatability of processes, activities, and artifacts are critical. By linking requirements or user stories with test cases, defects, and source code, the solutions enable teams to manage the strictest compliance initiatives. From requirements definition to test planning, through development, testing, delivery, and reporting, Seapine helps you manage all phases of the product development lifecycle. Manage development projects with TestTrack TestTrack is the definitive tool suite to manage all application development phases and artifacts. From requirements, user stories and release planning, through sprints, assignments and work items, to test cases, QA cycles, defect resolutions and releases, TestTrack helps you deliver quality products on time. Control changes to digital assets with Surround SCM
critical to delivering quality products on time. Surround SCM is a source code control solution that tracks and manages changes to source code, documents, test data and results, and other digital assets. Automate functional regression and load testing with QA Wizard Pro QA Wizard Pro is a tool to perform load testing of web applications and automate functional and regression testing of Windows, web, and Java applications. Integration with TestTrack’s test case management and defect management features further streamlines testing, automates defect entry, and facilitates reporting. Gain cross project visibility with Seapine ALM RP Seapine ALM Reporting Platform (Seapine ALM RP) gives you and your management team insight into every stage of the development lifecycle, enabling fact-based reviews of each project and team. Seapine ALM RP makes it possible to review the schedule status across active projects, investigate test plan performance across teams, analyse bug counts across release cycles, and so on, helping you stay on top of your quality initiatives.
Number of customers: 8500 A: 6960 Cintas Boulevard, Mason Ohio 45040 USA T: +1 513 754 1655 F: +1 513 754 1660 E: agara@seapine.com W: www.seapine.com United Kingdom, Ireland & Benelux A: Marimed House 4 Lindley Place Kew Surrey TW9 3PN T: +44 (0) 208 948 9460 F: +44 (0) 208 948 9462 E: salesuk@seapine.com
Whether your team uses waterfall, agile, or a hybrid methodology, Seapine’s solutions provide the most seamless way to manage product lifecycle processes available today.
Efficiently controlling and tracking change is
PAGE 52
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
20 LEADING TESTING PROVIDERS
SECFORCE Let SECFORCE help your CEO get a better night’s sleep!
CREST certification means you will always get independent advice
We at SECFORCE spend all day every day with one mission: helping you, your CEO and your clients to get a good peaceful night’s sleep by ensuring your IT infrastructure and applications are safe from malicious attacks.
As a vendor independent and CREST certified firm, we provide impartial security advice to all our clients. This means we only provide advice based on the interest of our clients, and not security software/hardware vendors.
A leading penetration testing consultancy
We love what we do
SECFORCE is a leading London based information security consultancy, highly specialised in the delivery of penetration testing services, driven by research and innovation. Our core services are:
SECFORCE security consultants are creative, well respected in the field and contribute to our clients’ success, not only with our broad technical knowledge, but also with our ability to translate complex technical risk into business terms. We really enjoy our jobs, and our clients like working with us too.
• Infrastructure penetration testing • Application penetration testing (web and mobile) We also contribute to the security industry regularly by developing security testing tools and sharing knowledge. Partnering with development teams to secure applications We work directly with development teams to review how secure their web and mobile applications are, and feed back on any security loopholes that could be exploited by internal or external malicious attackers.
A: G round floor, Aegon House, 13 Lanark Square, E14 9QD T:
+44 (0) 845 056 8694
W: www.secforce.com
ISO 90001 and 27001 As well as holding the coveted CREST accreditation, all of our processes and the delivery of our services are underpinned by ISO 9001 and 27001, giving our clients absolute confidence in the quality of our deliverables and peace of mind that their data is held in a secure manner.
SOGETI Sogeti UK is a leading provider of local professional services, with over 20 years of experience in delivering cuttingedge, structured testing solutions that help customers to achieve high quality, measurable and cost-effective results. In the UK we help our clients to benefit from our onshore and offshore testing solutions that include Test Process Improvement (TPI) assessments, functional and non-functional testing, performance testing, automation, agile development testing, managed testing services and mobile device and application testing. Our solutions and collaborative approach aim to increase speed to market, enhance software quality, mitigate risk and reduce costs. Together Capgemini and Sogeti have one of the largest dedicated testing practices in the world, with over 12,000 test professionals and 14,500 application specialists, and a common centre of excellence developed in India. We operate through an enlarged network of test centres and professional test resource units worldwide, with a combined capability of 12,000 test professionals and a further 14,500 application specialists with test experience, in Europe, US, India and Asia Pacific region. Our many test centres are fully equipped with state-of-the-art test processes, reusable test
frameworks and templates, test tools and test solutions. Our comprehensive portfolio of innovative business-driven quality assurance (QA) and Testing services, combine best-in-class testing methodologies (TMap and TPI) and a global delivery model, Rightshore, to help organisations achieve their testing and QA goals. Alongside our long-standing software testing services, we offer a range of non-testing solutions such as high-tech engineering and services that stem from our partnerships with IBM and Microsoft. These include business intelligence, analytics and Big Data, smarter commerce, social business and desktop migration.
A: 85 London Wall, London, EC2M 7AD T: +44 (0) 207 014 8900 E: enquiries.uk@sogeti.com W: www.uk.sogeti.com
Sogeti is a wholly-owned subsidiary of Capgemini S.A., a global leader in consulting, technology, outsourcing and local professional services, with more than 125,000 professionals in 44 countries. Capgemini S.A. is listed on the Paris Stock Exchange.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
PAGE 53
20 LEADING TESTING PROVIDERS
SQS The world’s leading specialist in software quality SQS is the world’s leading specialist in software quality. With over 30 years of experience, SQS has completed 7,000+ projects for organisations in banking and finance, energy, insurance, manufacturing, retail, logistics and telecommunications. Over 2,400 testing experts help clients to improve the quality of their business solutions, and we are expanding with plans to recruit 700 new roles internationally by the end of 2013. SQS Managed Services let you move away from headcount-based testing SQS introduced the concept of Managed Services to improve testing efficiency and move away from the more common headcount-based outsourcing of testing. SQS’ Managed Services offer highly scalable, flexible testing services combining output-based pricing, test automation, and offshoring/ nearshoring to reduce costs and speed up testing. Test Automation FaQtory – industrialised IT quality saves clients time and money Based in Pune, India, SQS’ Test Automation FaQtory combines best practice in testing with
lean manufacturing to offer a simpler and more reliable way of testing large, business-critical systems. Through automation, SQS increases test capacity without increasing the number of testers, and clients only pay for test cases delivered, executed and maintained. SQS PractiQ for consistency and best practice 30 years of experience have gone into creating SQS PractiQ, our industry leading, best-of-breed methodology. This provides or clients with confidence that SQS team members offer a standardised approach to quality.
A: 7-11 Moorgate, London, EC2R 6AF T:
+44 (0) 20 7448 4620
E: miisa.bergius@sqs.com W: www.sqs.com
SQS Academy – training for all testing roles SQS Academy offers a full range of testing training, new role-based courses and the 2012 Syllabus ISTQB Certified Tester: Advanced Level Test Manager training. SQS Academy’s training courses take place across the country at SQS’ offices in Edinburgh, Belfast, Leeds, Manchester, Birmingham and London. SQS also offers its full range of courses as private events. For SQS employees, it provides a structured framework for skills development. Whether you are looking for software quality services or a career in testing, please contact us.
TECHEXCEL TechExcel DevSuite provides an integrated platform for development and defect tracking, requirements management, project planning and quality management. With tools to effectively support both agile and traditional processes.
management. By dynamically linking your project timeline to implementation tracking in DevTrack, you achieve real-time visibility into the current project status, risk areas, burndowns and expected delivery dates.
Portfolio - Product Portfolio Management
DevTrack – Implementation and Defect Tracking
Product Portfolio Management tool provides a comprehensive and real-time view of all internal and external processes and resources to enhance the management practices for all of an organisation’s project and program portfolios. Integrating PPM across all DevSuite modules bridges the gaps between development teams communicating project status, managers reporting progress, and decision-makers quantifying projects with the greatest business value.
DevTrack comprehensively tracks and manages all aspects of a development project, from feature stories and implementation tasks to product defects and change requests. DevTrack can easily be configured to fit your needs.
DevSpec – Requirements Management DevSpec enables organisation-wide collaboration for the management of requirements, specifications, product ideas or agile stories, with integration points that allow you to drive development and testing directly from completed requirements. Whether you wish to adopt a simple agile process for building a product backlog or need to implement a defined and regulated requirements process, you will quickly achieve full traceability throughout the project lifecycle. DevPlan – Development Project Planning A better alternative to traditional project management tools, DevPlan provides robust development project planning and resource
PAGE 54
DevTest – QA Test Management
A: Crown House, 72 Hammersmith Road, London, W14 8TH T: +44 (0) 207 470 5650 F: +44 (0) 207 470 5651 E: emeainfo@techexcel.com W: www.techexcel.com
From test case creation, planning and execution through defect submission and resolution, DevTest manages the complete quality lifecycle. Implement quality processes earlier in the development lifecycle to manage shorter deadlines, address complex contemporary testing challenges, and improve your deliverable software. KnowledgeWise – Knowledge Base DevSuite includes KnowledgeWise, a central knowledge repository for managing documents, images, Wiki articles, Wiki books, and other digital assets and attachments. DevTime – Timesheet Management Adding the optional DevTime module to DevSuite creates a complete time sheet management system that is integrated with development time tracking in DevTrack.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
20 LEADING TESTING PROVIDERS
TEST PLANT TestPlant is an international software business based in London. It has development centers in the USA and the UK with sales and support centers in the USA, the UK and Asia as well as an extensive network of business partners. The company was founded in 2008 by George Mackintosh and Jon Richards with venture capital backing and it has shown rapid revenue growth organically and through the acquisitions of Redstone Software and Facilita. TestPlant's products are used in over 30 countries by well over 300 enterprise customers in sectors which include financial services, automotive, healthcare and life sciences, media and entertainment, retail and defense and aerospace. The eggPlant range is a set of tools which supports the design, development, test and management of software applications for mainframe, desktop and mobile use
in any technology platform environment. The tools are relevant in agile, mobile, web and DevOps deployments. eggPlant use improves and reports on the quality and responsiveness of software systems, reduces time to market and lowers costs by introducing process automation. The eggPlant tools have been granted patents with several applications pending. The company's achievements have been recognised within the software industry and by the venture capital community. TestPlant is also a Red Herring Global Top 100 company and was awarded a Queen's Award for Enterprise in 2013.
A: 6 Snow Hill, London, EC1A 2AY T: F:
+44 (0) 207 002 7888 +44 (0) 207 002 7887
E: milli.dyvoy@testplant.com W: www.testplant.com
TESTING SOLUTIONS GROUP Testing Solutions Group specialises in assuring successful outcomes for business critical programmes, that provide businesses with confidence in their testing programmes;
Why TSG?
• A recognised innovator of change to improve capability, time to market and quality of systems delivered to provide solutions for business.
TSG has been recognised as a point of excellence over the last 12 years, for many blue chip companies such as IBM, Fujitsu, Bank of England, FCA/FSA, The Royal Bank of Canada, Linklaters LLP and countless others in the UK. We have also supported client programmes in the USA, Hong Kong, Hungary, Portugal and Australia. Talk to us and we’ll help put together the right level of assurance you need to mitigate risk and gain confidence in what you do and generate a return on your investment.
• A recognised industry leader though our work with ISO 29119, the ISEB, ISTQB & iSQI testing certification bodies and at conferences worldwide. Our services include: • Vendor Management / Evaluation > Test Resource Provision
For us, it’s about business assurance and outcomes, not just testing.
A: 117-119 Houndsditch, London, EC3A 7BT T: +44 (0) 207 469 1500 E: testmagazine@testing-solutions.com W: www.testing-solutions.com
• Strategic Consultancy Risk – ISO 29119 > Performance Testing • Q uality Assurance Frameworks > Agile Testing • Test Tool Evaluation >Automation Testing • Specializing in Banking, Finance and Legal sectors > Certified Training
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
PAGE 55
20 LEADING TESTING PROVIDERS
THE TEST PEOPLE The Test People Limited (TTP) is an innovative testing solutions and consulting firm providing performance engineering, automation, test strategy and managed test services. TTP take a highly technical and innovative approach to testing, ranging from utilising the latest toolsets to developing bespoke testing solutions for complex test, performance and automation challenges. At the forefront of testing new technologies and utilising the cloud, TTP provides the most flexible and value-add test service available in the market today. From day one, The Test People has invested in an in-house R&D capability to solve testing problems caused by technology and to innovate to create solutions to testing challenges that had hitherto been addressed either by headcount or had been left untested because the technology was deemed untestable using current methods and tools. These solutions include our True Performance Test solution where true end-to-end repeatable performance tests can be performed, including the performance of the client. We have also developed our Optical Character Recognition (OCR) automation, this allows the automation of any technology, including Flash and Flex applications without the need to manage a build for test automation purposes and a build for production.
Founded in 2007, the company now has a headcount of more than 100. As the founders of The Test People all had extensive experience of retail operations, financial markets trading and egaming trading the business was launched to focus on marketing its services to these industry verticals. This focus still remains, however, The Test People has acquired numerous clients in other sectors. Clients include: William Hill, ICAP, Hitachi Capital Uk, Gain Capital, NHS Choices and Open Bet.
A: Albion Court, 5 Albion Place, Leeds, West Yorkshire, LS1 6JL T: +44 (0) 113 834 5975 E: contactus@thetestpeople.com W: www.thetestpeople.com
The business now has offices in both London and Leeds and has recently opened it’s Gibraltar headquarters to support it’s growing business in the bookmaking/egaming market. To support the growth plans for the business, TTP has invested in the creation The TTP Academy, which has been specifically built to train, accredit and deliver the testers of tomorrow. The academy, lead by world-class trainers with a wealth of experience and knowledge, enables TTP to provide clients with motivated, bright, young career focused people, who have mastered the technical and analytical skills required to be productive testers.
TRICENTIS Tricentis provides the leading enterprise software testing solutions for Global 2000 companies to accelerate business innovation with peace of mind. The company’s more than 400 customers include global brands such as Allianz, BMW, Deutsche Bank, HBO, Orange, Siemens, Southwest, Swiss Re, and UBS. Tricentis has the highest customer renewal rate in the industry, at more than 98 %, and its industry awards include Best in Test 2013. Gartner recognises Tricentis as a visionary in the recent 2013 Magic Quadrant for Integrated Software Quality Suites. A fast growing, privately held company, Tricentis has offices in Austria, Germany, Switzerland, the UK, the Netherlands, the United States, Australia, and Singapore. Tricentis benefits Innovate faster with peace of mind – Tricentis technology identifies the most critical areas of your software, enabling business managers to quickly assess risk and manage release schedules with more accuracy, visibility, and control. Optimize cost and efficiency – Tricentis enables companies to avoid the maintenance trap in software test automation, delivering automation rates as a high as 90%. Reduce business-related risks – Tricentis’ riskbased software testing approach enables companies to align software testing with business needs in a systematic way.
PAGE 56
Tricentis uniquely delivers Unprecedented automation rates – Tricentis helps enterprises achieve up to 90% test automation, speeding time to market while enabling teams to focus on business innovation. Strategic staff allocation – Tricentis enables companies to distribute testing efforts across a wider team and frees technical specialists to focus on highest-value initiatives.
A: Leonard-Bernstein-Strasse 20, 1220 Vienna, Austria T: +43 (0)1 263 24 09 E: office@tricentis.com W: www.tricentis.com
Minimized business risk – Tricentis enables companies to manage risk intelligently by prioritizing tests with the highest business value. Leverage of existing testing assets – Tricentis helps companies protect and reuse testing assets, to leverage investments in tools, IT, training, and effort. Simplified business decision making – Tricentis gives managers an easy way to make more informed business decisions based on factors such as risk coverage, time-tomarket, and resources. Technology change protection – Tricentis enables companies to leverage and extend existing software testing investments with minimal disruption to processes as new technologies emerge.
OCTOBER 2013 | WWW.TESTMAGAZINE.CO.UK
Event Loading...
London Autumn 2013
Transforming Testing: Identifying Solutions that Work This highly anticipated event now returns to London on 24th October 2013. Hurry – last few weeks to register! Just visit: testexpo.co.uk