Impact Magazine Autumn 2015 by Impact Magazine from The Operational Research Society

D R I V I N G I M P R O V E M E N T W I T H O P E R AT I O N A L R E S E A R C H A N D D E C I S I O N A N A LY T I C S

AUTUMN 2015

SPORTING TIMES ( A N D P L AC E S )

Image ÂŠ skynesher / iStock

Analytical approaches help sports administrators timetable competitions

FA S H I O N I N G A MODEL FOR ZARA

OPTIMIZING DESK C U S TO M E R R E L AT I O N S S E R V I C E S AT H E R A

How Zara worked with analytical

Algorithms enabled an Italian

experts to determine stock

utility company to deal with a

levels for new products

significant increase in demand

Make optimized decisions. Even if your data is incomplete. Robust optimization at both the solver and modeling level. Now part of FICO® Xpress Optimization Suite. Missing data and the challenges of harnessing Big Data have introduced a lot of uncertainty into the process of solving complex optimization problems. FICO has solved that problem. We’ve enhanced FICO® :RTGUU 1RVKOK\CVKQP 5WKVG YKVJ HGCVWTGU VQ JCPFNG VJG FKHƒEWNV[ QH WPEGTVCKPV[ KPVTQFWEGF by predictive analytics data. This robust optimization guarantees feasible solutions in the face of unknowns.

Learn more about the Xpress Optimization Suite: ƒEQ EQO ZRTGUU

E D I TO R I A L Judgement on a new magazine will often wait until the second issue is published. It will be expected that a great deal of effort is put into the first issue, and readers may wonder if the quality can be maintained. Those that take that view will now have the chance to judge as the second issue of Impact is now in your hands or on your screen! Of course, some didn’t wait for the second issue. I’m grateful to those who commented positively, and to those who made helpful suggestions. This issue introduces a new type of article: one that demonstrates the usefulness of one analytical approach or another. Here Mark Elder focuses on simulation, and his message is exemplified in the story of how simulation contributed to a significant, and vitally important, reduction in time to inform women of cervical cancer screening results.

The OR Society is the trading name of the Operational Research Society, which is a registered charity and a company limited by guarantee.

Seymour House, 12 Edward Street, Birmingham, B1 2RX, UK Tel: + 44 (0)121 233 9300, Fax: + 44 (0)121 233 0321 Email: email@theorsociety.com Secretary and General Manager: Gavin Blackett

One theme in this issue can be seen in stories of how academics have worked with organisations to achieve a significant impact on their performance. Jérémie Gallien and colleagues worked with Zara who wanted to determine how much of a new product to stock in each store. Zara’s CFO said that the project had made a cultural impact on Zara. When Italian multi-utility company Hera were faced with a significant increase in customer calls, Daniele Vigo and colleagues developed algorithms that allowed staff numbers and service quality to remain unchanged, while reducing customers’ mean waiting time. I hope you enjoy reading this issue and that sports fans will find time to do so. At least rugby fans should have more time available! As I said in my first editorial, if you have suitable stories to tell please let me know. I’m grateful for the responses I received to that request, the fruits of which will be seen in future issues. For those of you who are just becoming aware of O.R., the Operational Research Society will be pleased to advise you if you think that such approaches will be helpful to your organisation.

President: Stewart Robinson (Loughborough University) Editor: Graham Rand (Lancaster University) g.rand@lancaster.ac.uk

Print ISSN: 2058-802X Online ISSN: 2058-8038 Copyright © 2015 Operational Research Society Ltd Published by Palgrave Macmillan Printed by Latimer Trend This issue is now available at: www.issuu.com/orsimpact

Graham Rand

OPERATIONAL RESEARCH AND DECISION ANALYTICS

Operational Research (O.R.) is the discipline of applying appropriate analytical methods to help those who run organisations make better decisions. It’s a ‘real world’ discipline with a focus on improving the complex systems and processes that underpin everyone’s daily life - O.R. is an improvement science. For over 70 years, O.R. has focussed on supporting decision making in a wide range of organisations. It is a major contributor to the development of decision analytics, which has come to prominence because of the availability of big data. Work under the O.R. label continues, though some prefer names such as business analysis, decision analysis, analytics or management science. Whatever the name, O.R. analysts seek to work in partnership with managers and decision makers to achieve desirable outcomes that are informed and evidence-based. As the world has become more complex, problems tougher to solve using gut-feel alone, and computers become increasingly powerful, O.R. continues to develop new techniques to guide decision making. The methods used are typically quantitative, tempered with problem structuring methods to resolve problems that have multiple stakeholders and conflicting objectives. Impact aims to encourage further use of O.R. by demonstrating the value of these techniques in every kind of organisation – large and small, private and public, for-profit and not-for-profit. To find out more about how decision analytics could help your organisation make more informed decisions see www.scienceofbetter.co.uk. O.R. is the ‘science of better’.

Providing OR and Analytics Recruitment Solutions for 40 Years Our Markets With our roots going back 40 years to the original Operational Research and Management Sciences arenas, we understand quantitative analytics like no other recruiter. Whether your recruitment need is for fundamental RSWLPLVDWLRQ H[SHUWLVH HQDEOLQJ UREXVW GHFLVLRQ PDNLQJ LQ DQ XQFHUWDLQ ZRUOG ZLWK LQÂżQLWH YDULDEOHV DQG FRQVWUDLQWV IRUHFDVWLQJ RU SULFLQJ DQDO\WLFV WR IHHG LQWR D \LHOG PDQDJHPHQW V\VWHP ÂżQDQFLDO PRGHOOLQJ WR support transactions services projects; discrete event or system dynamics simulation for process or strategy decision support; customer insight or marketing analytics to underpin product strategy; or risk management in a consumer or business environment - we are well versed in the subtleties and Business Modelling Simulation distinctions which weave through this complex arena. Optimisation Financial Modelling

Our Model Understanding the required blend of technical, business and interpersonal skills to conform to a particular recruitment need, we are able to apply our unique knowledge of industry trends, evolving technology and prevailing market conditions.

Econometrics & Forecasting Revenue/Yield Management Pricing Analytics

Interpersonal Skills

Business Awareness

Process Modelling & Business Transformation

Applying Specialist Market Knowledge Since 1975

Data Science Customer Insight Marketing Analysis

Technical Ability

Credit & Risk Credit Scoring Behavoural Scoring

Data Warehousing Decision Support Systems BI / MI

No other recruitment company has our blend of specialist market knowledge, built up over the last 4 decades. We believe in forging long term relationships built on trust, positive experience and mutual respect. We place great emphasis on understanding both our clientsâ&#x20AC;&#x2122; and candidatesâ&#x20AC;&#x2122; individual needs, enabling us to provide genuine added value, as the basis for long term business partnerships.

Whether you are a client seeking to hire, or a candidate seeking a career move, contact Teresa Cheeseman, Kate Fuller or Mark Chapman for an informal discussion. 7HO Â&#x2021; ( 0DLO RU#SURVSHFW UHF FR XN :HE ZZZ SURVSHFW UHF FR XN

CONTENTS 7

FASHIONING A MODEL FOR ZARA

Brian Clegg reports how Zara have worked with analytics academics to produce a system for determining the initial stock levels at each store when a new product is introduced

18 Making an impact - O.R. in

healthcare Mike Pidd reflects on analytics and healthcare

OPTIMIZING DESK CUSTOMER RELATIONS SERVICES AT HERA Daniele Vigo shows how algorithms developed for a large Italian multi-utility company have enabled staff numbers and the service quality to remain unchanged, while reducing the mean waiting time for customers, in the face of a significant increase in demand

20 Using Simulation – you know it

makes sense! Mark Elder explains why simulation is such a useful tool

NHS CERVICAL SCREENING Hazel Squires and John Ranyard tell how the O.R. community contributed to a dramatic cut in the time taken to deliver cervical cancer screening results

26 Universities making an impact

LLAMASOFT – SUPPLY CHAIN BY DESIGN

31 Sporting times (and places)

Brief reports of two postgraduate student projects

Vicky Forman tells how LLamasoft goes about helping organisations design their supply chain

Mike Wright reflects on issues involved in scheduling sports competitions

DEVELOPING STRATEGY IN THE PUBLIC SECTOR Neil Robinson shows how Max Moullin has changed the approach to performance management to one founded on inclusiveness, cooperation and understanding

Seen elsewhere Analytics making an impact

47 Wrong numb3rs

Geoff Royston is concerned about the low numeracy level in the UK

ALPHA MAIL Neil Robinson tells the story of how O.R. transformed German firm Rhenania BuchVersand in the face of opposition from the top management

INTRODUCING NEIL ROBINSON

Impact is using the services of several professional writers. Neil Robinson is a managing editor at Bulletin, a strategic communications consultancy that works with academic institutions to help their research have the greatest possible economic, social or cultural impact. He studied English Literature at the University of Liverpool but left after three days to embark on a career in journalism. Before joining Bulletin he spent more than 20 years working as a reporter, sub-editor and editor, writing for national newspapers, TV, radio and a wide range of specialist and consumer magazines. He has worked with universities across the UK and in Europe and Australia, “translating” and disseminating research, book editing and delivering Research Excellence Framework consultancy. Visit www.bulletin.co.uk.

SEEN ELSEWHERE A HEALTHIER WALES

A team under the leadership of Professor Paul Harper has won a Cardiff University Innovation and Impact Award, for helping NHS Wales to better match capacity to demand, and to reconfigure services to help reduce waiting times and improve patient outcomes, by analytical modelling. The team is funded by and embedded within the Aneurin Bevan University Health Board (ABUHB) and employs four research associates. The team’s work is the first known case of an embedded O.R. modelling group in the NHS creating an ongoing dialogue between modellers, clinicians and managers, encouraging them to engage, innovate, test alternatives, and take on leadership roles.

DANNY ANTEBI AND PAUL HARPER

Professor Harper, Director of Health Modelling Centre Cymru and Deputy Head of the School of Mathematics, said, “We are delighted to receive recognition for our innovative work to help NHS Wales design and deliver prudent healthcare services to ensure resources are used to maximum eﬀect; this is a challenging yet vital task.”

IMPACT © THE OR SOCIETY

Judith Paget, ABUHB Chief Executive noted, “The project’s success has led to better planning for the organisation and better analysis: much better decisions are made as a result of the input of the modellers.” As a direct result of the collaboration, more than 250 NHS Wales’ staff have attended training in statistical and O.R. modelling techniques. Further collaborative benefits include a joint lectureship in O.R., the hosting of MSc O.R. student research projects, match funded PhD studentships, and successful research grant submissions. The unit has also attracted interest from the wider NHS and Welsh Government, and media coverage including the BBC news website and featuring on the BBC Radio 4 PM programme. Dr Danny Antebi, Director of Aneurin Bevan Continuous Improvement Centre, said, “This work has provided a robust and scientific evidence-base which is being applied to ABUHB’s major new acute hospital planned for 2018. Our work will help to ensure millions of pounds of the Board’s annual budget are best spent for generations to come.” GOOD GROWTH THROUGH ADVANCED ANALYTICS

INFORMS’ Edelman Award was won in 2015 by Syngenta, the global Swissbased agribusiness that makes and markets crop seeds and agrochemicals. Their cross-breeding soybean project resulted in up to 20% more productive crops, using O.R. tools to support one or more key “pipeline” phases: variety design, variety development and variety evaluation. A trillion design

options were evaluated to produce the greatest genetic gain for soybeans while comparing trait introgression and breeding strategies. The new analytical tools dramatically improve project lead training, decision-making and planning, resulting in cost avoidance for soybean R&D of more than $287 million from 2012-2016 and substantially improving the probability of successfully delivering a portfolio value exceeding $1.5 billion.

Syngenta recognizes the positive impact these tools have on soybean R&D and is initiating a multi-year effort to customize and launch similar tools across all major crops. The data-based transformation helps Syngenta achieve its commitment to meeting the world’s growing food needs in an economically and environmentally sustainable way. Dan Dyer (head of seed product development) said that, when it comes to O.R. and agriculture, the best is still to come. “This marks a new frontier for our industry. We are pioneers in using O.R. in the area of genetics development, but there are many other areas to explore. Climate and weather prediction, for example, are big issues for agriculture. We’re at an exploratory stage of figuring out how to really benefit from all the powers of analytics and data in agriculture.” The winning presentation is available at https://www.pathlms.com/ informs/events/352/thumbnail_video_ presentations/10893

IBM’S WATSON ANALYTICS BREWS UP BUSINESS FOR HONEST CAFÉ

Honest Café, a new venture from Revive Vending, is using Watson Analytics to discover client insights to help determine everything from coffee products and pricing, to marketing and promotions. The three cafés currently in operation in London, and four more planned, are all unmanned. The company deploys and positions high-end vending machines at the cafés, which offer a mix of healthconscious snacks, locally sourced and low-calorie organic options, juices, fruit teas, flavoured popcorn, baked vegetable snacks and Fair Trade hot drinks. Although the absence of staff can mean greater efficiencies and lower overhead costs, it makes understanding clients’ needs and desires a little more challenging. “Because Honest is unmanned, it’s tough going observing what our customers are saying and doing in our cafes,” says Mark Summerill, head of product development at Honest Café. “We don’t know what our customers are into or what they look like. And as a start-up it’s crucial we know what sells and what areas we should push into.”

What Honest Café did have was data. All those vending machines were capturing important information, from product sales and selections, to the timing of purchases and more. But even then, the company realized

it was not getting the value out of its data. “We lacked an effective way of analyzing it,” Summerill says. “We don’t have dedicated people on this and the data is sometimes hard to pull together to form a picture.” So the company turned to Watson Analytics, IBM’s breakthrough natural language-based analytics service and quickly began discovering insights. “We identified that people who buy as a social experience have different reasons than those who dash in and out grabbing just the one drink,” Summerill says. “They also have a different payment method, the time of the day differs and the day of the week. Knowing this, we can now correctly time promotions and give them an offer or introduce a product that is actually relevant to them.” Through IBM’s relationship with Twitter, Watson Analytics also enables customers to analyze social sentiment for insights around programs, products, trends and more. Watson Analytics Personal edition offers access to 25,000 tweets per data set. “Having direct access to Twitter data for insights into what our buyers are talking about is going to help augment our view and understanding of our customers even further,” Summerill says. “Watson Analytics could help us make sure we offer the right customers the right drink, possibly their favourite drink,” Summerill says. “I’m no analyst and data sometimes leaves me feeling cold, but Watson Analytics has helped me understand the data and the story we can tell. We can be focused on what really matters rather than what we think might matter.” UPS’S ORION IS HUGE O.R. DEPLOYMENT

UPS has invested heavily in a new project, ORION (On-Road Integrated Optimization and Navigation), that

will shorten drivers’ routes and save millions on fuel. Wall Street Journal reported in February that management and IT expert Thomas H. Davenport believes ORION is the largest deployment of O.R., and that UPS spent $200 million to $300 million to develop it, excluding many years of investments in underlying driver technology and communications infrastructure. UPS has 500 staﬀ working on its deployment.

The backbone of a logistics company is still its drivers, but even veteran drivers with years on the same route can’t compete with technology when it comes to finding the fastest, most fuel-efficient way to get every package to a customer’s door. “Today we rely extensively on our drivers’ knowledge on how a route should be delivered, and drivers create their own perceptions about the most efficient way of taking a route,” says Juan Perez, vice president of information services. “We’ve challenged our drivers to beat ORION, and we’ve found that the technology happens to be much better.” ORION does that through a combination of connected car-like telematics and a lot of data crunching of package information, user preferences and routes. Every parcel that UPS delivers contains extractable data on time of shipment and how it matched delivery commitments. But ORION also scans map data and historical GPS

IMPACT | AUTUMN 2015

tracking of similar routes to come up with a solution. The software has 250 million address data points to access and runs on an algorithm Perez says is the equivalent of 1,000 pages of code. Each individual route has an average of 200,000 possible ways to go. UPS saved 3 million gallons of fuel during its testing of the program from 2010-2012 and says it’ll reduce its consumption by another 1.5 million gallons this year. Once the program is rolled out to every driver by 2017, the company says it can save $50 million by taking just one mile oﬀ each of its driver’s daily routes. There’s an environmental beneﬁt to reducing fuel costs. The 1.5 million gallons saved this year will also cut 14,000 metric tons of carbon dioxide emissions. ADVANCED ANALYTICS NEEDED TO COMPETE IN THE DIGITALIZED MARKETPLACE

Retailers will need advanced analytic capabilities to be able to compete in the digitalized marketplace, according to Gartner, Inc., and should look to invest in advanced analytics providers that specialize in retail solutions. Gartner defines advanced analytics as the analysis of all kinds of data using sophisticated quantitative methods to produce insights that traditional approaches to business intelligence – such as query and reporting – are unlikely to discover. These advanced analytics tools enable deeper insights and discovery that will challenge business assumptions. They also put information in the hands of business analysts and business users and offer significant potential to create business value and competitive advantage. Robert Hetu, research director at Gartner says “As the Internet of

IMPACT | AUTUMN 2015

Things (IoT) continues to expand over the next five years, the effects on multichannel retailers will be more disruptive than anything seen to date and will require advanced analytics capabilities to cope with this disruption.” A customer’s refrigerator that can sense the need to replace its water filter is an example of the retailer’s need to compete and the associated need for advanced analytics. “While today the retailer may feel confident in that customer’s repeat purchase – perhaps because the customer acquired the appliance from the store or is comfortable locating the filter in the physical store – the refrigerator, seeking the replacement on its own, has neither loyalty to the retailer nor concern about finding the item,” explains Hetu. “As a result, the ‘thing’ (in this case the refrigerator) will seek the best combination of price and availability. This change will require the retailer to evaluate the potential transaction in an instant and determine a course of action to save the sale or allow it to pass to others.” The impact of digital business and the IoT will require advanced analytics to support real-time decision-making to take advantage of momentary business opportunities. These opportunities will require the retailer to be able to decide in a split second if an opportunity is potentially desirable or should be passed up in favour of the next momentary opportunity. Success will require retailers to use a combination of knowledge, innovation, speed and strategy to maintain and grow market share in the digital economy. More detail is available in the report “Retailers Find Success Using Self-Service and Advanced Analytics” at https://

www.gartner.com/doc/3037224/ retailers-success-using-selfserviceadvanced?srcId=1-2994690285 SWOT BEGS FOR BIG DATA

Kevin O’Marah (Chief Content Oﬃcer, SCM World) says, “One of the classic business strategy tools is the SWOT analysis – strengths, weaknesses, opportunities and threats.

As I have experienced it through the years, it’s always been a conference room exercise with people sitting around a table scrawling their preexisting biases on whiteboards in an orgy of groupthink. Wouldn’t it be nice if the same four pillars of this analysis were built on facts and applied constantly to supply chain risk?” ONE COMPLAINT LEADS TO ANOTHER

One ﬁnding of work published in the INFORMS journal Marketing Science, “The Squeaky Wheel Gets the Grease – An Empirical Analysis of Customer Voice and Firm Intervention on Twitter,” is that responding to complaints on social media will trigger new complaints. “[Some] people complain on Twitter not just to vent their frustration; they do that also in the hope of getting the company’s attention. Once they know the company is paying attention, they are more ready to complain the next time around.” The full paper can be found at http://dx.doi.org/10.1287/ mksc.2015.0912`

Image © Inditex

FASHIONING A MODEL FOR ZARA BRIAN CLEGG

THE SPANISH HIGH STREET FASHION CHAIN ZARA does not make life easy for itself. Each year around 8,000 new stock lines are sent out to its stores, most of which items will only have a very short shelf life. This means that the initial shipment can make up as much as 50 per cent of the entire stock for a particular product, so getting those shipment levels right can be make or break for the company. Zara staﬀ have

been working with O.R. experts from the London Business School (LBS), University of North Carolina (UNC) and Boston Consulting Group to put together a system to support these crucial decisions. A key individual involved with Operational Research developments in Zara over the last ten years is Jérémie Gallien, a native of France who has lived out of the country since 1996,

IMPACT © THE OR SOCIETY

IMPACT | AUTUMN 2015

making. Furthermore, that culture was fairly entrenched because Zara did have a history of success in showing that these subjective or non-quantitative approaches can pay oﬀ when applied to many of the key decisions it faces, such as design of clothes and store environments, hiring of designers, managing high-level relationships with suppliers and managing brand value.”

BRINGING TECHNOLOGY TO BEAR However, there were problems that Zara faced that did involve large amounts of quantitative data, with a short time in which to make decisions—not the ideal situation to apply human intuition. These included initial merchandise shipments to stores, store inventory replenishment, clearance pricing and supplier purchase quantities. These decisions were sometimes in the hands of a small number of experts, who could cause the company signiﬁcant problems should they leave. To make matters worse, as Zara was growing rapidly, the sheer volume of data involved was in danger of overwhelming the decision-makers. Thanks to the foresight of Zara management, Gallien was able

there were problems that Zara faced that did involve large amounts of quantitative data, with a short time in which to make decisions

This has been a fantastic career experience for me, in part because these teams have always managed to work together in a way that was productive and impactful from both industrial and academic standpoints. I believe we have succeeded to convincingly establish the value of O.R. and business analytics within Zara’s environment.”

THE FIRST MODEL The ﬁrst model to be constructed, back in 2005/6, dealt with replenishment of stock in stores, a problem that was made more complicated by a set of rules that Zara employed on whether or not stock was displayed in the shops (and hence available for sale). If, for example, a garment came in small, medium and large, it was typically only displayed if medium garments

Image ©London Business School

first in the US and now the UK. With an Engineering Degree from the École des Mines de Paris and a PhD in Operations Research from the Massachusetts Institute of Technology (MIT), he is now an Associate Professor in the Management Science and Operations area at LBS. He has specialised in supply chain analytics working with commercial giants such as Amazon, Dell, Zara and IBM. As an academic who has studied the potential benefits of O.R., Gallien identifies two contrasting areas of decision-making. Some involve a complex, experience-based decision driven mostly by qualitative information. These decisions, which typically are taken infrequently, still benefit from being made by human experts. Other types of decision are made frequently, where there is a large amount of quantitative information and where the decision process is capable of high automation – here O.R. can deliver significant benefit. Gallien notes: “When I started to interact with Zara in 2005, it was a very successful company with a culture that strongly favoured human intuition, vision, and judgment (as opposed to analytical methods) for decision

to bring O.R. to bear on some of these issues: “I have been incredibly fortunate to work continuously over ten years with several amazing teams of smart and hard-working individuals, including Zara/Inditex (Zara’s parent company) employees, academic collaborators (such as Felipe Caro from University of California, Los Angeles and Adam Mersereau from UNC) and Masters students (such as Juan Correa, Rodolfo Carboni and Andrés Garro), with the goal of leveraging modern tools and concepts in O.R. and IT systems to improve all these key data-intensive operational decision processes at Zara.

Image © Inditex

were still available. A system was constructed to compute the shipments required based on requests from stores and historical demand forecasts.

What should the initial stock levels at each store be when a new product was introduced?

multiplied them up by an estimate of the time the new product would be on sale – a so-called “coverage factor”. This provided a starting number for each store, which would then be tweaked to reﬂect the expectations that Zara’s experts had of potential sales. Not all the products would be allocated though. A percentage of total production was held back for restocking. The remainder would be allocated from the largest order

downwards until stock ran out – at this point, the remaining stores would not receive the product at all. This process had worked well enough when Zara was primarily a single country brand, but had come increasingly under pressure as the number of outlets grew, taking in countries across the world. This made it diﬃcult to be sure how robust the subjective assessments of potential demand were.

As a result it proved possible to increase revenue while keeping existing warehouse staﬃng levels, despite increasing demand. Zara Distribution Manager, Marcos Montes: “Prior to the introduction of the new distribution system, we thought that we had to increase the staﬃng level of the distribution teams. However, upon the implementation of the new system, we realised that we could have a stable structure in accordance with the growth of our company. The big change consisted in going from a fully manual process to one highly automated based on systematic data analysis.” The O.R. team,with a changing roster of academic members, went on to address other data-intensive operational decision challenges facing Zara over a period of several years, including clearance pricing and purchase optimisation. This latest challenge, however, was possibly the trickiest of the OR-susceptible decisions facing Zara management. What should the initial stock levels at each store be when a new product was introduced?

THE INITIAL STOCK PROBLEM Historically, Zara’s decision makers took one or more existing products that were considered comparable to the new one, calculated daily averages, then

IMPACT | AUTUMN 2015

The new system starts as before with comparable previous products, selected by Zara’s buyers, but then goes through a two stage process to recommend initial stock levels store by store. The ﬁrst step is to take these sales ﬁgures for existing products and to forecast sales for the initial week that the new products are available, on an individual store-by-store basis. This part of the model produces a probability distribution of demand by week for each store. Th is forecast cannot be purely based on historical data, because the sales systems don’t provide information on lost sales, which occur when a

IMPACT | AUTUMN 2015

product is sold out, or not provided to a speciﬁc store. Also the system has to take into account the variations in sales through the week and how the demand is distributed across the various garment sizes. As it turned out, these variations would provide the analysts with the ideal approach to estimate lost sales. A range of techniques to predicting lost sales were tested by taking data on 10,000 items that had stayed in stock and using the forecasting approaches to predict what sales should have been like on days that were randomly omitted from the data. The best forecasts came from taking an average of two key predictions. One approach

was to factor up the sales when the store went out of stock, matching the pattern of sales when a garment was in stock. In the second method, sales on garment sizes that had remained in stock were used to predict what the demand would have been for products that went out of stock, had they not run out.

The best forecasts came from taking an average of two key predictions.

A rather different approach had to be taken when estimating the potential demand in stores that never had the historical stock that was being used to predict demand for new items. In these cases, groups of comparable stores in the same geographical region were used to give predictions of expected demand, had the products been in store. With these techniques available to fi ll in the gaps, the model produces forecasts for historical sales, plus an error distribution – a statistical tool to predict what the spread of errors in estimation were likely to be – for the ratio of the actual historical sales to the forecast value. Th is error distribution is finally applied to the demand values forecasts for new products, so that at the end of this stage there is a forecast distribution by store for the items that will be sent out from the warehouses. These detailed demand forecast distributions are then passed into an optimisation module which first decides on the number of each product type that should be shipped in the initial distribution and then breaks down the numbers by size of garment, so that Zara is presented

Image © Inditex

SYSTEM STRUCTURE

Image © Inditex

with a requirement for each individual stock-keeping unit. An important model feature is to anticipate how much learning there is going to be based on observing the ﬁrst days of sales in each store, which drives the stock amount to be left in the warehouse after the initial shipment for replenishment purposes. Because transhipments between stores are typically more expensive than direct deliveries from the warehouse, it is important to keep the right quantity of stock at the warehouse upfront to leverage the better understanding of the geographic distribution of demand across stores that develops after a few days of sales. Fine-tuning is also applied by feeding back actual demand into the system from the ﬁrst three days of sales, enabling it to calculate how the subsequent second shipment should be optimised.

STRIKING A BALANCE This is a complex model and Gallien recognizes that there is a balance to be struck between complexity and usability: “In practice, finding that right balance for a specific industrial project is as much an art as a science, and I have personally found that the level of past experience with industrial problems makes a substantial difference. Another important factor here is that the ‘right’ model structure and level of detail for a given project is very much a relative concept which has rapidly changed in the recent past with advances in optimisation software components and hardware. In addition, the right model design could also be driven by the skill level and time constraints of the implementing organisation.” In the end, Albert Einstein’s maxim “as simple as possible, but no simpler,” proved effective as Gallien’s guide.

The computer system, which extracts data from Zara’s existing multi-computer SQL database, was developed by summer 2010 and underwent a year and half ’s testing, simulating new products using historical data. It was then given a pilot run using 34 new articles during 2012 to provide a live trial and look for opportunities to improve its eﬀectiveness. The prime measures of eﬀectiveness that Zara required were impact on sales, the proportion of stock sold before a product line went into clearance and the percentage of demand that was met before going into clearance.

STRESS TESTING To test the eﬀectiveness of the system, the team organised a pilot experiment which was essentially a randomised control trial. Although common when testing medicines and medical products or procedures, this type of experiment is much rarer in industry. To achieve this, the Zara stores were split into two groups, which randomly alternated as test and control group for diﬀerent products. When in the test

group, the new system was used, while the control group used the existing mechanism for deciding initial stock levels. Like any live experiment, there were some issues, notably that the shipment decisions could be modified by country managers, but their influence was unbiased because they did not know which were the test and which the control stores. Other issues arose around the way that the products to be tested were selected and that stores were split into the two groups. Significant lessons were learnt from a summer run which did not produce statistically significant results. In the end, the team was able to overcome several practical constraints linked with IT systems, physical infrastructure and organisational structure in order to generate statistically significant results the following autumn. The result was a more than 2% increase in sales across the season, which primarily was due to increased early shipments, though there was improvement through the cycle. At the same time, the percentage of units unsold before a product was reduced in

IMPACT | AUTUMN 2015

price for clearance was down 4.3%. The size of shipments to individual stores went down, but the number of stores receiving products went up considerably. These percentage improvements may seem small, but it was estimated that net proﬁt overall would have been increased by €150M in 2012 had the system been fully implemented, equivalent to the total contribution of roughly 140 stores. Interestingly, the country managers made signiﬁcantly greater adjustments to the stock numbers suggested by the existing approach than those from the new system, suggesting that the more sophisticated system was getting closer to the expert opinion on what was needed to meet local requirements. The system, along with others developed in the Zara O.R. projects, is still being used on a daily basis.

All our shipment decisions are now supported by O.R.

Overall, the impact of the Zara O.R. systems is impressive. As Chief Financial Oﬃcer of Zara, Miguel Diaz, commented on the original pilot study, “Th is experiment demonstrated very clearly that the implemented O.R. model had a positive impact on sales. Th is increase in sales can be explained by the model’s ability to move excessive inventory away from low selling stores where it is not needed, and send it instead to high performing stores where it reduces missed sales due to stockouts…. For a couple of years now, we have been at the point where every item found in any Zara store worldwide has been shipped to that store based on the output of the new distribution process. All our shipment decisions are now

IMPACT | AUTUMN 2015

FOR MORE DETAILED INFORMATION ABOUT THE WORK DESCRIBED PLEASE SEE: Caro, F., Gallien, J., Diaz, M., Garcia, J., Corredoira, J. M, Montes, M., Ramos, J. A. and Correa, J. (2010) Zara Uses Operations Research to Reengineer its Global Distribution System. Interfaces 40 (1), 71-84. Caro F. and Gallien, J. (2010) Inventory Management of a Fast-Fashion Retail Network. Operations Research 58 (2), 257-273. Caro, F. and Gallien, J. (2012) Clearance Pricing for a Fast Fashion Retailer. Operations Research 60 (6), 1404–1422. Gallien, J., Mersereau, A., Nóvoa, M., Dapena, A. and Garro, A. (2015) Initial Shipment Decisions for New Products at Zara. Operations Research 63 (2), 269-286

supported by O.R., and remarkably this has not diminished but rather expanded the role of the employees in the distribution team.” Zara has grown to be the backbone of the world’s largest clothing retailer Inditex, with over 1,900 stores worldwide by April 2015. The rapid turnover of products enables Zara to keep its fashion on trend, giving its lines greater customer value and enabling Zara to keep introducing more variety than its main competitors (typically stocking two to three times as many products) while holding price levels in a highly competitive market. O.R. now has a clear role in the company. Miguel Diaz again: “This project has also had a cultural impact on Zara. Speciﬁcally, Zara has initiated two additional major Operations Research projects in the areas of purchasing and pricing, and Zara is now also actively seeking to recruit graduates with strong O.R. backgrounds. In addition, within the Inditex Group there are plans to

deploy the new inventory distribution process in some of the other retail chains such as Massimo Dutti. The success of this project has shown that O.R. can signiﬁcantly contribute to Zara’s strategic goal of improving the scalability of its operations in order to support its continued growth.” With its new decision support systems online, Zara can look forward to surﬁng the chaotic waves of the fashion world with the best possible ability to survive and thrive. Brian Clegg is a science journalist and author and who runs the www. popularscience.co.uk and his own www. brianclegg.net websites. After graduating with a Lancaster University MA in Operational Research in 1977, Brian joined the O.R. Department at British Airways, where his work was focussed on computing, as information technology became central to all the O.R. work he did. He left BA in 1994 to set up a creativity training business.

Image © Hera Group

OPTIMIZING DESK C U S TO M E R R E L AT I O N S S E RV I C E S AT H E R A DANIELE VIGO

HERA GROUP is a large Italian multi-utility company that provides water, energy (natural gas, electricity and district heating) and environmental services (collection, treatment and disposal of urban an industrial wastes) to more than 3.5 million citizens in northern Italy (for more details see http://eng. gruppohera.it/group/investor_relations/ hera_overview/peers/). The company was created in 2002 as the aggregation of several municipal agencies from which it inherited a strong relationship with the territory

and the people served. For this reason Hera always relied on a high-quality Customer Relationship Management (CRM) service to keep contact with their users, by coupling standard channels, such as web and call centres, with a large network of physical Customer Contact Desks (CCDs). In 2011, the network comprised more than 80 CCDs employing 200 persons who served about 650,000 users per year which represented more than 20% of the yearly users’ contacts. By 2014, with the acquisition of some new companies in northern Italy, the

IMPACT © THE OR SOCIETY

Central planning unit

Desk Man1 CCD1

Desk Man2 CCD2

Desk Mann

...

CCDn

FIGURE 1 THE STRUCTURE OF THE PLANNING AND MANAGEMENT OF CUSTOMERS CONTACT DESKS

number of CCDs had risen to 120, serving more than 750,000 contacts. Most CCDs are small and open only a few days in a week, but 90% of the contacts are managed by 8 large and 20 medium CCDs located in the most important towns. The whole CRM service of Hera is managed by Hera Comm, the commercial company of the group that is also responsible for the commercialization of gas and energy. For Hera, the CRM service, and in particular the CCD network, represents an important advantage in such a competitive market for three main reasons: (i) such a widespread and highly-skilled service is normally not available to competitors who usually manage post-sales only through call centres or the web; (ii) Hera’s residential and business customers are used to top-quality desk services within easy reach; and (iii) the quality of post-sales customer care is crucial in order to gain, and keep, customers’ loyalty. The quality of service provided by Hera is regulated by Service Level Agreements (SLAs). The two most important SLA-related indicators are

IMPACT | AUTUMN 2015

the Mean Waiting Time (MWT) of users at CCD and the percentage of users waiting more than 40 minutes (PW40). Hera’s CRM service always ranked among the best in Italy, but improving such a high quality service in a fast expanding market requires relevant economic and human resources. The main target for Hera in the CRM area was to enhance the quality of service of the system, and in particular that of CCDs, without increasing costs.

the need to foster innovation towards more powerful and automated prediction and optimization techniques became clear to Hera’s management

After introducing in 2006-7 several procedural innovations (e.g., in CCDs layout, desk opening/closing rules, customer arrival forecasting and profiling, and desk staff training) based on office automation tools and

written instructions, the need to foster innovation towards more powerful and automated prediction and optimization techniques became clear to Hera’s management. As a consequence, a project was initiated in 2009 by Hera Comm for Optit to create an advanced Decision Support System (DSS), called SPRINT, for the complete management and optimization system for CCD staff to deliver customer services. Optit is a spinoff company of the University of Bologna that had been previously involved in several projects related to service optimization for the Hera group. The main requisites established for SPRINT system design were twofold. First, to give the planners access to forecasting and optimization tools in a user-friendly environment that offers simple controlling tools to guide the system to the desired solutions. In addition, the system should achieve good integration in the chain of processes for the planning, management and control of CCDs. To this end, it must take into account the needs of central planners who, being in charge of long and mediumterm planning and with control responsibility of the operational management, must have full access to all components of the system. SPRINT must also be helpful for desk managers, each responsible for the operational management of large and medium CCDs, who should have access just to the limited components required for the operational planning. The structure of the planning and management systems for CCDs at Hera is depicted in Figure 1. Figure 2 shows the central planning team during one of the meeting. Therefore, the functionalities of SPRINT are: • forecasting the arrival rate of users at the CCDs;

Image © Hera Group

• determining optimized scheduling of the staff of each CCD by guaranteeing that target SLAs are met; • performing “what-if ” analyses for scenarios of long and short-term planning; • monitoring the main Key Performance Indicators (KPIs) and objectives. SPRINT was implemented in 2010 and has been fully operational since February 2011. It initially supported the central planning office and the eight larger CCDs. During 2012 the system was gradually extended to medium CCDs and now covers more than 85% of the demand of Desk CRM for Hera. At the core of SPRINT there are effective forecasting and optimization modules which implement stateof-the-art approaches that compare favourably with other models described in existing literature. The forecast module is based on a so-called M5-model tree

that, by combining regression and classiﬁcation, predicts the daily arrivals at each CCD for a time horizon of one or more months. The tree-structured regression is built from the assumption that functional dependency between input values and forecast is not constant in the whole domain, but can be considered as such in smaller subdomains. The partition of input domain and the corresponding linear models are derived automatically by the method that uses as input the historical data on arrivals and other relevant service demand drivers, such as information about the billing process. The experimental results obtained by such a model are extremely good. As shown in Figure 3, which reports the average results obtained in 2011-14, SPRINT’s forecast turned out to be 25% more accurate, in terms of mean absolute percentage deviation (MAPD), than competing methods used in the literature for long term forecast of arrivals at service desks, such as

FIGURE 2 A PERIODIC MEETING OF THE CENTRAL PLANNING UNIT AT HERA COMM

de-seasonalized historical averages. Moreover, the number of days with large errors in forecast (i.e., with MAPD > 30%) were also considerably reduced.

At the core of SPRINT there are effective forecasting and optimization modules which implement state-ofthe-art approaches

The other main SPRINT module is the Optimizer, which implements a two-phase approach incorporating an explicitly-tailored Integer Linear Programming algorithm as Schedule Generator which determines the scheduling for the staff by relaxing some SLA-related constraints and defines the staffing requiring by using an innovative adaptive rule that is designed and tuned for the scheduling of desk staff. The overall quality and feasibility of the schedules is determined through a custom simulation model. The two components interact in an iterative process that converges within a few seconds to the solution that meets the required service level with minimum use of the available staff for desk activities, thus maximizing the resources available for other duties, such as back-office or sales activities. The schedules produced by SPRINT are generally considered by planners as very consistent and efficient with respect to those created manually, and favourably compare with respect to competing methods from the literature. The CCDs may include up to 20 counters and are open from 8am to 3pm of each weekday. The typical arrival rate of users at a CCD is given in Figure 4, which indicates that during peak time more than one

IMPACT | AUTUMN 2015

FIGURE 3 QUALITY OF THE FORECAST PROVIDE BY SPRINT AND THE COMPETING METHODS FROM THE SCIENTIFIC LITERATURE

user per minute enters the CCD. By considering that the average service time per user is well above 10 minutes it is clear that in some periods all available counters must be open to provide a timely service to the customers. On the other hand, in other periods an adequate service level can also be achieved when some counters are closed and the relative staff is detached for back-office duties. Such balancing between front-office and back-office duties for the counter staff is well illustrated by Figure 5, where a screenshot from the SPRINT console shows the daily plan for a large CCD. The green line represents the forecasted arrivals, the blue bar indicates the available staff in each 15 minutes time interval (note that it is halved during lunch break) and the red bar gives the number of open desks, hence that of staff employed in front-office duties. It can be clearly seen how the optimized openings and closing of desks (i.e., the red bars) try to anticipate arrivals

IMPACT | AUTUMN 2015

peaks and keep some staﬀ available for back-oﬃce for the longest possible time intervals. The quantitative results obtained during more than four years’ use of SPRINT are excellent. From 2011 to

FIGURE 4 TYPICAL ARRIVAL RATE DISTRIBUTION AT A LARGE CCD ON TWO DIFFERENT DAYS.

Image © Hera Group

2013 the total service demand of Desk CRM at Hera Comm increased by more than 25% and the staff number remained almost unchanged, while previously, keeping the service quality constant generally required an increase in staff proportional to that of the demand. Furthermore, also thanks to the support provided by SPRINT’s algorithms, in the same period the mean waiting time for the customers was reduced from 16 to 10.3 minutes and the customer satisfaction index of desk CRM rose from 72 to 81 points. In addition, the backlog of backoffice duties assigned to CCDs has been almost eliminated and the CCD staff was able to perform an intense proactive commercial activity during the time made available by the efficient front-office scheduling. During 2014 the achieved results confirmed those of the previous years. Moreover, in the same period Hera always ranked first among Italian utilities for the quality of CRM services. Sandro Bosso, director of the Consumer Market Division at Hera

Image © Hera Group

Comm, declared “SPRINT represents a perfect example of O.R. methods application in the real world. Its success was achieved by a good blend of high quality methodological support, strong managerial vision and state-of-the-art technological implementation. The achievements of the project will certainly boost the diﬀusion of OR not only within Hera but also in other Italian companies, as new performance standards are being set in this ﬁeld.”

SPRINT represents a perfect example of O.R. methods application in the real world

The results obtained by SPRINT motivated several projects that introduced the use of advanced analytics in the domain of back-oﬃce activity planning and operations management. Furthermore, after the first successful implementation in Hera, the main elements of SPRINT were incorporated by Optit into a software product that was adopted in 2012 by Gruppo Veritas, another multiutility active in Northern Italy. In addition a strategic version has been used extensively to support ENEL,

FIGURE 5 EXAMPLE OF PLANNING OPTIMIZED WITH SPRINT

the major Italian operator in the electricity market, in collaboration with SCS Azioneinnova, an Italian Management consulting firm.

For further details the reader is referred to: D. Vigo, C. Caremi, A. Gordini, S. Bosso, G. D’Aleo, B. Beleggia. SPRINT: Staff Management for Desk Customer Relations Services at Hera, Interfaces 44(5), 461-479, 2014.

Daniele Vigo is Full Professor in the Department of Electric, Electronic and Information Engineering at the University of Bologna, Italy. He has concentrated his research activities on the design and analysis of models and algorithms for Combinatorial Optimization problems arising in several application areas. He is a founder, and member of the management team, of Optit, an accredited spin-oﬀ company of the University, that produces decision support systems and provides consultancy based on state-of-the-art O.R. for the optimization of logistics, energy production and resources management.

SPOTLIGHT ON OR ESSENTIALS One of the newest books in the O.R. Essentials series, Mike Wright’s Operational Research Applied to Sports showcases how O.R. can be applied to sports including tennis, football and cricket, for: timetabling ﬁxtures, scheduling officials, optimizing tactics, forecasting outcomes and measuring performance. It brings together some of the best research papers on O.R. and Sport from the journals of The OR Society. Readers of Impact can enjoy a 30% discount on O.R. Essentials titles – please visit our website and enter promo code “PM15THIRTY” at checkout!

IMPACT | AUTUMN 2015

M A K I N G A N I M PAC T : O. R . I N H E A LT H C A R E Mike Pidd

The Nashville conference was fascinating, all the more so because most of the papers presented were based on work in the US healthcare system. This is very diﬀerent from the UK, even after the increased competition brought into the NHS by the 2008 Health and Social Care Act. There was a very healthy mix of academic researchers, clinicians and managers of healthcare services. BIG DATA AND PERSONAL HEALTH APPS

In July I attended the INFORMS Healthcare Conference in Nashville, TN. Though I’m officially retired, I’m still interested in the influence that O.R. can have on healthcare provision. O.R. people have been active in healthcare since just after WW2; that is, for almost 70 years. I am not alone in realising this must raise two questions: what impact have we had and can we increase it? They are one focus of MASHnet (http://mashnet. info/) and the Cumberland Initiative (http://cumberlandinitiative.org/), both of which take an O.R. view of healthcare improvement.

O.R. people have been active in healthcare...for almost 70 years

As in much early O.R., it seems that the UK once led the world, probably based on the creation of the National Health Service (NHS). This established a central organisation in which learning could be transferred from one site to another. The Nuﬃeld Provincial Hospitals Trust capitalised on this and encouraged some of the early O.R. Norman Bailey wrote about this early work under the title Operational Research in Medicine in a 1952 issue of the Operational Research Quarterly. He also wrote in the Journal of the Royal Statistical Society and in the Lancet, focusing mainly on appointment systems – still a current topic. O.R. work in US healthcare appeared some years later.

IMPACT © THE OR SOCIETY

Several things were very clear to me from the papers presented. The ﬁrst was that high hopes were vested in Big Data and its uses. Linked to this were quite a few papers describing or speculating about personal healthcare apps. These would allow people to monitor their own health and to decide what treatment is needed, where it should be done and who should treat them. I guess that some would also show how much this would cost. These apps seemed to be based on a belief, which I regard as rather naïve, that people are rational consumers of healthcare. I’m sceptical about the idea of a rational healthcare consumer, but not because I think that people are stupid. I believe that there is very little evidence that most of us are rational consumers, in the limited sense in which that term is often used, even when we make every-day or even large purchases. We all make irrational purchases despite the existence of Which? magazine and the many review sites on the Internet. Recreational shopping is a major hobby for some people and this hardly seems a rational pursuit. Even if we were rational about healthcare, we have no agreed measure of quality of provision. We also face an information problem: most of us simply do not have the context within which to understand some of the inevitable complexities. This was nicely illustrated in a BBC 2 programme by Michael Mosley shown in August this year. He chose to investigate health testing and screening by having several tests and talking about the results to camera. These tests included one in which he sent a saliva sample to a genetics lab and received a report on his genetic makeup and his susceptibility to various diseases. He expressed concern that most people do not have the context in which to interpret a report about their genetic status, no matter how well-written. He was also concerned that these tests would increase the number of “worried well”, a group that may already be too large. One role of a GP, in the UK at least, is to provide that interpretive context and, where appropriate, to propose treatments and other courses of action.

Image © Spotmatik/iStock/Thinkstock

This is not to argue that Big Data and Analytics will not have a major impact in healthcare, and personal health apps may indeed prove to be valuable. However, we do need to beware the hype; we need to look beyond the shadow to the real. We shouldn’t assume that healthcare will automatically improve once they are widely available. Real improvement is much tougher than that. SCALE MATTERS

Another thing that struck me at the conference was the existence of substantial systems engineering groups in large healthcare providers such as Kaiser Permanente and Mayo. The groups are large, with over 100 staff, and these include a sizeable number of O.R. analysts. The reports from those working in these groups suggest to me that there is some very useful work going on. It seems that the groups are strongly embedded within their parent organisations, which greatly appreciate what they do. They would grow larger if they could find suitably qualified staff who are able to do high quality, objective analysis and work closely with clinicians and managers.

the healthcare scene in the UK is very fragmented, which makes the transfer of learning based on O.R. work problematic

It is the scale of these analytical groups that gave me pause for thought. Of course, these are large organisations. Kaiser Permanente has over 9.5 million subscribers to its health plans, and generates over $50B income – not as large as the NHS, but still very large. The Mayo Clinic, which specialises in tertiary care, employs over 50,000 people on its various sites, though its revenue is much lower than Kaiser Permanente. Both have sufficient scale to make it worth their while to employ these teams of systems engineers. Since the move to NHS Trusts in England, pushed by recent governments, the healthcare scene in the UK is very fragmented, which makes the transfer of learning based on O.R. work problematic. The Trusts are simply not big enough to afford large analytical groups, though some employ a few staff. Hence they buy in consultants for their analytical expertise. I have no reason to doubt that the consultants do a good job and that their work has impact, but the learning often disappears when the consultants finish their assignments.

There are, of course, some substantial O.R. groups within UK universities that work in the healthcare sector. These do excellent work in the NHS and aim to ensure that organisational learning occurs, which is probably the most important result of successful O.R. With the possible exception of the group at ScHARR in Sheffield, however, none of these is large enough to match the efforts in the large groups in Mayo and Kaiser Permanente. Inevitably, therefore, their impact is limited. After the Second World War, the UK government nationalised several industries, including coal mining and railways. All were big enough to employ groups of analysts who conducted scientific and objective work that could lead to improvement with large payoffs. For nationalised industries these payoffs came in cost reduction and improved processes. I am not suggesting that we need to embark on a large scale programme of nationalisation, but that large scale integration probably eased the process of successful O.R. across different sites. The Department of Health, which is responsible for the NHS, now employs about 50 O.R. analysts, which is a sizeable number. Their main focus is policy, and they do work that is much valued by their colleagues and ministers. However, they do not focus on operational improvement and process planning. Due to the fragmented nature of English NHS provision, we lack sufficient in-house groups that are large enough to make the kind of continued impact claimed by our American cousins. Is this one aspect of US healthcare that we might wish to emulate in the UK? Mike Pidd is Professor Emeritus of Management Science at Lancaster University, former President of the O.R. Society, and a regular receiver of NHS care.

IMPACT | AUTUMN 2015

U S I N G S I M U L AT I O N – YO U K N OW I T M A K E S S E N S E ! MARK ELDER

SIMULATION HAS BECOME one of the more popular techniques coming out of the Operational Research/Management Science (OR/ MS) world – why is that? In my view there are five key reasons for this. I want to explore these from the perspectives of three groups of people – OR/MS workers, the clients of OR/ MS and also the people outside of OR/ MS who have recently picked up and used simulation without any particular realisation that simulation is one of our tools. But first, let’s define what we mean by “simulation” because it’s a word that get used for many different activities. In the context of improving processes, simulation is seen in Figure 1. On a computer screen you see a visual mock-up of your process. It’s animated so you can see what’s

happening in the simulation as it progresses through time and you can also interact with it, so you can change it in ways that you might consider changing in the real process. It behaves just like the real process (more later on how it does that) so that changes you make are reflected in the animation. A clock shows the time of day or year or whatever’s appropriate. The simulation runs through time much faster than the real process so you can try many different ideas quickly. It could be described as a computer game for process managers, but it’s more than that because it counts and measures its behaviour and produces reports that are easy to compare with previous reports, so you have numerical evidence about what configuration works best. Most simulations also include variability in their behaviour,

FIGURE 1 - A VISUAL AND INTERACTIVE SIMULATION FOR IMPROVING PROCESSES

IMPACT © THE OR SOCIETY

for example so that customers arrive in peaks and toughs rather than in a nice, easy-to-serve, smooth ﬂow!

a simulation... could be described as a computer game for process managers, but it’s more than that

The problem with explaining simulation to anyone is that they expect there to be something complicated behind it. Some magical mathematical technique invented by a winner of a prestigious mathematics prize maybe. Unfortunately (or maybe fortunately) that’s not the case. Simulation does exactly what it says. It mimics what happens in the real process. However, the really useful thing is that if you mimic what happens to one, say, customer (let’s assume we are simulating a store) when they reach the checkout, with all the likely behaviours if the server is available, or busy, or busy with a wait line of 5 people etc., then all you need to do is throw a bunch more customers in to see what’s going to happen on a busy Saturday afternoon. And so you can work out how many servers to employ. Then you ﬁnd yourself quickly able to predict the behaviour of a huge supply chain of hundreds of factories each with 1000s of product lines and 1000s of employees. That is all simulation does technically. By mimicking the behaviour of each part of the process as it interacts with other parts it is easy to look at how a whole system will perform and, of course, look at how it will perform if you try alternative ways to give it resources or clever ways to control the way it copes.

I like the example where three banks were merging all their call centre operations and they were adamant they should maintain their policy of keeping their best agents reserved for their “platinum” customers, but they also wanted to keep call hold times down to the levels of their competitors. With simulation they could easily try diﬀerent schemes for when to automatically “bend” their call allocation rules so the best agents could help out with lengthening queues. They were also able to show the bank’s board, visually, how the system of 10 merged call centres and nearly 100,000 agents would do a good job for every customer.

By mimicking the behaviour of each part of the process as it interacts with other parts it is easy to look at how a whole system will perform

One last thing before we look at why simulation is so popular. You might be asking yourself: “why can’t a spreadsheet give me the same answers?” (assuming we ignore the visual animation aspect of simulation). Spreadsheets are for static analysis not analysis impacted by time. In a process analysed by simulation you get to see the consequences of events being delayed by non-availability of resources just when you need them. So there might, on average, be enough agents in the call centre to answer 10,000 calls a day without a delay, but in practice calls do not arrive smoothly so agents may still be ﬁnishing an earlier complicated call when their next call comes in. This is reasonably easy to understand in a system as simple as

a call centre, but in a process where there is a chain of activities or even where transactions take different or circulating routes it becomes impossible to work out the nature of the knock on consequence of delays. It gets even more complex when the “history” of a customer impacts their flow through the process (for example if they waited longer than 6 minutes at stage 2, then when they get to stage 6 we need to . . . etc., etc.). Simulation individually mimics every customer (or other type of transaction) that flows through the process, with attributes that make them differ from other customers, including attributes that only occur because of how they flow through the process. It means simulation really can behave just like the real world.

PROJECTS INVOLVING SIMULATION

Now you have a real practical deﬁnition of simulation, how do projects work? To understand most of the reasons for simulation’s popularity you need to understand how simulations get created in practice. You might think I’m writing this from the perspective of the simulation company I founded over 20 years ago, but over my career I’ve worked for three simulation companies, in each case both building software and doing consulting with their products. I also know reasonably well at least four other simulation products and everything I’m saying here applies equally well whichever of those you use (and, I’m sure, for most others too). However, this particular way of creating simulations is not quite “by the book”, but it is how most experienced simulation builders actually work. So it may surprise some readers who have only read how simulation “should” be done (see Figure 2).

IMPACT | SPRING 2015

You will see that, for example, we do not collect the data for the simulation before we build the simulation. (By “data” we mean things like how long it typically takes to serve a customer or answer a phone call etc.). This is because building the simulation structure is the best way to work out what data needs to be collected, and in any case the act of building the simulation is actually the period where huge amounts of learning goes on for both the client and the simulation builder (if they are diﬀerent people, because, of course, these days simulation building is easy enough for many “problem owners” themselves).

FIVE REASONS FOR SIMULATION’S POPULARITY

Transparency. An issue with most OR/MS methods (not all) is that it’s quite tough to turn a clever piece of mathematics into something visual

Discuss

and indeed something that visually intuitively matches the things that are being improved or optimized by the mathematics. An exception to this is simulation. A story from the days when simulation first became “visual” will help illustrate the importance of this. One of my very earliest OR/MS projects was to answer the question “On which assembly line should we build the van?” I worked for the car company that made the original Mini and the OR group was asked that question by the manager of three assembly lines at the Longbridge assembly plant. The van was planned to be 15% of overall production of this particular model of car and its assembly was to be merged onto one of three lines, each of which differed in capability, mainly because they had been constructed at different times over the previous decade or so. We built a simulation using software from IBM known as GPSS (which was non-visual at the time). During the

(Re)Build or Enhance Simulation

Ask Questions

Learning Loop

Collect Data

Verify & Validate

Run

Gain Insights

Perform Experiments

Implement Decision

FIGURE 2 - TYPICAL SIMULATION PROJECT

work we obviously did some sensitivity analysis such as “Is the 15% important? what if it is 20%?, what if it is 10%? etc.”. When we went along to the client to present the results my boss stood up and started with a summary of our findings. Assembly line 3 was most cost effective under all the assumptions we had been given (including the 15%), but the chair of the meeting was not interested in hearing all our caveats that showed the interesting things we had discovered during the sensitivity analysis. Line 3 certainly was not the best line if the van was over 25% or less than 12% of total production (in reality the van never sold more than 5% of the volume). But it was too late. Line 3 was used. Somehow none of our presentations seemed important to our clients. We had to fix this. So we built a wooden board game simulation, using dice to generate variability and metal counters to represent cars in the factory. We took this board game to show the

IMPACT | SPRING 2015

The simulation process is very iterative. On early cycles, runs are just animations that aid communication, later they are focused and accurate enough for experimentation.

plant director at Longbridge in the hope of explaining how simulation worked so that future projects would get a better hearing. We were a little concerned that he would see the board game as trivializing the complexity of his factory. The opposite happened. He cancelled all his meetings for the day. He called in all his senior team and we did not get away from his oﬃce until 7pm that evening. During the day we simulated, manually using the board game and dice, only 15 minutes of real production but as we left he said, “Today has been the most useful contribution operational research has ever made to my plant”. He went on to say how much it had brought his team together and how much they now understood about each other’s problems. We did not know whether to cry or cheer. Was all our previous work less useful than the board game? So the outcome of this was we automated the board game on a computer screen and simulation became visual. However, a much more important outcome was that from then on clients were involved throughout the project. Because they could see and understand what we were doing they wanted to be involved. We could not keep them away. They did not need to listen to all our assumptions and caveats because there was no need for the presentation – they knew the answers, and all the sensitivity outcomes at the same time as we did.

Today has been the most useful contribution operational research has ever made to my plant

The second reason simulation is so popular is that it solves the right problem. Because the client is so involved in the whole solution process

it forces that solution process and the client’s thinking to stay in pace with each other. We were asked to help the company that runs the tugs that berth giant oil carrying tanker ships at the oil terminal on the Forth just outside Edinburgh. They owned five massive £5 million pound tugs. A new terminal of similar size, berthing similar tankers, was to be built next to the old one and the port authority was going out to tender to find the best operator for the tugs for the new terminal. Our client wanted to win the contract and felt his company was in an unassailable position because any new operator would definitely have to buy five tugs (the number required to work together to berth the largest tankers) but they felt sure some mathematical analysis would show that running both terminal berthing operations together would not need 10 tugs. Except for the complexity of the tide tables this was a very simple simulation to build and we left our client with a working simulation (without the tide tables) at the end of the first day. Next morning when we returned we found him dragging the mouse around playing with the simulation saying “it’s obvious, it’s obvious”. He had

proved to himself on the back of an envelope, prompted by what he was seeing in the incomplete simulation that he would certainly have to buy 3 additional tugs just to logically cope with certain combinations of ships but that it was very unlikely that a total of 9 (4 additional) would ever be useful. Indeed when we heard his logic it was clear to us that this should have been obvious without asking us to do the project! But what was really interesting was how his thinking had moved on. The project was to continue because he had realised that the really interesting question was not how many tugs but how to crew the tugs. This could now be organised differently because of the way two of the tugs could be scheduled to be idle most of the time. Twenty four hours later his thinking had moved on again and he was interested in bidding to run the fireboat operation too, by equipping some of the new tugs with fire fighting equipment. So the lesson was that, because the client is involved in the thick of the simulation project, the simulation can speed up the evolution of a client’s thinking and the project can change direction mid-stream to focus on what the client really needs to know rather than what

IMPACT | SPRING 2015

they said they wanted to know at the project briefing. I’ve often labelled this reason for simulation’s popularity as the “Journey of Discovery” benefit. The benefit of the project comes not from the end outcome (report) from the project but from the exploration that you undertake between the start of the project and the point that the client believes they have enough information to make their decision.

because the client is involved in the thick of the simulation project, the simulation can speed up the evolution of a client’s thinking

Include any rules. The third reason simulation is popular is that it requires little time/energy/skill on “problem structuring” in that you do not have to think of a way to ﬁt the problem to the mathematics. Of course this does not mean that building the simulation is completely pain-free but it does mean that it can be achieved by the people who understand the process being simulated, rather than by someone who understands the structure of the mathematical technique being used. For similar reasons OR/MS people like simulation because there is no limit to the degree to which they can innovate ways to improve the process and then measure and test the impact of these inventions. This is because, rather than ﬁtting the problem to the technique, the pure simplicity of creating a simulation that does the same as the real process at a detailed level (remember, “is there a server busy?, yes, so wait in the line”) and letting the simulation software aggregate this up to overall behaviour means that any

IMPACT | SPRING 2015

control rule for attempting to increase effectiveness can always be built in and tried. Speed is of the essence in much OR/MS work. We have already discussed the rapidity with which the problem owner’s thinking can change and how the simulation helps facilitate that thinking. If the simulation can’t actually handle the ideas the client and the solution team are inventing, at the pace with which they invent the ideas, then clients will often resort to intuitive decision making rather than evidence based decision-making. A well-known Detroit based simulation team has a policy of “We go from a blank screen and a client with a new problem to a finished simulation and a decision made by lunch time, otherwise we are being too slow”. Now, I think they achieve that because they are very good at the whole process. It is not just about building and using the simulation. How you help the client think through the decision in the session and what you have prepared beforehand (for example, they do have databases of past reliability data on tap) is very important to be able to work at that speed. However, the ability of that group to work quickly has led to that corporation mandating from main board level that nothing changes in any of their plants worldwide without being simulated first. Simulation’s ability to simply replicate any real behaviour is what makes this possible. Finds non-technical solutions. Partly because simulation is very visual it engages people. However a second way people become engaged is that their simulation becomes a repository of objectively quantified facts about how the process works. Teams of people stop arguing and watch the simulation. People ask it questions. Questions to which they should know the answers

because they supplied them in the first place! I won’t forget the first time I was called by a client and asked, “What is the speed of conveyor 15? He was calling from the shop floor and was probably close to conveyor 15. I could not understand why he was asking me the question, so I asked him to repeat it. “But you gave us that information along with all the other speeds, it’s an input to the simulation” “I know” he said, “but all the data is in one place now so it’s easier to look in the simulation to find it”. What this illustrates, and what I later understood, was that the simulation is a trusted source of knowledge and an arbiter between disagreeing parties. What this leads to is the generation of better problem solving, sometimes with solutions that are not part of the simulation. We helped the planning work at the new Edinburgh Royal Infirmary for the move of Accident and Emergency from the old to the new hospital. This included running a series of workshops for staff at all levels around a series of simulations, some of wide scope and some quite detailed. At the end of the final workshop we were approached by the chief exec who said, “This has been a really interesting experience, watching how the simulation bought us all together, indeed the biggest outcome is the suggestion by [the General Practitioner] to segment the waiting lists. This is to his personal disadvantage but it will benefit the whole process and it was not even something included in the simulations.” So we are almost using the simulation as a meeting facilitation device, getting everyone to think clearly how to improve the process with the simulation just taking care of a few factual calculations on the side. As a final reason in my five I guess I ought to mention that it’s quite good at improving processes the way it says on its

tin! It does this with both clients and OR/MS people. With any modelling method you can put in some inputs and get back the impact on the outputs. And yes simulation does this and does it to any required level of detail (on very large and complex systems, but also at a high strategic summary level when that’s right). But what it also does is to cause both those sets of participants to go around a “cycle of learning” and improvement. When you see the outputs and when you have watched the visual animation, you often realise what was constraining the outcomes and so the act of running the simulation is generating new ideas for the next run of the simulation. This makes it tough

to know when you have finished (and even tougher to predict at the start how long the project will take). We have had 5 day projects compress into half a day because the client saw the solution as soon as we asked a few sensible questions on the first walk around the plant and we have had 10 day projects turn into 64 day projects because the client kept finding more and more things to improve. And the only downside I see to all this popularity is that we do often find clients demanding “a simulation” rather than their “problem solved”! Mark Elder founded SIMUL8 Corporation in 1994. He was awarded the OR Society’s Beale Medal in 2012.

2015 BLACKETT LECTURE Blackett Memorial Lecture The OR Society is pleased to announce that the 2015 Blackett Memorial Lecture will be given by

Kenneth Cukier Data Editor for The Economist Kenneth Cukier is the co-author of the award-winning book Big Data: A Revolution That Will Transform How We Live, Work, and Think. He is a regular commentator on BBC, CNN, and NPR, and a member of the World Economic Forum's council on data-driven development.

Thursday 26 November 2015 at Grocers’ Hall, Princes Street, London, EC2R 8AD Lecture at 4.30 pm (Tea and biscuits at 4.00 pm; Drinks reception after the lecture) There is no charge for attendance at this event. To register and receive joining instructions, please go to

www.theorsociety.com/Pages/Conferences/Blackett.aspx Please note that places are limited, book up very quickly and therefore are reserved on a first come first served basis.

IMPACT | SPRING 2015

EACH YEAR STUDENTS on MSc programmes in analytical subjects at several UK universities spend their last few months undertaking a project, often for an organisation. These projects can make a signiﬁcant impact. This issue features reports of projects recently carried out at two of our universities: Cardiﬀ and Southampton. If you are interested in availing yourself of such an opportunity, please contact the Operational Research Society at email@theorsociety.com

SCHEDULING OF WINDSCREEN REPAIR AND REPLACEMENT VEHICLES FOR THE AA (Adrian Davis, Cardiff University, MSc Operational Research and Applied Statistics) added to the routing, at which point the algorithm terminates. A further tidying up phase is also performed after each improvement phase iteration; this ensures any unnecessary idle time for the vehicles is removed. The final model and recommendations have been implemented in the AA planning department and are being used The AA carries out approximately Depot Vehicle Routing Problem with to evaluate performance using post-live 150,000windscreen repairs or Time Windows (allowing repairs to be data and plan the mobile technician’s replacements per annum. The majority made close to a specified time) and added location for forecasted workload. of this work isautomatically matched constraints such as the limitation of one Simon Jones, AA’s Manager by a deployment system, in which vehicle per depot and varying resource of Business Optimisation and the appropriate resource for the job is shift time windows. The key objectives of Analysis, said “Adrian worked largely identified and it is placed on a mobile the algorithm were to incorporate as many independently, asking relevant questions technician’s plan. The lead time may of the jobs as possible into the routing, throughout the research period be many days, as parts will have to be whilst maintaining a minimal overall travel which enabled him to establish how ordered from glass suppliers and delivered time incurred by all service vehicles. to approach creating the model. The to a location for the technician to collect. The algorithm consists of a number standard set by Adrian throughout his The aim of Adrian’s project was to of phases, each of which has a different dissertation was exemplary. Adrian was provide a review of vehicle and workforce purpose. The phases include an initial enthusiastic, keen and professional route problems and related scheduling constructive heuristic which aims to build Following the project with Adrian, methodologies which are applicable to a reasonable quality, but not necessarily the models have been extended and the AA windscreen business. He was complete, initial solution to the problem. are implemented within the planning tasked to build and validate a practical Once complete, the programme goes department. At present, the data is solution to this problem, examining the through a number of improvement phase loaded daily and the vehicle routing impacts of changing parameters such as iterations attempting to minimize the model has been automated, highlighting drive times and on-the-job times. cost of the routings, only considering jobs improvement opportunities in the The Vehicle Routing Problem has that had been included in the solution so technician’s plan. A 7 day window has been of great interest to academics in far. Once no further improvement can also been included, analysing capacity recent decades with the development be made, the algorithm continues into and technician utilisation over the of sophisticated computer systems that the next phase of job insertion, where an period. The results of the models are are of significant importance when high attempt is made to insert omitted jobs circulated to the operations team within quality solutions to computationally into the routing. The algorithm alternates the AA enabling alternative schedules complex problems are required. Adrian between the improvement and insertion to be easily explored and more efficient developed an algorithm for the Multiple phases until no further nodes can be deployment decisions to be made.”

IMPACT © THE OR SOCIETY

Source: Cardiﬀ University and The Automobile Association

U N I V E R S I T I E S M A K I N G A N I M PAC T

Source: University of Strathclyde

MAINTENANCE TECHNICIAN SCHEDULING OPTIMISATION FOR THE ROYAL NATIONAL LIFEBOAT INSTITUTION (Rachel Purkess, University of Southampton, MSc Operational Reasearch)

The purpose of the Royal National Lifeboat Institution (RNLI) is to save lives at sea. It was founded by Sir William Hillary in 1854, since when it has saved over 140,000 people. The RNLI operates a 24-hour, on-call lifeboat service around the UK and Republic of Ireland. The Coastguard is the ﬁrst point of contact for people in distress at sea, but this only has helicopters and a small number of boats; the RNLI is the organization that is asked to carry out the search and rescue. The RNLI has over 340 lifeboats, split between all-weather lifeboats (large, medium and small) and inshore lifeboats. The RNLI is a charity: it does not receive any Government funding and its entire income is raised from donations. It has an annual turnover of approximately £150 million. Its workforce is 95% voluntary, including over 5,000 lifeboat crew members and 30,000 fundraisers. Clearly, it is important that the RNLI spends its funds wisely and eﬃciently. The RNLI has several links with the University of Southampton, both in Marine Engineering and Operational Research. Jodie Walshe, the client sponsor for the project described here, is a graduate of Southampton’s MSc in Management Sciences who was employed on an

EPSRC-funded Knowledge Transfer Partnership project between the Business School and the RNLI before taking up a permanent role a Decision Support Manager with the RNLI. In 2014, Rachel Purkess, a student on Southampton’s MSc in Operational Research, undertook a summer project for the RNLI. Rachel’s project concerned scheduling maintenance and repair of lifeboats. RNLI’s six regional divisions each have a team of Systems Technicians that undertake both planned maintenance and unplanned maintenance (repair work) on the equipment based in their division. Rachel’s project considered how to schedule maintenance jobs for technicians in the most eﬃcient way, particularly focusing on minimising travel time and reducing the necessity for overnight stays away from home. At the same time, the algorithm produced had to achieve maintenance requirements and satisfy a variety of hard constraints such as working time restrictions and the need for some maintenance jobs to be worked on by two technicians simultaneously. Rachel’s algorithm is implemented using Visual Basic for Applications and forms the basis of an Excel-based decision support tool that is easy

for regional managers to use. The scheduling algorithm is based on an iterated local search framework, which uses far fewer computational resources than exact methods, and produces higher quality solutions than classical heuristics. During her project, Rachel travelled to Wales to talk to RNLI technicians and also spent time at the RNLI headquarters in Poole. She said: “I used data from the West region (Wales) in my algorithm to compare scenarios to quantify the benefits of basing technicians in three different locations, rather than a single location (St Asaph in Pembrokeshire). In particular, the first scenario showed a 49% reduction in travel time and 78% reduction in overnight stays. The tool can be easily adapted for use in different divisions so that similar scenario comparisons can be conducted across the country.” Jodie commented: “By using Rachel’s tool to schedule work, the saving in the West region could be as much as £147,000 per year inclusive of reduced cost of fuel and cost of technician productive time vs. driving time. Also, on average, each technician will have 66 more nights at home (an important result for the RNLI’s commitment to its staff )”. He went on to say that “if similar savings are able to be made across all regions, then the cost efficiencies that could be realised over the whole of the UK and Republic of Ireland might be up to £1m per year, equivalent to a new Shannon class lifeboat every 2 years.”

IMPACT | AUTUMN 2015

Image © Mark Bowden/iStock/Thinkstock

NHS CERVICAL SCREENING Achieving a 14 day turn-round for results HAZEL SQUIRES AND JOHN RANYARDII I I I I I

IMPACT © THE OR SOCIETY

WHEN LABOUR’S 2005 general election manifesto pledged a dramatic cut in the time taken to deliver cervical cancer screening results – from six weeks to seven days – many thought such a reduction impossible. It fell to the O.R. community to bridge the gap between rhetoric and reality. According to Cancer Research UK, cervical cancer is the fourth most common type of cancer in women worldwide. The World Health Organisation has estimated that in

2012 alone the disease was responsible for 266,000 deaths. Incidence in the UK has fallen signiﬁcantly since 1988, when a screening programme to detect the early and treatable forms of the disease was introduced. Cancer Research UK reported a 42% decrease in the ﬁrst decade of the initiative. Nonetheless, the delay between screening taking place and results being received remains a crucial consideration. A shorter turnaround

Image © The OR Society

time (TAT) is desirable not only in terms of ensuring rapid treatment when needed but also to minimise anxiety in those awaiting the outcome.

cervical cancer is the fourth most common type of cancer in women worldwide

In 2005 the target TAT was six weeks. Even this was a goal that only 56% of Strategic Health Authorities were able to attain in practice, yet in the run-up to that year’s general election the Labour Party unveiled a manifesto commitment to lower the TAT to just seven days. There was little doubt that bringing about such a major reduction would be a sizeable challenge. In due course, when Tony Blair led his party to a third consecutive victory, the O.R. community was tasked with helping to determine how – and, indeed, if – it could be achieved. Shortly after the election, NHS Cancer Screening Programmes (CSP) commissioned a formal options appraisal from the School of Health and Related Research (ScHARR) at the University of Sheﬃeld. The initial modelling was carried out by Hazel Squires, under the supervision of senior ScHARR researchers. At the time the conventional test for cervical cancer, the Papanicolaou smear – commonly known as the Pap – was gradually being replaced by Liquid-Based Cytology (LBC). LBC was a superior method, able to reduce the risk of inadequate samples and to speed up laboratory throughput, but it would not be fully adopted until 2008. Both means of testing were factored into ScHARR’s evaluation. Five cytology laboratories around England were studied in detail. The data

FIGURE 1 SCREENING PROCESSES

they provided were validated nationally through questionnaires sent to a sample of 25 further labs. Together with material from relevant research papers and other literature, all of the information was used to assemble a discrete event simulation model representing a typical lab (small, medium and large). The Cervical Screening Process Model, as it was called, was implemented using the Simul8 software package (see Figure 1, taken from H Pilgrim and J Chilcott (2008) Assessment of a 7-day turn-around for the reporting of cervical smear results using discrete event simulation. Journal of the Operational Research Society 59, 902–910). It allowed a number of options for change – based on suggestions from CSP, other stakeholders and the project team – to be evaluated. The following measures emerged as those most likely to be eﬀective: • Limiting processing to samples from women eligible within national standards • Despatching results by ﬁrst-class post on Monday, Tuesday and Wednesday mornings

• Redesigning the workforce, including the training of advanced practitioners • Merging small labs’ workloads • Establishing electronic links between labs and call/recall oﬃces The study concluded that, if these changes were made, a seven-day TAT would not be possible but that over 95% of results could be returned within 14 days. It was also estimated that the proposals would deliver annual savings of more than £18m. A project report setting out the practical details of how this could be realised was duly presented to CSP.

It was estimated that the proposals would deliver annual savings of more than £18m

In addition to ScHARR’s appraisal, CSP commissioned Beaumont Colson Ltd, an IT company with extensive experience in the healthcare sector, to review local administration of cancer screening programmes. A key

IMPACT | AUTUMN 2015

recommendation was to streamline call/recall offices, so bringing cervical screening more in line with other programmes and providing more support to women throughout the process. A package of measures based on ScHARR’s and Beaumont Colson’s studies was developed with the assistance of principal stakeholders, including SHA representatives. The result, published in 2008, was the Cancer Reform Strategy, which stated that all women should receive the results of their cervical screening tests within two weeks – provided the LBC method was used. This last stipulation would not be an issue. Some 88% of labs in England had converted to LBC by November 2007, and all Primary Care Trusts indicated the remainder would follow by October 2008. Moreover, the benefits of LBC were becoming increasingly plain. Of the four million tests taken each year, the number classified as inadequate fell from 370,000 (9%) in 2004-2005 to 173,000 (4.7%) in 2006-2007. As a consequence, around 200,000 women did not have to attend a repeat test. Anxiety, additional expense and unnecessary workload were all being reduced. In conjunction with the move to LBC, the carrying out of ScHARR’s recommendations has since led to a dramatic fall in turnaround times. This is illustrated by the latest figures available, which cover the period from 2012-2013. During this time: • 4.24 million women were invited to undergo cervical screening • 3.32 million were tested • Almost 3.57 million cervical cytology samples were processed by cytology clinics (some women need repeat tests for clinical reasons)

IMPACT | AUTUMN 2015

• 97.8% of women received their results within two weeks. Professor Julietta Patnick, Director of the NHS Cancer Screening Programmes, has said: “This impact modelling work was instrumental in working out how to speed up the time it took for women to get their cervical screening results. Introducing a two week turnaround time forced several key improvements in cervical screening including a workforce redesign, restructuring our laboratories and introducing better electronic links between diﬀerent parts of the programme. The fact that all but a few women receive their results within 14 days now is crucial to minimising the amount of anxiety involved in screening.”

that all but a few women receive their results within 14 days now is crucial to minimising the amount of anxiety involved in screening

It is important to acknowledge that change did not happen overnight. Given the number of stakeholders involved and the depth of engagement necessary for a successful outcome, this was inevitable; moreover, some of the relevant measures, including not just the introduction of LBC but the merging and/or closure of labs, could not be hurried. Strictly speaking, Labour’s original manifesto pledge also remains in part unfulﬁlled. Although many women do receive their test results within seven days, the target TAT is currently set at 14. But that is an issue for politicians rather than operational researchers. .Even with projects as eﬀective as this, the gap between rhetoric and reality cannot always be entirely bridged.

TIMELINE 2003

Policy decision to replace PAP tests with LBC over 5 years

2005

Labour Party Election Commitment “screening results in 7 days”

2005/6 ScHARR asked to carry out an Options Analysis Project carried out by Lancaster O.R. Master’s student 2006

Results and recommendations reported to the Cancer Screening Programme

2007

Beaumont Colson Ltd review the local administration of the NHS CSPs NHS Improvement Team develop implementation measures with key stakeholders

2008

Cancer Reform Strategy states that all women should receive the results of their cervical screening tests within two weeks.

2010/1 78.9% in 14 days 2011/2 95.6% in 14 days 2012/3 97.8% in 14 days

Hazel Squires carried out the options appraisal project as part of her MSc in O.R. at Lancaster University. She joined ScHARR after graduating, where she is now Senior Research Fellow. Jim Chilcott and Simon Eggington were her supervisors at ScHARR. John Ranyard was her supervisor at Lancaster University, where he was then External Liaison Manager at the Department of Management Science.

Image © Phil Britt

SPORTING TIMES (AND PLACES) Mike Wright “We played against them on a Tuesday last year as well!” “Why do we only get this referee for away matches?” “That’s a long journey for the players to make before the match tomorrow!” “Why is there always a ‘Group of Death’?” As this issue of Impact arrives on your doorstep or through your emailbox, many people all over the world will be engrossed in the Rugby Union World Cup. Or perhaps (if Impact isn’t published on time) the Brazilian Grand Prix or the World Weightlifting Championships – or even, if it’s really late, the Australian Open Tennis or the Youth Olympics. However, it doesn’t really matter – an article about sport will always be topical. There are always sports ﬁxtures to be scheduled: globally, internationally, nationally or just at a local amateur level. Sport is a microcosm of life. There are big events (births, Olympics) and small events (catching a train to work, driving down the fairway) with plenty of middle-sized ones in between. There are predictable routines (cooking the dinner, preparing the cricket pitch) and surprises (children’s school reports, FA Cup upsets). Decisions need to be taken – big ones (whether to apply for a new job, how to restructure the World Snooker Championship) and little ones (which socks to wear, who to pick in the swimming

relay). And both sport and life wouldn’t work at all without a great deal of thought, organisation and planning. I’ve probably lost half my readership already. That’s OK, you’re the ones who probably wouldn’t have read this far anyway. But for the rest of you, this may give you a hint of some ways in which O.R. – which of course involves analysing decisions and making recommendations in order to get things done – may be of value in the realm of sport. Just as the whole of life can be thought of as an implementation of O.R., so can sport. There are many ways in which O.R. can be applied to sport – far too many for this short article. So I shall put to one side how O.R. can help with strategic decisions, or how it can help tactical decision-making by coaches and players, though there are plenty of examples. I shall not go into betting or forecasting, though plenty of people do. I shall concentrate instead on the topic of sports timetabling/ scheduling (the terms are often used interchangeably) for competitions. Broadly speaking, the organizational decisions to be made for a sporting season/tournament are: 1. who (which team or player) should play against whom? 2. how often? 3. where? 4. when? 5. who will oﬃciate? (Usually one or more referees/ umpires/etc.) In some circumstances, questions 1 and 2 can be answered very easily. A typical league-based competition

IMPACT | AUTUMN 2015

Image © RWC

is built around a round-robin structure – this means that every team plays every other team once – or a double round robin – twice, once at each team’s venue. And the teams in a league, or in a division of a league may be determined by what happened last year – teams get promoted or relegated, others may withdraw or go bust. Some competitions are based wholly or partly on a random draw. This generally doesn’t pose any kind of problem to the scheduler, though it doesn’t prevent the FIFA World Cup draw from being a major event lasting over an hour and televised in practically every country in the world. Although the element of randomness does usually ensure approximate fairness (in the sense that nobody is treated worse than anyone else in principle), it does inevitably mean that some competitors are treated more harshly than others (e.g. groups of death, as is the case for the Rugby Union World Cup with England, Wales, Fiji and Australia all in the same group, or difficult quarters of the Wimbledon draw, for instance), though these can be alleviated by some kind of seeding method. Where the competition is not fully round robin and does not rely on a heavy dose of randomness, however, it can be difficult to ensure that the schedule is fair to everyone – and, probably more importantly, is seen to be fair to everyone. What counts as fair, however, is not always obvious, even to people from the same team or club, especially for professional sports: for example, players and coaches might prefer to play against the weaker teams more often than the stronger ones, spectators might want their team to play most often against nearby teams, while those in charge of the finances would like to play as often as possible against teams with the most supporters. Here the Operational Researcher can wade in, propose a measure of fairness that most people can agree with, and come up (usually via a computer program) with a set of opponents for every competing team or individual that tries to be as fair as possible to everyone. Not easy. More complex are the where and when (often considered together) questions. Even then, for most amateur competitions at least, there are detailed templates that can be used. Thus the organizer of your local pub darts league probably works from a piece of paper that says: “In round 1, A plays at home against B, C at home against D, ……….”, continuing right up until “In round N (the last round), C plays at home against J, E against B, ……”. OK, maybe it requires more than one piece of paper for a large league, but in essence the job is done. All that remains is to determine (maybe randomly) which pub is A, which is B, etc.

The templates are arranged so that there is an approximate alternation of home and away matches for each team, and they even conveniently pair teams such that there is always one at home and one away; so if there are two darts teams in a pub, it can be arranged that their home matches never clash. So far so good. But this won’t always work, especially at professional level. Here all sorts of requirements and preferences have to be considered, from several diﬀerent types of stakeholder. Thus clubs will want to specify particular dates when they do or don’t want to play at home. This can be for a very wide variety of reasons, including stadium availability, holidays and clashes with other events. Take England’s county cricket as an example: Yorkshire can’t be at home when Leeds Rhinos have a home match; Gloucestershire always want to have a protracted period at Cheltenham in July; and Northants like to avoid clashing with the local balloon festival (yes, really). Some times of the year or days of the week will be more popular than others with

Image © England Rugby 2015 Ltd

the marketing people, and the players and spectators may have diﬀerent ideas. Players and coaches won’t fancy long journeys, especially those that have to be made overnight. Marketing folk and spectators will like a good spread of home matches. The sponsors and tournament organisers will have their say. And the biggest clout, certainly for the most popular professional sports, will be wielded by the televisers, who will have their own ideas as to what makes a good schedule and are the ones with the money to back up these ideas.

although mathematicians are doing their best to make their theorems as useful as possible, real situations tend to be too complex for them

Under these circumstances, the O.R. professional comes into his or her own. Except in relatively simple situations, there won’t usually be any sophisticated mathematics to help with this task; although the mathematicians are doing their best to make their theorems as useful as possible, real situations tend to be too complex for them. In any case, the idea of an “optimum” outcome doesn’t really hold water, when so many

points of view have to be incorporated, many of which will run counter to others. The best that can be hoped for is an outcome that everyone is reasonably happy with (especially the TV companies). Success is measured by smiles, or at least the absence of whines. So how does the O.R. analyst come up with schedules? Well, I use metaheuristic approaches – these are generic methods that can be adapted to fit the problem being tackled. Typically they will use randomness but in a structured way to make iterative improvements to any starting schedule. And to tell you any more would be outside the scope (and word limit) of this article. Other people use different methods, and the commercial outfits (some of which are very good but very expensive) won’t tell you what methods they use. The next stage is often to allocate officials, where needed (and they aren’t always needed at an amateur level). Again this can be easy or can be really hard. For a start, officials may not be available all the time; and to make it more complicated, often the organisers will want officials to be allocated to a wide variety of teams, or grounds, during the course of the competition. Sometimes teams of officials are scheduled together, as a team, but in other cases teams are deliberately changed so as to give these officials a lot of different partners in a season. Monogamy or polygamy, you could say. Some officials are better or more experienced than others, which will affect how they should be used. Travel is always an important consideration, especially where finances are tight. Again, the maths won’t work and the O.R. person needs to come up with some other way of producing the goods. Well, that’s the easy bit done. Now the competition just needs to be played. After which, people will come up with new ideas about what was fair, what wasn’t, what could be done better, how things ought to be reorganised – and so the O.R. work can start afresh next year. It all makes work for the working man to do, and the scheduler’s work is never done! Mike Wright, Professor in Lancaster University’s Department of Management Science, has enjoyed a long and very fulfilling collaboration with the England and Wales Cricket Board, scheduling the county cricket fixtures and allocating umpires. His other work has encompassed further scheduling applications in cricket at both professional and amateur level, as well as branching out into Rugby Union and Basketball. Palgrave have just published his Operational Research Applied to Sports. He supports Lancaster City Football Club – well, somebody has to.

IMPACT | AUTUMN 2015

Image © LLamasoft

L L A M A S O F T – S U P P LY C H A I N BY D E S I G N VICKY FORMAN

GLOBALLY, SUPPLY CHAINS are becoming more and more complex as procurement seek to find the new lowest cost producer and sales and marketing are always looking for the latest up and coming market to sell in. This has led to an increasing number of supply chains, not just crossing country borders, but now covering multiple continents. This creates ever greater challenges for those of us in the supply chain business: to make sure these networks are fulfilling the customer needs and are cost efficient.

current distribution network with a new online oﬀering. Or maybe a new product is being introduced and management wish to understand in which of their existing facilities the new product should be produced to minimise the cost of the introduction and the overall impact on the network. These are typical of the sort of projects in which LLamasoft ﬁnd themselves involved.

WHO ARE LLAMASOFT?

Geographic spread is just one aspect of supply chain complexity. Maybe there has been a merger between two companies and the new management wish to rationalise the new combined network to cut costs. Maybe a retailer is looking at how to best utilise their

IMPACT © THE OR SOCIETY

It may come as a surprise, but LLamasoft is not in the llama business! Nevertheless, though we are a supply chain design ﬁrm we still like to put pictures of llamas in all of our presentations and articles. Founded in 1998 in Ann Arbor, Michigan, LLamasoft, Inc. provides

software and expertise to help organisations design and improve their supply chain network operations. LLamasoft Supply Chain Guru® enables companies to model, optimise and simulate their supply chain operations, leading to major improvements in cost, service, sustainability, and risk mitigation. LLamasoft is dedicated to advancing technology focused on the continuous improvement of enterprise supply chains. The main European oﬃce is just outside Milton Keynes with smaller oﬃces in London and Paris and some remote workers in Netherlands, Luxembourg, and Germany. There is a range of backgrounds in the European team including 3rd Party Logistics, academia, consulting, defence industry, manufacturing and retail.

Optimisation is a great and widely used tool by many businesses with complex supply chains to find the most cost effective way of operating

Globally, LLamasoft is a rapidly growing company with offices on six continents to serve the global supply chain market across many different sectors including retail, automobile, food and beverage, chemicals, consumer packaged goods, and healthcare. LLamasoft invests more than 30% of annual revenue in research and has a large dedicated development staff of both software developers and O.R. scientists. There is also a large team of support and project consultants that guides customers through model building by ensuring the customer gets the best possible product, the

best possible support, and is able to successfully analyse the information to correctly inﬂuence the decision making process.

WHAT DOES LLAMASOFT DO?

The flagship LLamasoft software is Supply Chain Guru, which initially included network optimisation and simulation. Over the past few years inventory and transportation optimisation have been added as well. Data Guru was introduced in 2013 to aid with the process of data cleansing and processing into a Supply Chain Guru model, reducing the time from data to model output and helping with repeatable transparent data processing and model building. Customers of LLamasoft include large multinational companies as well as smaller companies. LLamasoft also has a team dedicated to public health solutions; working with World Health Organisation (WHO), World Bank and USAID among others. There is no such thing as a typical project in LLamasoft. Every customer is different. Some customers might need a week of training to get a team up to speed on the software, others may need several months working very closely with the LLamasoft team to build a model or suite of models that will go into continued use, while still others may only need help to build a one off model to answer a specific business question.

AUSTRALIAN BEER

The most commonly used technique in LLamasoft projects is Network Optimisation. Optimisation is a great and widely used tool by many businesses with complex supply chains to ﬁnd the most cost eﬀective way of

A variety of risks were modelled, allowing the impact of completely or partially moving capacity between facilities in the event of extreme weather and plant closures to be evaluated

operating. Optimisation can be used to answer questions such as: • What’s the best location for warehouses? • Which warehouses should serve which customers? • From where should we source products? • Do we need extra manufacturing capacity? • How much inventory should we hold, and where? Another question that can be answered using network optimisation is how to best utilise a combined network following mergers and acquisitions. A major Australian brewer used Supply Chain Guru speciﬁcally for this purpose. They wished to evaluate the production and distribution network following a period of mergers and acquisitions which had resulted in an increase in volume and change in product mix. They wanted to explore the possibility of balancing and redistributing the production and packing capabilities within the current network while keeping in mind the potential impacts to the network of the occasionally extreme Australian weather. Network optimisation in Supply Chain Guru was used to create a model of their current optimized network and then run scenarios. A variety of risks were modelled, allowing the impact of completely or partially moving capacity between facilities in the event of

IMPACT | AUTUMN 2015

USING OPTIMISATION NOT TO OPTIMISE

It is quite common that high costs, low service level and low vehicle utilisation might not be the result of a suboptimal network design alone but other factors such as lead time variability need to be addressed before a network redesign can take place. These were the variables of a project carried out by the public health team; working with the Ministry of Health (MOH) in a West

African nation who are responsible for the distribution of all medicines in the country.

the Ministry of Health... wished to evaluate whether a decentralised network, holding the stock closer to the clinics, would increase the service levels

Medicines were stored in a central warehouse and then delivered by the MOH to district pharmacies where they were then collected by the local clinics. The number of clinics had grown signiﬁcantly over recent years and the network had not adapted accordingly, leading to suboptimal results. They wished to evaluate whether a decentralised network, holding the stock closer to the clinics, would increase the service levels and where these new stocking points should be.

LLamasoft worked with Supply Chain Management System, a coalition of 13 private sector, non-governmental and faith-based organizations that help organisations like MOH manage their medical networks. A baseline model was created before multiple scenarios were used to determine the impact of adding district hubs to the network. Through these scenarios they were able to identify the optimal decentralised network. They also found that, due to existing supply chain issues, the level of stock outs and poor service would continue in the decentralised network if these were not addressed ﬁrst. These issues need to be addressed and then the MOH has the optimal locations for the three district hubs that are needed in the decentralised network. In this case network optimisation wasn’t just useful for identifying the optimal network but also other ineﬃciencies that needed to be addressed.

IS OPTIMISATION ENOUGH?

Optimisation will provide the mathematically optimal solution but there is no such thing as a perfect supply chain. A supply chain in which the customer is perfectly predictable, the transportation and production times are not subject to delays and stoppages, and the network is not subject to external risks such as natural disasters and political instability, doesn’t exist. Optimisation manages to consider so many possible solutions so quickly because it simplifies reality into average global flows – how much stock flows over a quarter, a year, or several years. It ignores some of the detail of day-today operations – how much variation a business can cope with, and the detailed processes that make the business tick.

IMPACT | AUTUMN 2015

extreme weather and plant closures to be evaluated. The scenarios examined also allowed analysis of the impact of demand and cost changes. With the knowledge gained from the Supply Chain Guru models they were able to reconﬁgure their products and distribution network to increase eﬃciency and improve service levels. Further projects focused on optimising safety stock levels and minimising stock outs.

At LLamasoft we believe that sometimes two is better than one and simulation is a tool that can help an organisation bridge the gap between the mathematical optimum (network optimisation) and the real world. Simulation is still a simpliﬁcation of reality (the deﬁnition of a model), but instead of aggregating demand and production at a global level the inputs are more granular, allowing for detail to be at product or order line level which results in much more detailed outputs.

LLamasoft believe that communicating the results of the model effectively is just as important as building the model itself

One important distinction between optimisation and simulation is that they are prescriptive and descriptive respectively; optimisation will tell you what to do but simulation will not. Once you have found the optimal network configuration with optimisation then you can use simulation to test how it is likely to perform under different scenarios, such as: • Fluctuations in demand • Natural disasters • Industrial action • New product introductions One or more performance metrics are commonly analysed to assess the impact: measures such as customer service level, inventory level and vehicle utilisation. In many of these scenarios it is not just one scenario but a range, for example when looking at demand fluctuations it is unlikely that you would only test for a 15% increase in demand: you would run a series of

scenarios from 5% to 25% increase and analyse the range of results. The process is also often an iterative one, running both optimisation and simulation several times with the results of each informing the inputs of the other.

OPTIMISE THEN SIMULATE

A global pharmaceutical manufacturer had been using SAP APO for tactical production planning, but this process lacked capability for scenario analysis on potential network and sourcing configurations. First, the company built a network optimization model to analyse alternate production plans under multiple conditions, including demand growth and contractions in various regions, different pricing plans (profitability analysis) and new suppliers for critical raw materials. Simulation was used to introduce business logic, lead time variability, and demand variability in order to quantify how susceptible the “optimal” production plan was to unexpected changes. Simulation results showed the actual predicted on-time rate for customer orders, work centre utilisation including

changeovers and down time and lost sales caused by raw material delays and production plan deviations.

SIMULATION AS A BUSINESS PLANNING TOOL

At LLamasoft we don’t only use simulation for analysing the impact of unforeseen events and variability but also as a tool for testing the impact of a proposed change in inventory levels, production strategy, or location and transportation frequency. A European food and beverage manufacturer wanted to understand the impact on their customer service level following a proposed move of production facility for one product category. They ﬁrst created a baseline service level on the current network structure then ran several scenarios varying the safety stock at the ﬁnal distribution centre with the new distribution network. They aimed to maintain the service level currently achieved and understand how much safety stock was now required to compensate for the longer lead time from the new production facility. From this analysis they determined that it was possible to move the

IMPACT | AUTUMN 2015

production without impacting the end customer and with a minimal increase in the required safety stock.

A MODEL IS NO GOOD IF YOU CAN’T COMMUNICATE THE RESULTS

Supply Chain Guru can create great models that have the potential to inform decisions and drive business change but without effective communication of the results to those making the decisions the model won’t drive the intended business change. At LLamasoft we believe that communicating the results of the model effectively is just as important as building the model itself. With most models the modeller and the decision maker are presented with a lot of information and complex relationships between the many model inputs. Optimisation reduces the problem to a selection of mathematical optima obtained via different user

deﬁned scenarios and conditions. This reduction of the problem makes it more manageable to understand and compare the outcomes to aid the decision making process. Optimisation is still a mathematical model and as such is sometimes seen as a black box. A simulation model is descriptive and a much more transparent model to analyse. The rules a simulation uses are transparent and can be followed logically and often include a lot of the detail that the decision makers recognise from the real networks. This makes the communication and acceptance of the model by the decision maker easier. Whether the model is optimisation, simulation or another technique, ﬁnding the right method of communication is very important; it needs to be tailored to the audience. Do they like seeing the numbers or do they want graphs and tables? Do they want all the detail or just a high level summary or maybe just the headline savings number? These

THE JOURNALS OF THE OR SOCIETY Linking research, practice, opinion and ideas The journals of The OR Society span a variety of subject areas within operational research and the management sciences, providing an essential resource library for OR scholars and professionals. In addition to the Journal of the Operational Research Society (JORS) – the ﬂagship publication of The OR Society and the world’s longest-established OR journal – the portfolio includes leading research on the theory and practice of information systems, practical case studies of OR and managing knowledge, and journals taking a focused look at discrete-event simulation and applying a systems approach to health and healthcare delivery. OR Society members receive complete online access to these publications. To learn more about member beneﬁts please visit the membership section of The OR Society website: www.theorsociety.com

www.palgrave-journals.com/orms/

IMPACT | AUTUMN 2015

are important questions to ask while analysing the results. Getting the message right can be the diﬀerence between a successful model and an unsuccessful one.

WHAT DOES THE FUTURE HOLD?

Companies across the globe will continue to seek out savings and eﬃciencies in their supply chains and enter new sourcing and manufacturing markets and need to adapt the supply network accordingly. Computing power continues to improve, leading to more and more powerful machines that will run the models quicker than ever. With continued investment in research, LLamasoft products will need to continue to evolve by adding and extending features to make the analysis and design of supply chain networks easier. Vicky Forman is a supply chain design consultant with LLamasoft

D E V E LO P I N G S T R AT E GY I N T H E P U B L I C S E C TO R NEIL ROBINSON

PERFORMANCE TARGETS may have become a cornerstone of everyday working life, but they remain a source of considerable controversy. In meeting the particular demands of the public sector, the Operational Research community has helped move the dreaded “management by metrics” away from a culture of fear and blame. Depending on which attribution you believe, it was either Albert Einstein or William Bruce Cameron, an American professor of sociology, who ﬁrst observed: “Not everything that can be counted counts, and not everything that counts can be counted.” The remark, originally made more than half a century ago, has come to encapsulate much of the scepticism that dogs the rush to “turn data into decisions”. With metrics and performance indicators seemingly aﬀorded a role in almost every facet of working life, critics are nowadays quick to warn that when we measure something too much we might just end up changing the very thing we are measuring.

They do have a point. There are undoubtedly many things that really cannot be measured. Yet the fact remains that it is hard to bring about improvement without a meaningful basis for comparison. Perhaps the single most serious failing of many performance targets is that they are imposed from the top down. The “unforeseen” effects of such measures are in large part the inevitable consequences of a culture that too frequently compels staff to prioritise an imperfect goal over service to the public. Genuinely effective performance management demands collaboration. It should involve frontline staff, end users and other stakeholders in establishing metrics that are truly constructive. It should take into account outcomes that matter to a variety of interested parties, and it should gauge them accordingly. These and other habitually overlooked concerns were to the fore when Max Moullin, a longtime member of the operational research community, created

OUTCOMES

PROCESSES

CAPABILITY FIGURE 1. THE PUBLIC SECTOR SCORECARD – CORE MODEL

the Public Sector Scorecard, a novel framework for strategy mapping and performance management in the public and voluntary sectors. At the heart of his approach was a simple axiom that sought to encompass the views of both camps in the metrics debate: “All performance targets are ﬂawed – but some are useful.”

BEYOND THE BOTTOM LINE

Max Moullin knows the public sector better than most. He spent his early career working for the Departments of Transport, Health and the Environment and later became a section leader in British Coal’s Operational Research Executive. Now director of the Public Sector Scorecard Research Centre and a visiting fellow at Sheffield Business School, where he was a lecturer for more than 25 years, he speaks from experience. “Most organisations in the public and third sectors struggle with two major problems,” he says. “The first is how to improve outcomes without increasing overall cost. The second is how to develop performance measures that ensure quality without motivating staff to achieve arbitrary targets at the expense of poor service.” The notion of using a scorecard to assist strategic planning was by no

IMPACT | AUTUMN 2015

means new when Moullin set about addressing these issues some 15 years ago. The balanced scorecard (BSC), a management tool designed to present a mix of financial and non-financial measures and compare them to target values, had by then enjoyed growing popularity for almost a decade. There were mounting fears, though, about the BSC’s suitability for notfor-profit sectors. Research in several arenas, including healthcare and local government, would subsequently confirm a tendency to prize financial outcomes above all others – a less than ideal propensity in settings where success is seldom determined purely by the bottom line. “The BSC has always been orientated towards the private sector, with little emphasis on the involvement of service users, risk management or the need to work across organisational boundaries,” says Moullin. “This being the case, it seemed senseless to keep trying to adapt it. What was needed was something much more specifically tailored to the needs of the public and voluntary sectors.”

Measurement and evaluation

The result was the Public Sector Scorecard (PSS), a workshop-based approach driven by diverse inputs and focused on three fundamental considerations: capabilities, processes and outcomes – see Figure 1. “The model at the heart of the PSS has always been simple but powerful,” says Moullin. “The basic idea is that capabilities lead to processes and processes lead to outcomes. That might sound obvious, but it’s amazing how routinely the relationships between these elements aren’t fully appreciated – usually because the essential factors that should make them work are somehow ignored from the outset.” The importance of guarding against this problem is manifest in the first phase of the PSS’s methodology shown in Figure 2. This stage is Strategy mapping, which is centred on a series of interactive workshops attended by a range of stakeholders. Managers, staff and service users work together to discuss desired outcomes – strategic, financial, user-centric and so on – and the processes and capabilities needed to achieve them. A draft strategy map

Clarifying outcomes

Learning from performance measures

Identifying process and capability outputs

Developing performance measures

Strategy mapping

Addressing capability

Service improvement

Strategy mapping

Integrating risk management Re-designing processes

FIGURE 2. THE THREE PHASES OF THE PUBLIC SECTOR SCORECARD

is produced and subjected to feedback, including the addition of risk factors and the optimum means of reducing, mitigating or eliminating them.

Any evaluation of performance without giving thought to risk is incomplete

“The need to integrate risk management and performance measurement is imperative in any high-performing organisation, yet it’s consistently neglected,” explains Moullin. “Any evaluation of performance without giving thought to risk is incomplete. It’s also vital to foster a risk-conscious culture without stifling innovation.” In the second phase, Service improvement, the strategy map is used as a prompt to appraise the effectiveness of different processes in satisfying the desired outcomes. Workshop participants are encouraged to relate their discussions to available evidence or data, supplemented where appropriate by tools such as process maps, systems thinking and lean management. They then address the capability aspects where the critical aim is to ascertain how management can support staff and processes to attain the required objectives. “Avoiding a culture of blame is an absolute priority,” says Moullin. “What’s needed instead is a culture of improvement, innovation and learning. This might mean devoting extra resources to a particular area, boosting staff morale or offering clear leadership. The key is to avoid apparently arbitrary targets that are based neither on agreed outcomes nor on activities that have been shown to produce results.”

The third and final stage, Measurement and evaluation, identifies possible performance measures for each component of the strategy map. Workshop participants examine data quality issues with a view to minimising potential unwanted or “perverse” effects and ensuring costeffectiveness and value for money. “Performance measures don’t have to be strictly quantitative, not least in a public sector setting,” argues Moullin. “In many cases – especially in some capability areas – more qualitative approaches are preferable. If improving partnership working is a primary goal, for instance, then a summary of progress and people’s perceptions of what has been achieved will be much better than recording the number of meetings or some other irrelevant measure.”

THE MISSING LINK

The PSS was originally conceived for an evaluation of an NHS Modernisation Taskforce. At the time – 2001 – taskforces were rapidly becoming all the rage, yet their success or failure was seldom clearly discerned. “There would typically be a report stating that targets had been ‘largely met’, with little impartial evidence to back up the claim,” says Moullin. “That doesn’t really help anyone in terms of learning lessons. You can’t build on an intervention’s strengths or eliminate its weaknesses unless you know its precise impact.” The PSS helped close that gap. Crucially, it highlighted the beneﬁts of involving stakeholders – not just in judging performance but in devising how performance should be judged in the ﬁrst place. “We showed the evaluation strategy should be formulated early in the life of a taskforce so it can assist in moulding

strategies and processes to meet the desired goals,” says Moullin. “We also showed a taskforce needs to be assessed on all relevant factors, including achieving the objectives it has been set, meeting stakeholder needs, delivering value for money, being innovative and providing lessons that can be used more widely within an organisation.” There was, though, another dimension to consider. It was eventually incorporated at the turn of the decade, when Moullin and Dr Robert Copeland, of Sheffield Hallam University’s Centre for Sport and Exercise Science, were asked to evaluate Sheffield Let’s Change4Life, a £10m initiative to reduce obesity in children and families. “Success in the public and third sectors is often defined by a change in behaviour,” says Moullin. “For example, an organisation’s principal objective might revolve around persuading people to improve their health or to act more responsibly – say, by stopping smoking, drinking less or taking more care on the roads. Our challenge with Let’s Change4Life was to integrate into the PSS the factors that influence behaviour change.” The answer lay in the theory of planned behaviour (TPB). Developed by Icek Azjen, a professor of psychology at the University of Massachusetts, the concept holds that intention is driven by three major constructs – classified by Azjen as “attitude”, “subjective norms” and “perceived behavioural control”. For Moullin’s purposes, simply put, it meant adapting the PSS to give weight to the key issues surrounding a change in behaviour – people’s beliefs regarding its significance, their perceived ability to achieve it, the barriers they might face and their attitudes (and those of others) towards it. “Using TPB, we were able to assemble a strategy map that helped

IMPACT | AUTUMN 2015

FIGURE 3 STRATEGY MAP

Let’s Change4Life focus on outcomes, monitor and evaluate performance and explore and understand how change might happen,” says Moullin. “The latter was particularly important, because by providing a pathway to understand how change might and does occur we supplied the missing link between outputs and outcomes for behaviour change.” Stakeholders recognised and appreciated the breakthrough. Carol Weir, Let’s Change4Life’s programme director, described the PSS strategy map as a “great success”, noting: “It visually told the story of Let’s Change4Life, what we were trying to achieve and how. It also helped all those involved, at any level, understand the outcome and process measures the programme was trying to achieve and therefore being evaluated against.” The map (Figure 3) was produced after workshops with the programme

board, operational managers and Sheffield Youth Council. Its first two rows showed the main outcomes required for the project, including better diet and nutrition, increased physical activity, value for money and satisfied stakeholders. The third row featured TPB outputs and outcomes in relation to changing people’s behaviour, and the fourth and fifth rows contained the intended outcomes of the various strands of the initiative. The final two ‘capability’ rows show what the programme needed to achieve to support the individual strands and help them achieve their various objectives. Jayne Ludlam, Sheffield City Council’s executive director for children, young people and families, praised the process for distilling “a complex issue with a complex response” into something everyone, from managers to users, could grasp with ease.

The PSS has helped organisations in the UK and beyond in the 15 years since Moullin first put the idea into practice. It has informed and assessed the strategies of health services and central and local governments and has played a part in addressing concerns such as ethnic minority employment, eating disorders and smoking. Bob Penna, the New York Senate’s former director of research and communications, has described Moullin’s work as “groundbreaking”. “Performance management in the public and third sectors is controversial,” admits Moullin. “When it’s done poorly it can alienate employees and lead to a culture of blame. It’s when staff constantly have to look over their shoulder that they’re most likely to meet targets at the expense of service to the public. “But when performance management is done well it can motivate staff to improve. It can empower them to provide services that are always getting better. And for that to happen you have to measure results across organisational boundaries and ensure ‘buy-in’ from everyone involved. That’s what the PSS is all about.” This, ultimately, is the crux of the matter. In moving performance management from a top-down, blinkered, blame-game approach to a system founded on inclusiveness, cooperation and understanding, Moullin has established the middle ground implied by Einstein’s – or, if you prefer, William Bruce Cameron’s – enduring maxim; and that, however we might choose to measure it, is no mean achievement.

Further information on the Public Sector Scorecard is available at www.publicsectorscorecard.co.uk

IMPACT | AUTUMN 2015

THE REAL MEASURE OF SUCCESS

ALPHA MAIL NEIL ROBINSON

THERE IS A school of thought that says radical innovation is the only way to bring about genuine change. The story of how O.R. transformed German ﬁrm Rhenania BuchVersand certainly lends weight to this claim, but the drama was by no means conﬁned to the scale of the eventual turnaround. In the internet age, with the entire retail world seemingly ours to trawl at our leisure and any purchase just a few clicks or swipes away, it is all too easy to believe direct marketing has had its day. In fact, it remains as relevant and as potentially compelling as ever. Take direct mail – nowadays customarily saddled with the pejorative term “snail-mail” yet still, according

to its supporters, a highly eﬀective marketing tool. Proponents maintain that it gives a better impression of a company, is perceived as a more professional means of communication, is more likely to be read by recipients and makes customers feel more valued. These may sound like rose-tinted views, but there is ample evidence to support them. One recent study suggested almost four ﬁfths of consumers would act on direct mail immediately, compared with only 45% who would deal with email straight away. Even global information services group Experian has acknowledged the enduring worth of traditional strategies, cautioning that companies “run the risk of getting too caught up in all this tech buzz”.

CHALLENGING “BEST PRACTICE”

Rhenania BuchVersand was founded in 1946 by journalist Franz Albert Kramer and publisher Peter Josef Stein. Originally a news service and a printing and publishing business, it first moved into mail order in the early 1960s. By the late 1970s it was supplying customers throughout Germany from its headquarters in Koblenz, and by the mid-1990s it was mailing well over two million catalogues and other materials to the names on its proprietary database. It was also in trouble. In 1996 its trajectory was one of falling sales and tumbling profits. Three of its rivals had cornered more than half of a domestic market characterised by an unpromising combination of maturity, stability-cum-stasis and low margins. Rhenania was still a top-10 player, but this was anything but a guarantee of sustainability. What particularly puzzled senior management was that the firm was experiencing an alarming downturn in spite of following best practice. The accepted approach in the mailorder industry was to send catalogues to clients only if expected revenue was higher than the cost of the merchandise, the fulfilment of the order and the mailing itself. In short, Rhenania was following the standard heuristic of trying to ensure marginal

IMPACT | AUTUMN 2015

sales would exceed marginal costs – and yet the business was failing. Enter new marketing director Ralf Elsner, who, with his background in economics and his experience in quantitative methods, was seen by his superiors as someone who could provide the sort of innovative solution so desperately needed. The urgent task was to augment Rhenania’s customer base and proﬁtability. Elsner would not disappoint those who expected radical thinking; but his vision would also prove divisive.

RALF ELSNER

At the heart of his proposal was a devastatingly straightforward contention: “best practice”, he argued, was actually nothing of the sort. The long-held, universally employed ethos of optimising individual mailings was in truth suboptimal even in the medium term, because over time it led to a shrinking base of active customers and reduced proﬁtability. Using mathematical modelling, Elsner showed that in the long term

it would pay to mail to clients who might be deemed unprofitable from an orthodox, short-term standpoint. “Even though the modelling I used was quite simple, what I was advocating went against all conventional thinking in the mailorder business,” he recalls. “In essence, I was saying that even customers who haven’t ordered for a while or have placed only small orders can contribute to a company’s bottom line. Shortterm losses from low-valued customers can be more than compensated for in the long run, because those very same formerly lapsed or now reactivated customers are more likely to order again in the near future. In addition, more mail volume generates economies-of-scale effects.” The argument was too contrarian for some – foremost among them Rhenania’s then CEO, who flatly refused to implement Elsner’s concept. As a result, perhaps predictably, the downward spiral continued. Rhenania sent 20 catalogues a year to each of its active customers, its faith in established optimisation strategies so complete, so resolutely unshakable, that managers did not even bother to evaluate whether such a level of frequency was genuinely ideal. At the time the firm had the capacity to contact as many as 400,000 customers, yet its rigid adherence to the purported prudence of the traditional approach condemned it to contact only half that number. In tandem, although overall market volume was growing by around 5% annually, Rhenania was losing approximately 10% in sales volumes every year. It was not until 1998, when a new and more open-minded CEO, Frederik Palm, was appointed, that Elsner was finally given his chance. Like his long-

Tradition, though, must inevitably have its limits, as Ralf Elsner demonstrated when he joined German mail-order ﬁrm Rhenania in 1996. Elsner’s use of O.R. would transform Rhenania’s processes and fortunes – but only after a hard-fought battle against scepticism, conventional wisdom and common knowledge.

frustrated marketing director, Palm recognised the dwindling of the active customer base was edging towards the point of no return. Drastic action was imperative. With the situation worsening, there was now every possibility that Elsner’s idea – known as Dynamic Multi-Level Modelling or DMLM – would make or break Rhenania.

RISK AND REWARD

Speaking almost two decades on from those turbulent times, Elsner can afford the luxury of reflecting that his original model was “pioneering yet only moderately sophisticated technically”. Since the value of a particular customer might change tremendously, even within the space of a year, the core aim was to develop a dynamic approach capable of maximising profit over time via several catalogues rather than through single campaigns. “The fundamental idea was that some low-value or low-frequency customer segments that yield shortterm losses are profitable in the long term,” explains Elsner. “This is because a fraction of these low-value customers can be expected to become good customers and because the company can obtain economies of scale per mailing.” With this in mind, Elsner developed a system allowing for the dynamic promotion and demotion of customers across a number of segments. Using a rolling horizon and a three-tier process, DMLM considered a year-long period to optimise the annual number of mailings; to determine customer segments and the number of customers who should receive catalogues; and to identify the economic value of smaller segments, especially those containing

low-value customers who should still be included in mailings. Importantly, DMLM’s secondlevel focus was strictly on recency, not frequency. The company’s database was split into three segments: customers who had made a purchase within the previous 12 months; customers who had made a purchase within the previous 24 months; and customers who had not made a purchase for 24 months or more. Analysis showed the number of promotions and demotions to be virtually constant throughout a given year, with a generally fixed number of customers either buying and therefore moving up or failing to buy and therefore staying put or dropping down. A feasibility study indicated that mailing to all three segments would lead to an increase in profitability of nearly 6%. “We had very little time to conduct large-scale experiments,” admits Elsner. “The company’s economic plight was so serious that we had to act quickly. We ran our first big test series of the basic model in May 1998,

and two months later we began a full run. Only the third-level analysis wasn’t ﬁnalised at this time, yet even then a senior executive from the company’s holding organisation tried to intervene, saying what we were doing went against industry wisdom and was too risky.” One of the complaints Elsner and Palm faced was that the mailing strategy recommended by DMLM was leading to a deterioration in accounting metrics. The measure of productivity of mailing budgets, for instance, set the holding organisation’s alarm bells ringing: costs were rising, but revenues were largely unchanged. Amid all the urgency, all the uncertainty, it was yet another hurdle to overcome. Elsner and Palm could only protest that what was being witnessed at this early stage was to be expected. In some ways, they said, it was even intended. They pleaded for the chance to carry on, insisting that in due course each mailing would produce lower costs and higher earnings per unit.

IMPACT | AUTUMN 2015

They did not have to wait long for vindication. The ﬁrst signs of a positive shift came within just two months. Nobody could deny the turnaround was under way. Rhenania’s proﬁts quadrupled the following year, and the merits of DMLM were never questioned again.

it acquired one of its rivals, Akzente, and applied DMLM to similarly impressive eﬀect; a further acquisition, this time of Mail Order Kaiser, was completed the following year. In 2001 Rhenania’s record of achievements in the immediate wake of introducing DMLM was described by Börsenblatt, a leading industry publication, as “astonishing”.

BUILDING ON SUCCESS

To say DMLM saved Rhenania is no exaggeration. It is certainly difficult to dispute the sentiment when it is expressed by the company’s current CEO – none other than Ralf Elsner. “On the one hand, Rhenania’s success can be attributed to the management’s courage, leadership and willingness to try an innovative and non-traditional approach,” he says. “On the other, the evidence shows we succeeded because of the robustness of the mathematical model we used. All in all, it made for a powerful combination.” The numbers speak for themselves. In the summer of 1998, when DMLM was first used, the firm was the fifthlargest of its kind in Germany; by 2001 it was the second-largest. The customer database grew by more than 55% between April 1998 and 2000. Rhenania started to consistently outperform the market, having underperformed it for years. The project paid for itself within weeks. Moreover, the company did not just strengthen economically: it also gained a competitive edge. In December 2000

in the age of big data, mathematical modelling has the power to help smaller companies gain a valuable competitive advantage

Since then DMLM has been extended and modified to leverage Rhenania’s customer base even more effectively. Refinements have included the further differentiation and segmentation of inactive customers, leading to better response and reactivation rates. The model continues to serve as an excellent forecasting tool, presaging the development of active customers, sales and profits. It is also a key component of Supply and Marketing Integration (SMI), a seven-stage, profit-maximising process that Elsner began working on in 2004. SMI was designed to go far beyond DMLM’s focus on customer selection, covering the whole decision and production chain to address a vital

issue in the sphere of direct marketing: finding an optimal merchandise mix based on demand. As with DMLM, it was a matter of challenging accepted industry practice. Pricing, variety, inventories and catalogue content had long been decided principally on management assessments and experience. SMI championed a much more scientific approach for the first time– one capable of establishing the profitmaximising pricing of an entire catalogue, as opposed to the individual items contained therein. Again there was resistance. Purchasing department staff and catalogue editors were notably reluctant to embrace SMI, doubting that the use of O.R. could improve on the merchandise mix suggested by their own know-how. The onset of the global financial crisis was instrumental in persuading the naysayers, and by 2009 – the year after SMI’s adoption – total profits were three times greater than in 2007. Today, with the influence of both DMLM and SMI still absolutely central, Rhenania remains in good health – proof, believes Elsner, of O.R.’s ability to make a lasting difference to a business. “O.R. isn’t a matter of size,” he says. “Especially now, in the age of big data, mathematical modelling has the power to help smaller companies gain a valuable competitive advantage. I think our story demonstrates that very well.”

If you wish to investigate the mathematical models developed by Elsner see Elsner, Ralf, Krafft, Manfred and Huchzermeier, Arnd (2003) Optimizing Rhenania’s Mail-Order Business Through Dynamic Multilevel Modeling (DMLM), Interfaces 33(1), 50 - 66 and Elsner, Ralf (2004) Optimizing Rhenania’s Direct Marketing Business Through Dynamic Multilevel Modeling (DMLM) in a Multicatalog-Brand Environment. Marketing Science 23(2),192–206).

IMPACT | AUTUMN 2015

WRONG NUMB3RS Geoﬀ Royston

According to a “Skills for Life” survey commissioned by the UK Department for Business Innovation & Skills, nearly half the working-age population have numeracy levels at or below the level expected of a child leaving primary school. A survey by the Nationwide Building Society of young teenagers found that more than half could not work out the correct change from £100 for a bill of £64.23. The UK came 26th in the last OECD international assessment tests of maths proficiency amongst 15 year-olds, behind, for example, Estonia, Poland and Vietnam. Low numeracy amongst the UK population has serious consequences at individual and national levels. At individual level it can lead to problems ranging from getting a job to understanding information about health, and indeed with dealing with life generally. At national level, a study by Pro Bono Economics estimated that low numeracy is associated with costs to the UK economy of some £20billion per year. And it makes for easy pickings for politicians, journalists and other repeat offenders when it comes to confused thinking or crooked talking about numbers, damaging the civic fabric. This may be why public perceptions of key facts and figures can be so wrong – for example an IPSOS MORI survey found that members of the public thought the teenage pregnancy rate was 15% (actual, 0.6%), benefit fraud was £24 in every £100 (actual, around 70p) and a quarter of the UK population were Muslim (actual, one-twentieth). Problems with mathematical literacy are not confined to the less highly educated part of the population. For example, what are the chances that your doctor will give you accurate assessments of your health risks? In his (very

readable) books “Reckoning with Risk” and “Risk Savvy” the psychologist Gerd Gigerenzer describes an experiment in which he presented a large group of experienced medics with a question along the following lines: “About 1% of women in a certain age group have breast cancer. If someone has this condition the probability that they test positive for it on a diagnostic screening is 90%. If they do not have the condition the probability that they nevertheless test positive for it is 9%. Given just this information, what is the best estimate of the chances that someone in the age group who tests positive actually has breast cancer: 9 in 10, 8 in 10, 1 in 10, or 1 in 100?”. Three-fifths of the clinicians said 8 or 9 in 10, a fifth said 1 in 100 and the other fifth got the correct answer, of about 1 in 10. Many (apart from those who were just guessing!) were confusing two superficially similar but actually very different questions – “what are the chances that someone with the condition will have a positive test” (9 in 10) and “what are the chances that someone with a positive test has the condition?” (1 in 10; out of 100 people in the group, 10 will test positive, of whom only 1 will have the condition). Similar confusion happens in courts of law e.g. equating the chances that someone who is innocent has a DNA match (say, one in a million) with the chances that someone with a DNA match is innocent (if this was the only evidence, as high as 59 in 60, as out of 60 million people in the UK there would be about 60 matches). This confusion - known as “the prosecutor’s fallacy” - has contributed to wrongful convictions. In a notorious example, a mother was convicted of murder and jailed following the cot deaths of two of her children; partly because an expert witness had testified that the chance of two children from an affluent family suffering sudden infant death syndrome was less than 1 in 70 million. That was not only an erroneous calculation (the number was arrived at by squaring a 1 in 8500 chance of a single cot death in similar circumstances, a calculation that could be appropriate only if the two events are independent, ignoring the possibility of, for example, there being a genetic predisposition to cot death in a family) but more importantly was also, as in the DNA example, completely the wrong calculation to put before the jury, the more appropriate question being, in the rare occurrence of a double cot death, how often does it involve child abuse? The answer is in about one in three cases. Another common problem is confusing relative and absolute risks. The oral contraceptive pill scare of the mid 1990s arose from a narrow focus on relative risk.

we have radio programmes like “More or Less” with Tim Harford and TV programmes like “School Of Hard Sums” with the comedian Dara O’Briain and Marcus du Sautoy, Oxford University Professor for the Public Understanding of Science, and websites like www.nationalnumeracy.org.uk and www.understandinguncertainty.org (founded by David Spiegelhalter, Professor for the Public Understanding of Risk at Cambridge University, who gave an illuminating and entertaining talk at the OR Society’s annual Blackett lecture in 2013). Finally, what about incentives for and recognition of improvement? The Royal Statistical Society commendably introduced an award for statistical excellence in journalism. Time perhaps for some “fostering numeracy” awards for politicians, doctors, lawyers, managers and others involved in areas where we all need to avoid getting a wrong number.

Dr Geoﬀ Royston is a recent president of the O.R. Society and a former chair of the UK Government Operational Research Service. He was head of strategic analysis and operational research in the Department of Health for England, where for almost two decades he was the professional lead for a large group of health analysts.

WRITING FOR IMPACT

The editorial team at Impact are always happy to hear from interested readers who would like to write for the magazine. Any stories of the analysis of real-world decision problems and their impact are welcome – please email Graham Rand: g.rand@lancaster.ac.uk

IMPACT | AUTUMN 2015

Covers courtesy of Penguin Random House

It was reported in the press that, compared to an earlier generation of pills, the latest ones carried double the risk of a, sometimes fatal, thrombosis. It was not reported that the annual risk of a fatal thrombosis was still tiny, rising from about 2 in a million to around 4 in a million. A sizeable percentage of women stopped taking the pill, and there was an estimated increase of 13 thousand abortions the following year. These examples illustrate an important general point; numeracy is not just about doing calculations right, it is also about understanding the right calculations to do. Both are things that analysts should be good at, but analysts, engineers and project managers are not immune from making mistakes! You may recall for instance the errors, costing about £40m, in the economic assessment of bids for the West Coast Main Line rail franchise; the swaying Millennium bridge, where the effect of synchronised footfall had been computed in the vertical direction but not side-toside; or the total loss of a Mars spacecraft because one team used imperial units and another used metric. Naturally, for readers of Impact, a key question will be what can be done to improve matters. Formal education would be the obvious first port of call, and there must surely be something to learn from the countries that were in the top five in the international assessments - Singapore, Hong Kong-China, Chinese Taipei, Korea, and Macao-China. But what about less formal pathways? For example there are some excellent books. A classic, over fifty years old but still remarkably fresh, is Darrell Huff’s “How to Lie with Statistics”. Two others, both by John Allen Paulos, are “Innumeracy” and “A Mathematician Reads the Newspaper”, a personal favourite that I (not a mathematician) think should be read by every politician, journalist, student and indeed pretty well everybody. Much the same could be said about “The Tiger That Isn’t” by the journalist Michael Blastland and Andrew Dilnot, Chair of the UK Statistics Authority and about Jordan Ellenberg’s, “How Not to be Wrong”, recently out in paperback. Of course books are not the only popular medium that can be employed to improve mathematical literacy -

Why do over 70% of Fortune 50 companies use SIMUL8 simulation software?

Powerful. Flexible. Fast.

Some of the world’s most successful organizations rely on SIMUL8 simulation software because it helps them to make and communicate their most important decisions. Don’t rely on spreadsheets to be your process improvement toolkit. Make bold, confident decisions because you have the evidence to be sure you are making the right choice. SIMUL8 has helped Industrial Engineers for over 20 years – saving money, reducing waste and improving efficiency.

Simulation software for innovative Industrial Engineers www.SIMUL8.com