Impact Magazine Spring 2015

D R I V I N G I M P R O V E M E N T W I T H O P E R AT I O N A L R E S E A R C H A N D D E C I S I O N A N A LY T I C S

SPRING 2015

R E P L AC I N G T H E LO N G H AU L F L E E T When British Airways needed a major fleet replacement, an O.R. model utilised hugely complex data to give an informed decision

Image Š Rufus Stone / Alamy

A N A LY T I C S A N D YO U R CREDIT CARD

WO R K F O R C E PLANNING IN THE NHS

Every decision a bank makes System dynamics helps determine about your credit card is our need for doctors and dentists, guided by analytics and deliver substantial savings

E D I TO R I A L Welcome to the first issue of Impact, the magazine of the OR Society. Welcome to the magazine itself, now published after a lengthy gestation, and welcome to you, a reader. Impact is aimed at those who are interested in the impact that can be achieved by the use of an analytical approach to decisions. We believe that this will not only include practitioners, whether members of the OR Society or not, but also those who may see a potential use for the sorts of methods whose implementation is described within the magazine.

In this first issue you can read about how British Airways determined which planes to buy, how ambulance stations were relocated in Sao Paulo, how the NHS was helped to determine how many doctors and dentists should be trained, how analytics supported the coalition’s armed forces in Afghanistan and how credit card decisions are made. You can also read about the work of the decision support organisation, CORDA. The production of any issue of a magazine doesn’t happen without a great deal of hard work, not only on the part of the writers. This is particularly true of a first issue, so many thanks are due to Jane Torr and Paileen Currie at Palgrave for overseeing the production of the magazine, and creating its design. The OR Society is grateful to Palgrave for their support in this venture.

Now it’s in your hands. I hope you enjoy reading it, of course. If you have suitable stories of the efficacy of analytics, please let me know. If you think that approaches, such as those described within the magazine, will be helpful to your organisation, please contact the OR Society, who will be able to advise you.

Graham Rand

The OR Society is the trading name of the Operational Research Society, which is a registered charity and a company limited by guarantee.

Seymour House, 12 Edward Street, Birmingham, B1 2RX, UK Tel: + 44 (0)121 233 9300, Fax: + 44 (0)121 233 0321 Email: email@theorsociety.com Secretary and General Manager: Gavin Blackett President: Stewart Robinson (Loughborough University) Editor: Graham Rand (Lancaster University) g.rand@lancaster.ac.uk

Print ISSN: 2058-802X Online ISSN: 2058-8038 Copyright © 2015 Operational Research Society Ltd Published by Palgrave Macmillan Printed by Latimer Trend This issue is now available at: www.issuu.com/orsimpact

OPERATIONAL RESEARCH AND DECISION ANALYTICS

Operational Research (O.R.) is the discipline of applying appropriate analytical methods to help those who run organisations make better decisions. It’s a ‘real world’ discipline with a focus on improving the complex systems and processes that underpin everyone’s daily life - O.R. is an improvement science.

For over 70 years, O.R. has focussed on supporting decision making in a wide range of organisations. It is a major contributor to the development of decision analytics, which has come to prominence because of the availability of big data.

Work under the O.R. label continues, though some prefer names such as business analysis, decision analysis, analytics or management science. Whatever the name, O.R. analysts seek to work in partnership with managers and decision makers to achieve desirable outcomes that are informed and evidence-based. As the world has become more complex, problems tougher to solve using gut-feel alone, and computers become increasingly powerful, O.R. continues to develop new techniques to guide decision making. The methods used are typically quantitative, tempered with problem structuring methods to resolve problems that have multiple stakeholders and conflicting objectives.

Impact aims to encourage further use of O.R. by demonstrating the value of these techniques in every kind of organisation – large and small, private and public, for-profit and not-for-profit. To find out more about how decision analytics could help your organisation make more informed decisions see www.scienceofbetter.co.uk. O.R. is the ‘science of better’.

OR ESSENTIALS Series Editor: Simon J E Taylor, Reader in the Department of Computer Science at Brunel University, UK

COMING SOON:

The OR Essentials series presents a unique cross-section of high quality research work fundamental to understanding contemporary issues and research across a range of operational research (OR) topics. It brings together some of the best research papers from the highly respected journals of The OR Society.

Operational Research Applied to Sports Mike Wright | August 2015 Operational Research for Emergency Planning in Healthcare Navonil Mustafee | October 2015

If you would like to submit a proposal for the series, please email: Josephine.Taylor@palgrave.com and simon.taylor@brunel.ac.uk

www.palgrave.com/series/or-essentials/OREss/

CONTENTS 6

REVIEWING THE FLEET Brian Clegg tells the story of how O.R. contributed to BA’s decision on which planes to purchase

14

A DAY IN THE ANALYTIC LIFE OF A CREDIT CARD Andrew Jennings of FICO demonstrates how pervasive the use of data and models have become to decision making in banking

22

STUNG INTO ACTION

CORDA - DELIVERING SUCCESSFUL FUTURES A leading defence decision support organisation reveals all – well, perhaps not quite all!

33

A HEALTHY OUTLOOK

12 Making an impact

Mike Pidd reflects on the links between universities and organisations

Brief reports of two postgraduate student projects 26 Predictions of elections

Graham Rand wonders whether forecasters will be celebrating on May 8 43 Engineering happiness?

Neil Robinson shows that a recent review of medical training in the UK has demonstrated that system dynamics can prove to be a powerful and cost-effective tool

38

Analytics making an impact

20 Universities making an impact

Neil Robinson tells how bees have helped to reduce ambulance response times in Sao Paulo, Brazil’s most traffic-plagued city

28

4 Seen elsewhere

Will Geoff Royston bring a smile to your face, and will you be able to measure its size?

SHAPING PEACE SUPPORT OPERATIONS IN AFGHANISTAN (2011) Brian Clegg describes how a sophisticated war-gaming technique has enabled an Operational Analysis team to give the best possible support to operational decision-making in a theatre of war

INTRODUCING BRIAN CLEGG

Impact will be using the services of several professional writers. Amongst these, Brian Clegg has a very relevant pedigree, as his career started in Operational Research. After graduating with a Lancaster University MA in Operational Research in 1977, Brian joined the O.R. Department at British Airways. The acknowledgements in his book Dice World begin with ‘particular thanks to Keith Rapley and the late John Carney, my mentors when working in operational research – arguably the ultimately discipline for applying probability and statistics to real world problems’. The work he did there was heavily centred on computing, as information technology became central to all the OR work he did. He ended up running first the new PC Centre and then later a group investigating emerging technologies. He left BA in 1994 to set up a creativity training business. Brian is now a science journalist and author and he runs the www.popularscience.co.uk and his own www.brianclegg.net websites.

SEEN ELSEWHERE ANALYTICS JUST KEEPS ON GROWING

The top five analytics software companies, SAP, IBM, SAS, Microsoft and Oracle made some $7.5 billion or 55.5% of business intelligence and analytic tools world sales according to IDC’s Worldwide Business Analytics Software 2014–2018 Forecast and 2013 Vendor Shares. SAP retained its leadership position of this market segment, with a 17% share.

This market segment is made up of software tools focused on such functions as production report development, dashboarding, visual discovery, advanced and predictive model development, model execution, multidimensional analysis, content analytics, and spatial information analytics.

The annual growth for the worldwide market through this fiveyear period is expected to be 9.4%.

However, the forecast growth rates of individual segments of the market vary, and are as high as 16.9% for content analytics tools: from $950 million in 2013 to over $2 billion in 2018.

ROBUST PLANNING OF BLENDING ACTIVITIES AT REFINERIES

A recent paper in the Journal of the Operational Research Society, and the winner of the Society’s Goodeve Medal for the most outstanding paper in the 2013 volume, demonstrates that there is still more to learn about a classic example of the use of O.R. techniques, and linear programming (LP) in particular. The optimisation of blending crude oil products so as to meet demand and maximise revenues has been

4

IMPACT © THE OR SOCIETY

investigated for many decades. Refinery operations planning is a complex task since refinery processes and inventories are tightly interconnected and ships arriving with crude oil for processing have uncertain arrival times.

An important input in short-term planning is a delivery plan where product quality, quantity and expected delivery times for each crude oil tanker destined for the refinery are specified.

Whilst the product quantity and quality are known fairly accurately, the arrival times are subject to uncertainty. For the first time Norwegian researchers have successfully combined this uncertainty into a robust optimisation approach.

They then used a simulation case study to demonstrate that this methodological innovation out-performs conventional deterministic approaches. The outcome is a lower average stock level of inrefinery products and also fewer stock-outs of essential components for blending, and so a more profitable operation. This development will be of great interest to both researchers and practitioners. Further information can be found in: J Bengtsson, D Bredström, P Flisberg & M Rönnqvist (2013) Journal of the Operational Research Society 64, 848–863.

INFORMS PRESIDENT CONSIDERS THE SHORTAGE OF ANALYTICS PROFESSIONALS

Robin Keller, incoming president of INFORMS authored an interesting article titled, “How we can grow analytics intelligently in the year ahead.” Published on November 11 in DataInformed to coincide with the INFORMS Annual Meeting in

San Francisco, the article examines the expected shortage of analytics professionals from the supply side perspective (academics and business leaders).

Her recommendations are that: • analytics programs at business schools and in other university departments synchronize their efforts with industry as different fields express their analytics needs.

• current analytics professionals should be trained in new techniques.

• a certification process should be available that shows employers that candidates are trained to their needs (and allows analytics professionals to demonstrate their expertise).

• simple ways for organizations using analytics should be available to tell if they are on target with their planning.

INFORMS offer continuing education, a Certified Analytical Professional programme, and have recently launched an Analytics Maturity model (see elsewhere in Seen Elsewhere).

EXPERTS ARE TURNING TO “BIG DATA” TO BRING THE EBOLA EPIDEMIC UNDER CONTROL

Mobile phone calls, airline bookings, tweets, field reports, government announcements and population statistics are among the vast amount of information being collected, filtered and analyzed by sophisticated computer software tools around the world.

The information is enabling data mining experts to predict where the virus could be headed next and how many people are likely to be infected.

Swedish non-profit organization Flowminder, for example, has been using cellphone data to map population movements in West Africa.

Such intelligence gathering is helping governments and health agencies in West Africa respond more quickly and effectively to the crisis.

In Senegal, for example, local cellphone carrier Orange Telecom supplied Flowminder with data from 150,000 handsets, which it was able to use to develop maps of travel patterns in the region and help the government and health agencies anticipate the trajectory of the virus. The data had been stripped of information that could identify the user. Senegal is now officially Ebola-free.

HealthMap is an infectious diseases tracking system developed by a team at the Boston Children’s Hospital, who have been praised for raising the alarm about Ebola days before the WHO officially announced the outbreak.

The system uses computer algorithms to scan thousands of websites, such as news, social media and government websites, for mentions of infectious disease outbreaks around the world. It then presents the data in easy-to-digest formats, such as maps and graphs.

During the Ebola crisis, HealthMap has been using data from the WHO to show when and where people died from the virus. It has also modelled the spread of the outbreak in Guinea, Liberia and Sierra Leone, which have been hardest hit by the epidemic, and estimated the number of future cases.

INFORMS HAVE LAUNCHED AN ANALYTICS MATURITY MODEL

to help organizations evaluate the way they are using analytics across their business, and determine how they can improve their best practices over time. Whether an organization is in its initial stages of using analytics to improve business processes or is at a more advanced level, the INFORMS Analytics Maturity Model approach helps assess strengths and weaknesses across an organization in three important areas of analytics: the organization itself, its analytics capabilities and its data and infrastructure.

According to an INFORMS survey the concept of “analytics maturity” is important or very important (65%).

Yet, 82% of the same respondents admitted they do not have any mechanism in place for measuring the efficacy or maturity of their analytics best practices over time.

“The ability to fully assess the maturity level of an organization’s analytics best practices is paramount to their efficacy,” says Aaron Burciaga,

senior manager, operations analytics at Accenture. “With more access to information than ever before, organizations must have a strategy in place for how they leverage data and analytics, and assess the maturity of their programs to empower decision-making and drive organizational strategy.”

Other survey findings include: • Of organizations that do have an analytics maturity model or process in place, 72% are only assessing their analytics practices by certain departments, not holistically.

• 52% of those respondents said they have not made any changes to their approach or best practices based on the input they have received from the analytics maturity model they are using.

• 80% say the model they are currently using does not allow them to benchmark their analytics maturity against their peers.

TWITTER’S WHO-TO-FOLLOW

system is an algorithmic data product that recommends accounts for Twitter users to follow. O.R. and analytics techniques played a key role in building the system.

This product has had significant direct impact on Twitter’s growth and the quality of user engagement, as well as being a major driver of revenue.

Over 1/8 of all new connections on the Twitter network are directly due to this system, and a substantial majority of Twitter’s revenue comes from its Promoted Products, for which this system was foundational.

To place this contribution into perspective, Twitter is now a publicly traded company with a market cap over $30 Billion, a projected annual revenue close to $1 Billion, and over 240 Million active users.

More detail can be found in Interfaces Jan-Feb 2015, pp98­–107.

IMPACT | SPRING 2015

5

REVIEWING THE FLEET BRIAN CLEGG

WHEN THE BRITISH AIRWAYS planning department needed a major fleet replacement, a model developed by O.R. cut through hugely complex data to give an informed decision.

Last year, the British Airways Operational Research group celebrated its sixtieth anniversary. This might seem poor arithmetic, as the airline only came into existence in 1974, but separate O.R. groups played a role in BA’s predecessors BOAC and BEA before the merger. Though the BA O.R. group has gone through a number of organizational structures and sizes, it has always played an important role in supporting the airline.

FLEET REPLACEMENT DECISIONS

Image © British Airways

One of the biggest and most complex decisions any airline faces is

6

IMPACT © THE OR SOCIETY

replacing parts of the fleet. The driver for the latest decision was the need to replace the 747 fleet which is advancing in years. These long haul planes, covering routes outside Europe, were first introduced by Boeing in the 1970s and since this time more fuel efficient and technologically advanced aircraft have been introduced to the market.

The initial selection was made in 2007, when orders were placed for the first 36 planes, a mix of the huge, double-decker Airbus A380 and the smaller Boeing 787 ‘Dreamliners’.

The decision of how to replace the remaining 747s was made several years later. Choosing a replacement may seem a simple cost/benefit analysis, but according to Neil Cottrell, Head of BA’s planning department, the factors involved are messy, multi-dimensional and difficult to pull together, requiring a more sophisticated method.

Cottrell has a background in Operational Research. After taking a degree in mathematics, he worked in O.R. at the National Coal Board

and joined BA in 1985, moving from O.R. to play a range of roles in operations, commercial and strategy before becoming Head of Planning in 2010. This department of about 50 people is responsible for the future of the airline’s network, fleet and airport infrastructure, typically covering the period from six months to 20 years out.

As well as the straightforward costs and potential for revenue of the fleet decision, depending on varying seat configurations, the planners had to factor in the impact on infrastructure – would the new planes even fit with existing airport stands and maintenance bays? – the time and effort required in familiarisation, the aircraft’s performance and range, flexibility, environmental impact and more.

Getting the decision right was essential.

The planning department at BA has a long working relationship with Operational Research and realised that a decision of this complexity would benefit from O.R. input.

Andrew Long, Principal Operational Research Consultant at BA headed up the project from the O.R. side. Long followed a degree in Management Sciences and Accounting with an MSc in Operational Research at Southampton and joined BA from university in 2006.

According to Long the decision involves strongly interlinked factors: ‘The benefits obviously depend on how many passengers we think we will carry, and that depends on where we fly, which aircraft we fly and how many seats those aircraft have. So, for each of the 170+ flights we needed to decide between 9 aircraft types, each of which could have up to 15 different configurations, for both the summer and winter season. We had to do this for each year over the life of the aircraft, which is typically 25 years.

Whilst some of the costs are relatively straightforward to model, others are dependent on passenger volumes, aircraft type or aircraft utilisation.’

Neil Cottrell again: ‘The last big fleet decision would have been the purchase of 777s in the late 1990s which was, I think, rather more adhoc. We then went ten years without a big long haul fleet change before in 2007 making the decision to replace the first batch of 747s. It was then that we moved into using modelling techniques of a less ad-hoc kind. The 2013 order from BA’s parent company International Airlines Group (IAG) builds on the work done then.’ Where the planners had expertise in looking ahead over the timeframe required for the changes an airline has to face, O.R. could bring specialist analytical, modelling and problem solving skills.

MODELLING THE AIRLINE

The way the model was developed reflects the evolutionary nature of O.R. itself. In the early days of Operational Research, most of the work was done with pen and paper and there was noticeable suspicion from much of the community about use of computers.

The British Airways O.R. department was one of the first to take the plunge, not just using computers to churn out the results from a mathematical model, but building complex systems that the user departments could interact with directly.

For a long time the department had its own dedicated mainframe, a DECsystem-10, which, when introduced in the early 1970s, provided a radical new approach to interacting with computer systems. Apart from expensive bespoke developments like airline reservation systems, computing at the time was largely performed in

batch operations, where a program would be constructed as a stack of cards, which were fed into a computer, processed and an output produced, a routine that could take several hours or even days.

there was noticeable suspicion from much of the community about use of computers

With the DEC-10, once a program had been written, users could interact with it directly, initially using teletype terminals and later VDUs, screens and keyboards that allowed for an instant response to typed commands, not unlike the experience of running DOS on early PCs, though the processing and storage capabilities of the DEC10 far exceeded the first personal computers.

If the O.R. department back then had been faced with the same challenge, they would have written a bespoke program from scratch to deal with the fleet decision. It would have coded line by line in the NELIAC programming language (a variant of ALGOL, produced by the US Marine Naval Experimental Laboratory) that was run on the DEC-10.

Skip forward to 2005 when the new project began, though, and the tool of choice was an Excel spreadsheet, though admittedly making use of some internal VBA coding to link to external off-the-shelf software called lp_solve that runs one of Operational Research’s trademark techniques, linear programming. The result was the Falcon model, written initially to support the 2007 decision and significantly updated for the latest acquisition round.

IMPACT | SPRING 2015

7

Linear programming is an optimisation technique – a method of searching for the best outcome in a set of data. It tries to maximise (or minimise) a set of values given a number of constraints – in effect limits that keep variables within a fixed range. When a linear programming model runs, it discovers the best combination of factors within the constraints. A simple example might be to maximise the profit from using a fleet of different sized vans to transport a set of varying sized and

8

IMPACT | SPRING 2015

value cargoes. Linear programming would tell you which van or vans to use for each cargo consignment.

The classic approach to solving a linear programming problem is the simplex algorithm, a mathematical methodology that considers the possible space of solutions in multiple dimensions. These spaces look a little like a polyhedron with many, complex shaped sides, though there may be many more than the familiar three dimensions. To use the simplex algorithm, the analyst has to first

discover one of the vertices of the shape and then the program will effectively work along the edges of the polyhedron – cutting out much of the solution space – until it finds the vertex that represents the optimum solution.

This mathematical manipulation, taking place in numerical matrices, is hidden away in the model, which handles the whole British Airways network. By using historical data to pull together revenues and costs – effectively the profit per seat – the Excel-based model is able to recommend the best deployments of the available aircraft options.

Just as the technology can be used in different ways, the relationship between O.R. groups and their clients has varied with time. In the early days of Operational Research the ‘boffins’ were a breed alone, an oracle of information.

Users of their services would present a problem and be given a solution that was totally mysterious with no real points of reference. More recently it has been common for models to be produced by an O.R. group and run by the users, making for much closer working relations.

It wasn’t realistic for the planners to become expert users of Falcon. Where, for instance, those working in yield management to maximise the day-today return from each individual flight will be interacting with their systems every day, fleet purchase decisions are less frequent. Even so, the modelling process was highly collaborative.

The planning department pulled together the data and requirements while the O.R. team built the model and tested it. There was then an iterative process of reviewing the outputs and refining the model until both planning and O.R. were happy that they were achieving best results.

Initially this was a lengthy task, with a

Image © British Airways

SIMPLY THE BEST

Image © British Airways

full set of scenarios taking three people as much as five days to run them, but as the process became slicker it was condensed to taking two people less than two hours.

like a lot of O.R. applications, it’s not just modelling, it’s understanding what the models tell you and knowing what questions to research

How to achieve best working between O.R. and users is always a difficult decision. It might seem that the best results could be delivered if the O.R. team were embedded in the planning department, but experience has shown the BA group that they can work more productively by operating separately, with more opportunity to share expertise across the group. As it happens in this case the team was only located 20 metres from the planning department, so arguably had the best of both worlds.

To improve the understanding of the clients’ requirements, the British Airways O.R. group also encourages staff to be seconded into client departments, and in this case one of the team working on the model spent over a year seconded to planning, proving a real benefit in achieving an effective outcome. Neil Cottrell: ‘I think it’s not just having the modelling skills but also knowledge of the business area, having business acumen, is really important, because like a lot of O.R. applications, it’s not just modelling, it’s understanding what the models tell you and knowing what questions to research. The more people understand about the business,

the better they are at asking the right questions.’

FUTURE GAZING

The problem, of course, with planning was neatly summed up by physicist Niels Bohr who made famous the quote ‘Predictions can be very difficult – especially about the future.’ In the case of predicting future requirements of an airline, it is impossible to be sure of the external factors that will come into play, and so the model was run making use of various scenarios.

Neil Cottrell: ‘These aircraft are expected to fly for the next 20 years – you can’t be accurate to the nth degree.

What we’ve tried to do here is to look at a number of different scenarios, by varying several factors. One of the things we’ve done with the results is what I’d describe as a heat map. From the different combinations of factors

that have been varied you can produce a heat map where bright red is aircraft X is better by a long way, bright green is aircraft Y is better by a long way, and then you’ve got various interim shades depending on how close the competition is.’ This approach allowed the planners to take into consideration external variables over which they had no control, but that could have a significant impact on the profitability of the airline and the best choice of aircraft for the fleet.

Forecasting a complex future with a set of partially independent variables is a familiar problem in meteorology.

The big revolution in weather forecasting that has taken place over the last few years has been to recognise that such a complex and chaotic (in the mathematical sense) system is impossible to forecast accurately because very small variations in starting

IMPACT | SPRING 2015

9

conditions of the model can result in big changes in the results.

What weather forecasters now do is to take an ‘ensemble’ approach, running the model many times with subtly different inputs and producing probabilities for the potential outcomes, depending on how frequently they emerge. The position is different for BA, where there is a better idea of a likely scenario, and so rather than adopt a probabilistic approach (which is always difficult to sell to senior management), the analysis starts with the most likely scenario and then includes a sensitivity analysis, seeing if and how the recommendations are changed by, for instance, increasing oil prices.

Long points out ‘One key thing to note is that we use the model to measure relative differences between scenarios, not absolute figures. For example, we are less concerned that Scenario A will be showing a contribution of £Xbn a year and Scenario B £Ybn contribution a year, and more interested in the fact that there is a £(X-Y)bn difference.’ Cottrell adds ‘We are trying to get a general picture across a range of scenarios, rather than one forecast which we know will be wrong.’

CHECKS AND BALANCES

There is always a difficulty when using a complex, opaque model, with so many variables in play, of being certain that the outcome is correct. Long explains how this was monitored: ‘The model has undergone rigorous error checking by lots of different people over the years, and still does whenever we change anything. In addition to this, we always sense-check each stage of the model against our business understanding and historic data, and every time we do a major update of the

10

IMPACT | SPRING 2015

base data we check the results of the model for that year against actuals. Any discrepancies are investigated and only once we are confident that the model is working correctly do we use it to run scenarios and to help make decisions.’

There is also the problem that management, who need to take the decision based on the information provided, may not trust the output, as there is no clear way to understand what is going on in the model. Without training in complex techniques, linear programming is a black box, where a pile of numbers is poured in, the handle is cranked and a result mysteriously emerges. In part to reduce any concern arising from this, the process was backed up by simpler, broad brush and more transparent analyses, for instance calculating the operational cost per seat on a typical route, both to act as a cross check and to provide more confidence in the output of the model.

By the end of the process there were two key scenarios: a mix of Airbus and Boeing or an all-Boeing fleet. Long explains how this worked: ‘Given this was a multi-billion pound decision, the business wanted to be absolutely certain that it was doing the right thing. This meant completing over 1,000 model runs in order to cover all possible scenarios and talking to every department that would be affected by this decision to ensure their thoughts and concerns had been captured and considered.

Long again: ‘The composition of the order wasn’t chosen because it was a simple optimal result, but because it was the result that would allow us the most flexibility to adapt to changing circumstances and give us the best chance of realising as much value from our assets as possible.’ The recommendation to the board was a

mix of 18 additional Boeing 787s and 18 of the competing but larger Airbus A350-1000s.

LOOKING FORWARD

For the moment the decision has been made. Planning, with O.R., are looking for ways to improve the model and provide it with better and more comprehensive data, but it will not be mothballed until the next fleet order.

There have already been opportunities spotted to make use of the model in different ways, for instance to study the impact of changing configurations, where the number of seats in, say economy, business and first class are varied to see what impact this would have on route profitability.

The benefit of having an O.R. group is that you have on tap a group of analysts who are very quickly able to provide understanding, insight and solutions

How does BA benefit from Operational Research? Neil Cottrell provides the last word: ‘I view an O.R. group as a bunch of highly intelligent individuals, skilled in modelling and structuring problems, and with a degree of business acumen and good communication skills. Like a lot of large businesses, British Airways is multifaceted and faces complex issues on a regular basis. The benefit of having an O.R. group is that when those issues arise, you have on tap a group of analysts who are very quickly able to provide understanding, insight and solutions to these problems, which I think leads to better decision making and ultimately increases profitability. For BA they are a very valuable resource.’

Why do over 70% of Fortune 50 companies use SIMUL8 simulation software?

Powerful.

Flexible.

Fast.

Some of the world’s most successful organizations rely on SIMUL8 simulation software because it helps them to make and communicate their most important decisions.

Don’t rely on spreadsheets to be your process improvement toolkit. Make bold, confident decisions because you have the evidence to be sure you are making the right choice.

SIMUL8 has helped Industrial Engineers for over 20 years – saving money, reducing waste and improving efficiency.

Simulation software for innovative Industrial Engineers IMPACT | SPRING 2015 www.SIMUL8.com

11

M A K I N G A N I M PAC T Mike Pidd

Though, once again, my experience may be unusual, I think that O.R. practice is usually a cooperative endeavour that often involves quite a few people if the work being done is important. The INFORMS publication Interfaces specialises in publishing cases studies with multi-million dollar savings or extra profit. These are always claimed to be a direct result of the O.R. work that led to changes or new processes and systems within an organisation. Though these numbers are impressive, I suspect that the changes that led to the benefits emerged from work in which O.R. was a voice at the table, but not the only voice. That is, many different people and groups may also claim the impact and they are probably correct to do so.

A WORRYING COMMENT

O.R. practice is usually a cooperative endeavour that often involves quite a few people if the work being done is important

“That’s an easy question to answer,” he said, “University research has no impact whatsoever on the business world.”

Maybe he was having a bad day, but he clearly meant it and he was the Chief Executive of a very large multi-national business. He’d been asked to help a university assess the impact of its research. I was then, and am now, sure that he was wrong, though it wasn’t a good start to a conversation.

But, as well as worrying me, he stimulated two questions in my mind. Does the captain of the ship know what happens in the engine room? If he’d been asked asked about the impact of O.R. on his business, whether in-house or from consultants, what would he have said?

I’ve never had to earn my living as a consultant, but I suspect that it’s very hard to work with an organisation in which only the consultants are convinced of a particular course of action that differs radically from what is done now. I imagine that, in most cases, at least some people within the organisation will already know what needs to be done. Most senior managers are, after all, intelligent and experienced people. Without close cooperation with these people within the organisation, I suspect that the external consultant won’t have much impact.

O.R. IN PRACTICE

O.R. IN UNIVERSITIES

Though I’m known as an academic, I have maintained an interest in O.R. practice and managed, over the years, to do useful work in a range of organisations. My experience might be unusual, but my projects have rarely ended in a big-bang implementation decision after which it never rained on the parade again. More typically, insights appeared as the work proceeded and if these made sense, people seemed to get on with them, making whatever adjustments were needed.

For me, it’s often difficult to know when implementation (impact, if you prefer) occurred, but I know it did.

It’s difficult to know exactly long Operational Research has been a subject of teaching and research in UK universities.

I’m content to let others argue the toss about the exact start dates and about which institution led the way. It seems accurate enough to say that we’ve been active in UK academia for over 50 years. Much has changed in those 50 years, though the same is true of O.R. practice, with the disappearance of many large, in-house groups.

Whether we like it or, we UK academics are subject to a periodic review of the quality of our research, conducted by the funding councils. This started in 1986, became known as the Research Assessment Exercise (RAE), in the early 1990s, and had two contrasting features. On the one hand it was rather casual and demanded little in the way of preparation from the universities. I recall receiving a typed memo (remember those?) from our head of department asking if

my projects have rarely ended in a bigbang implementation decision after which it never rained on the parade again

12

IMPACT © THE OR SOCIETY

Image © Orlando Rosu/iStock/Thinkstock

anyone had any interesting research that could be entered in this competition. On the other hand, it was used to decide how much money would be distributed to each institution to support research - and this money was significant. As a contest, the rules were fairly relaxed, but the prize (a pot of gold) was worth having.

If we fast forward to 2014, things are very different. The Research Excellence Framework (REF) that replaced the RAE in 2014 has the opposite contrasting features. The rules by which the REF runs are far from relaxed and are almost byzantine in their complexity. However, there is less money at stake and it’s now bragging rights that matter.

Money is still important, of course, but the hope is that bragging rights will lead indirectly to a rather more distant pot of gold.

I Chaired the REF 2014 panel that assessed UK research in business and management studies. Many academic OR/ MS groups are based in business schools and departments, so we received quite a few submissions of OR/MS research.

For my sins, I was also part of the 2001 RAE and Chaired the equivalent panel in the 2008 RAE. Despite that I haven’t become too cynical, and remain impressed by the quality of UK research in OR/MS. THE IMPACT OF ACADEMIC O.R.

In REF 2014 there was a new element for us to assess: research impact. As I hinted earlier, I don’t think that impact is something that can be neatly packaged and date stamped, ready for someone to weigh and place in the shop window.

To assess impact in the REF, the funding bodies required each submission to include a set of impact cases studies.

These cases were required for all subjects covered by the REF, from theology through to engineering. The cases would describe the impact of some research, link it through to the underpinning research and say how this was achieved. This latter pathway to impact could have been serendipity, which seems reasonable to me, since academics are not employed to be consultants.

The panel members worked with users, representing different areas of business and management activity from the public and private sectors, to assess the impact cases.

We were asked to consider whether they demonstrated impact that was outstanding (4*), very considerable (3*), considerable (2*), modest (1*) or less than modest (unclassified). I’m pleased to say that the work submitted for us to assess included a good number of cases based on O.R., which was very gratifying. Though I was not directly involved in assessing all of these case studies, it’s clear from

the comments of my colleagues that some were very, very good. Not only is there very high quality research in O.R. in UK universities, but there is also strong evidence that this research has led to demonstrable benefits.

Not all academics agree with me, but I am very much in favour of including the impact of research in research assessment for subjects like O.R.. After all, our research time is paid from the public purse and from other research grants.

Assessing research impact needn’t mean that we reluctantly accept that he who pays the piper calls the tune, making us dance a dismal jig to an out of tune violin. Neither should we expect all research to have immediate benefit, or even any benefit – no commercial R&D lab would expect that.

there is very high quality research in O.R. in UK universities…there is also strong evidence that this research has led to demonstrable benefits

For O.R., this new focus on research impact should act as an encouragement for closer cooperation between academic researchers and those who might use that research. It should help keep practitioners in touch with latest developments and help ensure relevance in academic research. The assessment of research impact gives O.R. academics a chance to demonstrate what we say we’re good at; let’s exploit this.

Mike Pidd is Professor Emeritus of Management Science at Lancaster University, former President of the O.R. Society, and Chair of the 2014 Business and Management Studies REF panel.

IMPACT | SPRING 2015

13

A DAY I N T H E A N A LY T I C LIFE OF A CREDIT CARD A $13 Trillion Success Story for Operational Research ANDREW JENNINGS

CREDIT CARDS TODAY seem so blandly utilitarian that most of us give them little thought — other than, of course, whether that next purchase will put us over our credit limit, or whether we’ve paid the bill!

We certainly don’t credit the humble credit card as an analytic product. But in fact, that’s exactly what it is.

Every decision a bank makes about your credit card — and there are more decisions than you may imagine — is guided by analytics. Every transaction you make with your card is monitored and enabled by analytics. As a success story in operational research, the credit card is right up there with airline bookings and supply chain optimization.

This is more than an analytics bragging story, though. Easier credit has helped fuel economies worldwide. And much of banks’ operations are funded by credit card profits. In fact, about half of the net income from consumer banking at such global banks as Citi and Chase comes from credit cards, and while the ratios vary, cards are huge profit drivers for most major banks.

Cards’ importance is growing, not diminishing. Card spending was estimated at $13 trillion worldwide in 2012, and is expected to hit $27 trillion by 2018, fuelled largely by growth in Asia Pacific.

We all know what makes cards so convenient – but what makes them so profitable? This is the story of analytics at work in the day of a life of a credit card.

AN ANALYTIC MELANGE

First, it’s useful to note that there are three broad categories of analytics that drive credit cards’ success. These move along the analytic continuum from understanding and reporting to action, to answer multiple questions that can improve performance.

These analytics are in turn applied across what we think of as the cardholder lifecycle, from card marketing to origination of new accounts to customer management decisions to fraud and collections.

We’ll look at some of these decisions shortly.

DESCRIPTIVE ANALYTICS (E.G.,

PREDICTIVE ANALYTICS (E.G.,

PRESCRIPTIVE ANALYTICS

BUSINESS INTELLIGENCE,

CREDIT SCORING, FRAUD

(E.G., DECISION MODELLING,

DEMOGRAPHIC CLUSTERING)

MODELLING)

OPTIMIZATION)

What is my delinquency rate by

What is the risk of this person?

On what terms do I accept this

vintage?

Will this person carry forward some

application?

What is the average risk of new

of their balance (revolver) or pay it

What credit limit do I assign?

business?

off each month (transactor)?

What action will result in a payment

Are the usage trends stable?

Is this transaction fraudulent?

of an overdue balance?

14

IMPACT © THE OR SOCIETY

THE SCIENCE OF SAYING “YES”

The original application of analytics to credit cards began in the late 1950s, when two O.R. practitioners named Bill Fair and Earl Isaac decided to build their new business around a relatively new concept: credit scoring. Their work – originally in personal loans – led to the widespread adoption of credit scoring for all kinds of credit, from auto loans and mortgages to credit cards.

The concept is fairly simple: gather all the data known about cardholders

when they applied for a card, compare it to their subsequent performance, add some reject inference, and you can determine which characteristics and attributes relate to credit risk, and how to weight them. Credit risk models are built to separate future “good” payers from “bad” payers. (The performance definition for “bad” is generally something like a payer who has ever been more than 2 cycles (60 days) delinquent.)

Originally, these credit risk models were developed on lenders’ own data,

and were specific to their business.

In 1989, the company Fair and Isaac founded (now known as FICO) released the first FICO Score. These scores are now used in billions of credit decisions a year. While custom origination models are based on application data, FICO Scores are based solely from credit history data contained in an individual’s credit bureau report. An extract from an example FICO Score model is shown in Figure 1.

CATEGORY

CHARACTERISTIC

ATTRIBUTES

POINTS

Payment History

Number of months since

No public record

75

the most recent

0–5

10

derogatory public record

6–11

15

12–23

25

24+

55

Average balance on

No revolving trades

30

revolving trades

0

55

1–99

65

100–499

50

500–749

40

750–999

25

1000 or more

15

Below 12

12

12–23

35

24–47

60

48 or more

75

Number of inquiries

0

70

in last 6 months

1

60

2

45

3

25

4+

20

Number of bankcard

0

15

trade lines

1

25

2

50

3

60

4+

50

Outstanding Debt

Credit History Length

Pursuit of New Credit

Credit Mix

Number of months in file

FIGURE 1 –SAMPLE FICO SCORE MODEL (SIMPLIFIED)

IMPACT | SPRING 2015

15

While this sample has been simplified, an actual FICO Score development involves the analysis of more than 600 candidate variables. For a typical consumer, well over 30,000 floating point operations are executed in order to return a valid FICO Score.

Scores do not predict an individual’s specific credit performance or profitability. Rather, they rank-order individuals. People scoring 720 will as a whole perform better than people scoring 680, for instance. The actual ratio of good:bad accounts in any given score range, known as the odds ratio, will vary based on the economy and other factors.

KEEPING THE CREDIT MOVING

calculations designed to keep the payments system safe – and protect you from fraud.

Every single card transaction goes through a real-time authorization process, where information on the prospective purchase is communicated to the merchant acquirer (who has relationships with a set of merchants), a card processor (which processes transactions for the card type), a card network (Visa, MasterCard, Discover, AmEx) and the card issuer.

Focusing on the customer management decisions, one thing that’s being checked during this process is whether the transaction keeps your card within its credit limit or not. A transaction that doesn’t is referred to as an overlimit transaction, and your card issuer has options for handling those.

Provided you scored high enough, you now have the credit card in your hand.

What happens next?

Once you start using your card, you become part of a network of analytic

• Allow the transaction and raise your credit limit • Allow the transaction at your current credit limit

Data

Models

Masterfile Billing Credit Bureau Past Campaigns

• Allow the transaction at your current credit limit, but charge a fee for going overlimit • Decline the transaction How do they decide which action to take? Your credit score is involved — and now that you are a cardholder, there is another kind of score that can be used, your “behaviour score,” which is based on your payment performance with that issuer. But other factors may also be considered. It’s in the issuers’ interest to approve the transaction if possible — you’ll be happier, and they will get transaction revenue — but if you’re going to default on your card payments they need to shut the spending down.

Card issuers make these decisions using strategies defined as decision trees, which are a handy way to apply several variables in order to segment a population into very specific groups for targeted decisions. In the 1980s, FICO introduced adaptive control technology

Calculated Components Cost of Funds

Balance Interest

Probability of Revolving

Interchange Objective

Purchases

Actions

Rewards Costs

Probability of Attrition

Profit

Credit Line Increase Offer

Marketing Costs

Upgrade Offer Balance Transfer Offer

Probability of Bad

Fees, Operation Costs, Collection Costs Loss

Loss for Bads

FIGURE 2 – INFLUENCE DIAGRAM FOR DECISION OPTIMIZATION

16

Net Revenue

IMPACT | SPRING 2015

to credit card customer management.

This enabled card issuers to run “champion/challenger” tests of decision strategies, changing a variable such as the cut-off score at which an overlimit transaction is approved and running the new strategy in parallel with the current “champion.” This closedloop learning process is now used almost ubiquitously in card account management, as issuers incrementally evolve their strategies to make them more profitable.

Of course, the complication in these strategies can become overwhelming.

And it can be quite daunting to determine which variable to alter, and by how much, in order to make your next challenger strategy more successful. For about 15 years now, card issuers have been adopting optimization (prescriptive analytics) to derive their card account

PROTECTED BY NEURAL NETWORKS

management strategies based on data.

This involves influence diagrams (see Figure 2) and decision modelling to map all the components of a decision and their relationships with the target variable. These relationships can then be represented as a set of equations with an objective function and constraints which can be solved and implemented into the same operational systems that make the transaction authorisation decisions.

This in turn leads to some quite scientific analyses of how different strategies or scenarios move an issuer toward the “efficient frontier” — that min/max point where profit is maximized subject to other constraints.

The example in Figure 3 illustrates how this evolution of credit strategies can produce substantially stronger profits.

If you use credit cards, you have probably received at least one frantic call from a card issuer, asking you to confirm a recent purchase was yours.

You may even have imagined a cavernous room full of fraud analysts, scanning each transaction to see if it looks legit — occasionally an analyst strokes her chin and says, “Funny, I don’t remember Marge buying that many cat toys before. Perhaps I’ll give her a ring.”

That’s nearly how it works, too — except cut the number of fraud analysts down to a handful of case managers, who do the actual transaction monitoring with an ingenious analytic system. Ever since 1992, when HNC Software introduced its Falcon software, the chief card fraud defence around the world has been based on artificial intelligence.

PROJECTED PROFIT PER CARDHOLDER

Efficient Frontier Trade-off of Profitability vs. Multiple Business Objectives Scenario G Increase profitability by 12% per cardholder, without incurring additional exposure

F

G

H

I

J

E D

C

Efficient Frontier

Current Operating Point Where you are today

B

A

Scenario B Maintain profitability per cardholder and decrease exposure by 7%

PROJECTED CHANGE IN LIMITS OVER “BASELINE” Typical Benefits: $47m incremental profit over 3 years $390m incremental balances over 3 years on a portfolio of 3.0M with 40% roll-up year 1 and 80% in year 2

FIGURE 3 – HOW BETTER DECISIONS YIELD MORE PROFIT

IMPACT | SPRING 2015

17

Falcon introduced neural networks to financial services from military applications. Neural networks are modelled on the way the human brain works, and find connections between, for example, credit card transaction characteristics and the probability of fraud. The kinds of risk models described above are largely built by testing different combinations of data for their predictive power: transparency is important, since ultimately a lender may be required to

tell a consumer why they didn’t meet a score cut-off. In fraud, this kind of transparency is not as important, and neural networks are “trained” on massive datasets, and can even adapt to changes in patterns, which is critical in a realm where attackers are constantly changing their schemes to fool the system.

The models are developed in a supervised learning scenario – the input layer consists of features of transactions that are randomly weighted to start,

and fed into neurons in the “hidden layer” of the network, which identifies relationships. The neural network makes predictions based on these weights, shifting the weights around to improve predictions until it reaches the best predictions. The neural nets output a score that identifies the fraud risk of a given transaction. This process is illustrated in Figure 4.

The neural networks are complemented by cardholder profiles,

Data (Transactions)

Input Layer (Engineered Variable Features)

Number of Transactions in Last Hour

Hidden Layer (Identification of Feature Relationships)

Large Dollar Amounts

Merchant Category Code

Frequency Last Transaction to Current MCC

Rapid High-Risk Expensive Transactions

Output Layer (Score Calculations)

Calculation Time = 40–60 Milliseconds

Neuron-Like Network Node (Processing Element)

Fraud Score

FIGURE 4 – NEURAL NETWORK FOR FRAUD DETECTION

18

IMPACT | SPRING 2015

Connection “Weight”

Source: FICO

which identify normal and abnormal behaviour for a specific cardholder (not a type of cardholder). Because cardholder transaction histories can become quite massive, it is often impractical to retrieve the entire history in order to evaluate a single transaction.

Transaction profiles contain recursive variables to summarize the relevant predictors.

For fraud, time is of the essence. The analysis of a transaction happens in about 40-60 milliseconds – about onefifth the time it takes you to blink. In that time, the Falcon analytics perform 15,000 calculations.

The power of these analytics is incredible. While card fraud losses have multiplied globally over the last 20+ years, this is a function of card transaction growth. In fact, fraud losses as measured in basis points (1/100th of 1%) of card sales have declined dramatically during this period. In 1992, when Falcon was introduced, fraud losses in the U.S. stood at 18 basis points. They dropped to 8.4 by 2000, and have been below 6 since 2007. A very conservative estimate puts the savings for U.S. card issuers, based on these numbers, at more than $10 billion.

Falcon today, protecting more than 2.5 billion payment cards worldwide, has also contributed to dramatic reductions in losses in other countries.

In the UK, card losses dropped from a high of £725 million in 2008 to £535 million in 2013, as the industry adopted chip and PIN technology and stepped up its use of analytic fraud detection.

The phenomenal rise of the credit card worldwide has been fuelled by analytics

Fraud protection is more than just a crackdown on criminals. Consumers want to be protected, but they also don’t want to be inconvenienced by blocked transactions or unnecessary phone calls.

To improve fraud detection and reduce the number of “false positives,” data scientists have invented creative ways to boost performance. Some of the recent additions to the technology are: • Profiling of merchants and devices to recognize patterns of fraud at specific point-of-sale terminals and ATMs. • Self-calibrating technology that allows a system to fine-tune itself in real-time in response to shifts in transaction trends.

• Adaptive analytics that adjust the weighting in the neural networks as fraud patterns change.

• Global intelligent profiles that identify the riskiest ATMs, merchants and regions so extra scrutiny can be applied where the risk is greatest, without delaying the processing.

• Behaviour-sorted lists, which use a cardholder’s most frequent transaction locations to reduce false positives.

• Proximity correlation, which compares the location of a cardholder’s mobile phone to the place where the card transaction is occurring.

• Behavioural archetypes, which use Bayesian analysis to find soft clusters of cardholders with similar patterns, and can therefore determine that a rare transaction for a cardholder (say, buying a TV) is not unusual for someone in their cluster.

O.R. EVERY STEP OF THE WAY

The phenomenal rise of the credit card worldwide has been fuelled by analytics — in the UK all of this is now common practice and in APAC, the market with the greatest growth outlook, issuers are quickly adopting the latest technology to keep their profits strong and their payments safe. Every time you use your card, remember: it’s not just a piece of plastic or a gateway to debt, it’s a supercharged analytic engine!

Dr. Andrew Jennings is chief analytics officer at FICO, a leading analytic software company. He blogs on the FICO Blog at www.fico.com/blog.

IMPACT | SPRING 2015

19

EACH YEAR STUDENTS on MSc programmes in analytical subjects at several UK universities spend their last few months undertaking a project, often for an organisation. These projects can make a significant impact. This issue features reports of projects recently carried out at two of our universities: Lancaster and Strathclyde. If you are interested in availing yourself of such an opportunity, please contact the Operational Research Society at email@theorsociety.com

MEASURING THE IMPACT OF THE IMPLEMENTATION OF A NEW SALES FORCE STRUCTURE AT BOEHRINGER-INGELHEIM UK (Mengjia Chen, Lancaster University, MSc Operational Research and Management Science)

During 2012 and into 2013 Boehringer lngelheim Ltd underwent a major restructuring of its sales field force. The new structure was completely innovative for both the local organisation and its Global parent, and was seen as the first step in moving the company to a much more customer­centric approach.

However, it was faced by a high level of scrutiny from the Global organization, a top 20 world- leading pharmaceutical company. This meant that the UK affiliate needed to be able to demonstrate to its parent company, using an evidence-based approach, that the new sales force model was having an impact on the bottom line.

The work undertaken by Mengjia allowed Boehringer-Ingelheim UK to demonstrate beyond any reasonable doubt that the new structure was delivering improved results.

This project measured the impact of implementation of the new sales force structure on sales performance of three important products for Boehringer-Ingelheim in three very different therapy areas (diabetes,

20

IMPACT © THE OR SOCIETY

cardiovascular and respiratory disease respectively). In order to measure the impact, several modelling methods were developed. One, Cumulative Prediction Interval Detection Method, calculates point forecast values and prediction intervals to compare with actual post-restructuring sales values.

This provides valuable information for senior management because it enables them to see cumulative prediction intervals. These allow them to observe whether the actual YearTo-Date sales have gone above the cumulative prediction intervals. This solution has been instrumental in being able to demonstrate in a robust way the effectiveness of the new way of working.

Andrea Beddoni (BI’s Head of Business Intelligence and Analytics) said “I thoroughly recommend the work that Mengjia has done has a sterling example of application of sophisticated statistical techniques in a business setting delivering business relevant insights.”

One outcome of the project was that BI UK employed Mengjia, and

she has been able to participate in exciting day-to-day business analytics projects, for example, forecasting brand sales at national and local level to assist sales target setting, track delivered activity individually to help calculate incentives to be paid out, or refine target customer lists with most potential based on statistical analysis.

In addition to the various interesting projects in which Mengjia has been involved, she has also taken part in producing and updating critical business reports. She says, “I’m very grateful to be part of BI Business Analytics team. I have been impressed by the friendliness of people here.

I’m working in an international team which makes it even more fun. The daily work is challenging, but in a good way, which won’t put too much pressure on myself but at the same time stays interesting and demanding”.

Raj Nagra (BI’s Business Analytics Manager) said “Mengjia has settled well into a very fast paced and pressured environment. Her attention to detail and the accuracy in her work has aided some key decision making. Her willingness to take on responsibility and demanding analytical work has impressed her colleagues.”

Source: Lancaster University

U N I V E R S I T I E S M A K I N G A N I M PAC T

Source: University of Strathclyde

DEVELOPING A DATA MINING AND DATA MANAGEMENT TOOL FOR HEALTH AND SOCIAL CARE INTEGRATION AT THE SCOTTISH GOVERNMENT (Christoph Werner, University of Strathclyde, MSc Operational Research)

Health and social care integration is a key area of Scotland’s strategy to cope with a steadily ageing population and the associated increasing consumption of health and social care services.

Despite future demographic pressures, the Scottish Government focuses on improving services for those in need.

This is a complex task which involves numerous stakeholders and spans a range of strategic initiatives including service design, procurement and planning as well as performance measurement. Effective analytic support is key and provides a better understanding of the distribution of resources. In particular, variations in the consumption of health and social care at different age and geographical levels are of interest. This provides an overview of hospital, community based and local authority costs and allows for better informed investment decisions with focus on joint resources rather than more traditional separate budget lines.

Christoph’s work supported the resource analysis by considering both qualitative and quantitative aspects of managing the complex data set resulting from integration.

The qualitative framework provides guidance on how to achieve knowledge discovery and communicate this knowledge to different stakeholders.

It can be seen as a starting point for integrated data management by making

use of a self-assessment tool, based on Denison’s Organisational Culture Analysis, which allows investigating the ‘degree of analytical culture’ within a network of partners that use and/ or produce analytical insight. A gap analysis is constructed that compares, for instance, partners’ perceived importance of analytical insights for decision-making to openness towards continual refinement of skills or transparency of data gathering and analysis. Thereby it benchmarks different strategic partners to establish best-practice analytical cultures. The whole assessment is achieved through a custom-programmed Excel spreadsheet with printable surveys and result visualizations.

The quantitative tool provides dashboards for decision support by extending ideas on how raw data can be turned into information. Features include, for example, exploratory data analysis and predictive modelling of resource use at different demographic and geographic levels through elements of data mining and forecasting. A variety of display formats are provided to help the user understand and investigate the data. This might be seen as a prototype of what insights the current data offers and how those might be extracted automatically. Based on this, a proof-of-concept in the form of a system dynamics approach was constructed to show future possibilities

for decision support. The exemplary modelled subsystem concerns the dynamic behaviour of the demand side for intermediate social care. It links analytical insights from the resource use dashboards to evaluation of preventative policy options by determining stocks for intermediate care based on planned and unplanned hospital admissions.

The client, Christine McGregor (Economic Advisor, Health Analytical Service Division) stated: “The tools Christoph developed […] pulled together a range of complex data and provide a useful resource for both ASD [(Health) Analytical Service Division] and stakeholders. The latter [quantitative tool] is particularly important as often analytical expertise does not exist locally where the output from the analysis is critical to successful commissioning and integration.”

Regarding its future impact she commented: “I am particularly looking forward to incorporating Christoph’s ideas that are contained within these two tools into future qualitative and quantitative analysis.

Specifically, I have shared these with NHS ISD [Information Services Division] who are currently developing a benchmarking type database which will be available to the NHS and Local Authorities”. Indeed, the project has had a positive impact as the data tools are now in the process of consideration to explore how partnerships can benefit from them. Additionally the project has been presented to two regional Health and Social Care Integration Programme Managers who are currently sharing the ideas in their boards to explore how they can make use of them.

IMPACT | SPRING 2015

21

Supplied by photographer, Luiz Andrade

S T U N G I N TO AC T I O N NEIL ROBINSON

MOST PEOPLE WOULD imagine the only link between bees and ambulances is an especially bad sting.

In Brazil’s largest and most trafficplagued city, however, the relationship is something quite different – and far more beneficial.

Regardless of how advanced our thinking might become, irrespective of our technological triumphs and achievements, the natural world still has many valuable lessons to teach us. And why shouldn’t it? It has, after all, had millions of years to get things right.

Great minds throughout the ages have recognised as much. When Gaudí set about building the Sagrada Família he rooted his design in the beautiful efficacy he saw all around him – curves, wood, muscle, tendon – famously

22

IMPACT © THE OR SOCIETY

declaring: “Originality is returning to the origin.” Swiss engineer George de Mestral invented Velcro after microscopically examining the burrs he removed from his dog’s coat following a walk in the Alps. The loud claps that once accompanied Japanese bullet trains’ emergence from tunnels were eliminated when engineers fashioned a nosecone inspired by a kingfisher’s beak.

Nowadays we even have scientific terms for this sort of problem-solving: biomimicry, bionics and biomimetics.

The fact is that we are more aware than ever that the answers to many of the most complex and daunting puzzles that confront us might be found in the behaviour of other species. We truly appreciate the relationship between the nature of innovation and the innovation of nature.

Image © Mihajlo Maricic/iStock/Thinkstock

One of the foremost examples in operational research is the Bees Algorithm, a program that imitates how honey-bee colonies forage for food. Drawing on this analogy has helped to shape optimisation strategies in a range of settings in which multiple possible scenarios and solutions require analysis.

In a city like São Paulo what hope is there for an ambulance service and the people it is intended to serve?

The Bees Algorithm contains several important elements, one of which is its ability to replicate the significance of a bizarre ritual known as the waggle dance. Maybe it is fitting, then, that it has been put to such good use in the land of the samba.

GROWTH AND GRIDLOCK

Brazil is an emerging economic power. Rich in resources and with a population of almost 200 million, it has the second-largest industrial sector in the Americas and a diverse and sophisticated services industry.

The World Economic Forum named it the top country in terms of enhanced competitiveness in 2009.

Inevitably, it must increasingly strive to meet the standards set by developed nations as it continues its economic journey and furthers its attempts to raise millions out of poverty. That means better procedures and better practices. The operational research community is committed to supporting this transformation.

By way of illustration, every three years the International Federation

of Operational Research Societies presents the IFORS Prize for O.R. in Development. The award is made to the authors of “a practical O.R. application in a developing country, conducted to assist an organisation in its decision-making process, with original features in methodology or implementation”. In 2014 the honour went to Luiz Augusto Gallego de Andrade and Dr Cláudio Barbieri da Cunha for their work in lowering response times for ambulances in São Paulo.

Meeting response times can be a challenge anywhere on Earth, but rarely are the basic circumstances less favourable than in Brazil’s biggest city. One of the most visible manifestations of economic ascent and the concomitant growth of the middle class is car ownership, which in São Paulo has become as much a curse as a blessing. It has been estimated that on Friday evenings, when the rush-hour mayhem is at its suffocating peak, the tailbacks out of the city extend for a total of up to 183 miles. In 2005 a dedicated radio station was launched to report on jams 24 hours a day, seven days a week.

“No city can afford to have so many cars,” says Dr Cunha, an associate professor of engineering at the University of São Paulo. “The big problem is that we Brazilians are terrible with planning. Traffic will become more manageable only if we start looking into real, long-term solutions. The key is to find a balance – the point at which it’s worth using public transport because it’s faster than driving.”

Clearly, though, the misery is by no means confined to commuters. The cost of doing business is also affected.

As Dr Cunha points out, the knee-jerk reaction when a truck is capable of making only 10 deliveries instead of 20 is to buy another truck. The expense mounts up. The economy suffers.

And then there are the sectors in which success or failure must be measured in much more than mere profit and loss. In a city like São Paulo, with almost 11 million residents and one of the most crushingly overcrowded road networks on the planet, what hope is there for an ambulance service and, more pertinently, the people it is intended to serve?

IMPACT | SPRING 2015

23

LUIZ ANDRADE AND DR CLÁUDIO CUNHA RECEIVING THE IFORS PRIZE FOR O.R. IN DEVELOPMENT FROM PRESIDENT NELSON MACULAN (LEFT) AND CHAIR OF THE JURY, ANDRES WEINTRAUB (RIGHT)

FROM JAM TO HONEY

In 2007 the average response time for an ambulance dispatched by São Paulo’s Serviço de Atendimento Móvel de Urgência (SAMU) was almost 30 minutes. To put this into context, in the same year the average response time for an ambulance dispatched in London was 14 minutes; in Montreal it was just 10 minutes – the accepted international benchmark. However fuzzy the oft-quoted notion of the “golden hour” in emergency medicine, improvement was imperative.

One option was to invest in more ambulances. Unfortunately, as highlighted in Dr Cunha’s comment about delivery trucks, the desired tradeoff between extra cost and superior efficiency does not automatically ensue.

Another possibility was to open new ambulance stations, but a combination of factors – including financial constraints, a lack of available buildings and the sheer weight of Brazilian bureaucracy – suggested this ploy, at

24

IMPACT | SPRING 2015

least in its simplest form, could also hold limited appeal.

A third idea was to use mobile stations, of which SAMU then had 10. These are not only cheaper than their permanent counterparts but can be relocated to ensure good coverage.

Such an approach could prove advantageous in a city with distinct areas of population density, land use, infrastructure and, arguably above all, traffic congestion.

SAMU could benefit enormously from a reliable mathematical means of identifying an optimal distribution of its ambulances and/or bases

At the time Luiz Augusto Gallego de Andrade was studying under Dr Cunha. The travails of São Paulo’s ambulance service struck him as a

worthy subject for his dissertation on logistics systems engineering.

He realised SAMU could benefit enormously from a reliable mathematical means of identifying an optimal distribution of its ambulances and/or bases.

The first task was to assemble the relevant data. This included patterns of disease and other health issues, information from SAMU and a wealth of facts and figures about the city’s infamous traffic chaos. There were also numerous variables to process, among them costs, demand, location, numbers and capacity. The principal question was how all of this might usefully be deciphered to reveal how SAMU should deploy its 140 vehicles in the face of approximately 9,000 calls per day.

It was, Andrade reasoned, a conundrum that could be addressed by scenario simulation and combinatorial optimisation. What was needed was a particular kind of solution search procedure – one based on exploration and exploitation. The answer lay in nature, where the finest exponent of exploration and exploitation, by common consent, is the not-so-humble honey-bee.

CREATING A BUZZ

It is no exaggeration to say the members of the average honey-bee colony get about. Various studies have shown they will venture more than eight miles to look for food. The manner in which they communicate and learn from their forays is one of the natural world’s most remarkable and informative instances of what is known as collective intelligence.

Scout bees make up only a small proportion of a colony, yet they play a massive role in its life. Their job is to forage in multiple directions, hunting for nectar and pollen and evaluating

the yield of any sources they find.

Interestingly, they initially embark on their sorties entirely randomly: they do not set out on a predetermined course.

Those who return with food, deposit their harvest and, if the source is especially rich, go to an area of the hive that scientists have named the “dance floor”. It is here that they perform the aforementioned waggle dance, through which they are able to convey the location of a source to their watching nestmates.

Suitably compelled, the onlookers join in the exploitation of the promising patch. Since the length of the dance is proportional to the scout’s assessment, more bees are recruited to harvest the most rewarding sources. Former onlookers sometimes perform the waggle dance upon their own return, further boosting productivity where it is most needed. The process is brilliantly autocatalytic and allows the colony to consistently readjust its focus to best effect.

“What’s significant is that bees have mastered a method of constantly adapting to their surroundings,” says Andrade, now a director of TEVEC Systems, a company specialising in mathematical modelling and computational simulations. “It isn’t a case of a single ‘planner’ bee telling all the other bees where to go and what to do. They start out in an apparently random fashion and then build on what they discover to deploy their resources with maximum efficiency, always concentrating their efforts where they’re needed most. And when we think of it as an object lesson in resource deployment, of course, it’s not hard to see how the model can be applied elsewhere.”

Hence Andrade’s SAMU-centric algorithm. Consider the parallels. Just as a colony’s foraging is at first conducted randomly, so demand for medical care

arises in an arbitrary fashion. Just as scouts find food sources and report back to the hive, so ambulances are dispatched to emergencies before returning to their bases. Just as other bees are impelled by the waggle dance, so other ambulances might be mustered according to emerging demand.

Drawing on their findings, Andrade and Dr Cunha recommended a restructuring intended to maximise coverage while minimising disruption.

The flexibility to reallocate and reposition was a cornerstone of the new system: additional mobile bases were employed – usually in squares, parks and other public spaces – allowing ambulances to be temporarily stationed in “hotspots” when needed and recalled to their original bases at other times.

As envisaged, just as bees’ foraging cycle fosters collective intelligence, so Andrade’s modelling of various scenarios had pieced together a way of reducing SAMU’s response times – which, sure enough, were duly more than halved.

DRIVING SUCCESS

Between 2007 and 2012, under the guidance of its managing director, Colonel Luiz Carlos Wilke, SAMU cut its average response time from nearly half an hour to 10 minutes. It also became the first Latin American ambulance service to be recognised as an Accredited Centre of Excellence.

Its dream of attaining international standards had been realised.

It is perhaps right to acknowledge that many influences drove this dramatic turnaround. Wilke, a dynamic leader and one of the founders of the São Paulo Fire Search and Rescue Service, assumed office with a marked determination to render what he inherited genuinely fit for the 21st century. Objectives were established

and pursued. Novel protocols and policies were introduced. Almost every aspect of operations was subtly refined or radically overhauled. Yet there is no doubt that Andrade and Dr Cunha, through their work, made a pivotal contribution.

“The concept we proposed wasn’t an easy sell,” admits Dr Cunha. “We had to clearly demonstrate the benefits that mobile stations, if properly located, could bring when compared to traditional facilities in regular buildings – and we had to do it in the context of a limited budget.

“Fortunately, our algorithm enabled us to do precisely that. We were able to show how SAMU could cope with fluctuating demand, and we were able to show how it could make the most of its resources during different time periods. The bottom line is that we were able to show how, just like a bee colony, SAMU could constantly adapt to its surroundings.”

The citizens of São Paulo have reason to be grateful. For them, despite their debt to the bee world, there is no sting in the tale. In 2012 the new, more efficient SAMU successfully dealt with half a million call-outs. The next goal is to cut response times again – from 10 minutes to five.

“It’s been our mission to meet the requests classified as medical emergencies or urgencies in the shortest time possible,” says Wilke, a veteran of more than 40 years in the public service sector. “It’s with pride that we say the São Paulo SAMU is the largest and most modern in Latin America, but nothing is more important than our ability to save lives.”

Neil Robinson is the managing editor of Bulletin Academic, a communications consultancy that specialises in helping academic research have the greatest economic, cultural or social impact.

IMPACT | SPRING 2015

25

Graham Rand I was watching the TV news last autumn, when an item referred to important dates in 2015. My 8-year old granddaughter piped up “that’s the day before my birthday!”

The date she had noticed was that of the General Election: 7th May. No doubt I’m required to remember both dates and will have many reminders to that effect. A week or so later, when watching a discussion of the Rochester and Strood by-election, the analyst mischievously showed a prediction for the General Election based on UKIP’s recent performance: Conservatives with 5 seats and UKIP with over 600! That’s some prediction!

Now, as I write at the start of the year, the media is full of discussions about the election, and the general view seems to be that the outcome is unpredictable, or, at least, it is not possible to predict with any degree of certainty. What might be predictable is that all this discussion four months before the day is enough to put many people off. At the risk of this article having that effect on you, here are a few thoughts about forecasting election results.

In November, the media reported that the Labour Party had hired a self-employed betting expert, Ian Warren, to be

26

IMPACT © THE OR SOCIETY

its general election data guru. He was widely described as the “British Nate Silver”, which I’m sure boosted his ego a great deal. Apparently, Ian Warren correctly predicted the precise results in the 2008 and 2012 American presidential elections, which gave rise to the comparison with US analyst Nate Silver. The Independent’s columnist, Matthew Norman, later reported that Labour is going to pay him handsomely – perhaps not the experience of every analyst! The sub-heading was “General elections are now won far less by ideas than by algorithms”. This thought does not appeal to Mr. Norman: “there is something plainly melancholic about the growing supremacy of the science over the art, and about the prospect of a general election being won … far less by ideas than by algorithms”. This is a rather surprising misunderstanding of what algorithms do.

only an idiot would tell you who is going to win

Warren’s comparator, Nate Silver, has quite a track record.

Not content with correctly forecasting US Presidential elections (in 2008 his model accurately predicted the winner in 49 of the 50 states, and he called every Senate

Image © TylaArabas/iStock/Thinkstock

PREDICTIONS OF ELECTIONS

Image © saintho/iStock/Thinkstock

race correctly too. In 2012 — now at the New York Times — he got all 50 states right, while some pundits were still claiming the race was ‘too close to call’, or even that Mitt Romney would win) he turned his hand, or model , to the Scottish referendum. A year before last September’s vote, James Kirkup, a Telegraph commentator, wrote “It’s all over.

Scotland’s political class can pack it all in. No need for campaigning, adverts, arguments, persuasion. The debate about Scottish independence is done, and the Union has won. Never mind that no one has actually cast a vote, Nate Silver has spoken”. “There’s virtually no chance that the ‘yes’ side will win,” Silver said in an interview with the Scotsman.

“If you look at the polls, it’s pretty definitive really where the no side is at 60-55% and the yes side is about 40 or so.”

“General elections are now won far less by ideas than by algorithms”

Kirkup sarcastically refers to Silver as “the Man Who Knows”. He writes “good luck to Mr Silver and the political scientists and the other neo-Silverites in their quest to bring comforting certainty to what can seem a distressingly uncertain and disorderly field of human activity. Maybe one day they’ll prove beyond doubt that they really can see the future in the polling day, and do so consistently and verifiably. In the meanwhile, I’ll go on trying to cope with the idea that the world is a complicated and unpredictable place and that sometimes, just sometimes, you can’t see it coming”. But Silver did see it coming, as the result turned out to be a 55% no vote.

Nate Silver has equally strident views. In the second chapter of his book, The Signal and the Noise: The Art and Science of Prediction, (Penguin 2012) he describes an analysis which demonstrated that political pundits might just as well toss a coin when making their predictions. In an interview with the Spectator’s Jonathan Jones he goes even further. ‘I think some of them are very skilled at the art of bullshit; I think some of them are just deluded; some of them aren’t very smart; some of them are immoral; some of them are well-intentioned but wrong; some of them are behaving as party hacks. And there’s not a lot of incentive for them to change that.’ Most columnists are ‘a waste of space’, he says.

‘A lot of them are nice people, but they’re literally a waste of column inches.

How does Silver make his predictions? He spells it out in his FiveThirtyEight blog http://fivethirtyeight.com/features/

how-the-fivethirtyeight-senate-forecast-model-works/ (socalled because of the number of votes in the US electoral college). I leave the detail for you to read at your leisure, but he starts with four principles on which a good model should be based: (1) It should be probabilistic, not deterministic: the model might estimate the % chance of a specific candidate winning the seat. (2) It ought to be empirical. (3) It ought to respond sensibly to changes in inputs. (4) It ought to avoid changing its rules in midstream: Silver doesn’t “tweak” the forecast in a given state just because he doesn’t like the outcome. As for the model itself, he describes seven major steps, starting with a weighted polling average, and ending with estimating the margin of error and simulating outcomes and estimating the probability of a party being in control.

On the day I wrote this, David Dimbleby said on The One Show that “only an idiot would tell you who is going to win, and only a real idiot would tell you who is going to be Prime Minister”. Maybe, but at least one person will be celebrating on May 8th – my granddaughter.

Graham Rand is Senior Lecturer at Lancaster University, former editor of the Journal of the Operational Research Society, and a Companion of Operational Research

IMPACT | SPRING 2015

27

Image © BAE Systems 2015

CO R DA – D E L I V E R I N G S U CC E S S F U L F U T U R E S NOEL CORRIGAN

THIRTY YEARS AFTER its creation, CORDA continues to deliver highly valued decision support services to the Ministry of Defence (MoD) and to the defence industry.

CORDA is the centre of excellence for decision support, modelling and analysis within BAE Systems, the largest manufacturing company in the UK. As such, CORDA services all the BAE Systems’ businesses, and the UK MoD is its main external customer.

This article describes how CORDA addresses its customers’ problem types, how it adapts to changes as the world has turned, and how its tools and techniques have evolved.

28

IMPACT © THE OR SOCIETY

O.R. as applied in the defence domain was initially concerned with the analysis of military operations to increase the efficiency and effectiveness of military affairs at the tactical and strategic levels. After WWII, the use of O.R. expanded into industry, including the defence industry. With the advent of computers and their wider uptake in the 1970’s, modelling and analysis services began to be provided to the MoD, largely through software houses.

CORDA TURNS 30

Which brings us to 1984, probably best known as the year that George Orwell’s nightmare did not come to pass. What

Image © BAE Systems 2015

you might not know is that it was also the year that CORDA was formed.

Originally part of CAP Scientific and now part of BAE Systems Shared Services, CORDA has been providing modelling, analysis and consulting services for the last 30 years. Steve Allan, General Manager of CORDA, realises how unique this makes his team.

“To last this long in occasionally challenging climates is testament to the work that we do. Over half the work we do is directly for the MoD, where we have to prove our worth competing against other providers.”

we are trusted to provide high quality solutions that really make a difference to our customers

“The fact we are actively chosen time and time again just goes to show that we are trusted to provide high quality solutions that really make a difference to our customers, be they within the MoD or within BAE Systems”.

THE WORLD TODAY

In the current environment none of the military or commercial certainties of the 80s remain. Notwithstanding current events in Ukraine, there has been much more emphasis on stabilisation operations and nation building, rather than high intensity war-fighting. The early part of this century saw the advent of contracting for availability and contractor logistic support contracts, where industry takes a greater share of the risk in delivering equipment and associated services to the front line. Coupled with the

DECISION SUPPORT TOOLS IN USE

economic downturn of 2008, this has an impact on the type of O.R. that is required, and the timescales for its delivery.

Information Technology and the growth of computing power has made the models faster and given amateurs the false impression that it is easy to “knock together” a quick spreadsheet model to conduct a thorough analysis.

The speed of communication of ideas and the general pace of life and business has increased to the extent where it seems no-one has the time to contemplate a decision. But following the Laidlaw Inquiry, and subsequent MacPherson Review, Evidence Based Decision Making and understanding associated risks has become de riguer across government.

HELPING THE MOD MAKE THE RIGHT DECISIONS

In competition, CORDA won an operational analysis collaboration contract to help develop the Defence Science and Technology Laboratory (Dstl)’s operational analysis tools and techniques in preparation for the next UK Strategic Defence and Security Review (SDSR) in 2015. This strategic

contract, worth up to £3m over three years, covers all domain areas, and the full breadth of qualitative and quantitative analytical techniques, from workshop facilitation to campaign-level simulations.

To meet these demands, CORDA is working closely with NSC, a leading provider of modelling and simulation services, and is able to draw on a broad community from industry and academia. This will ensure that MoD decision makers get the maximum benefit from the breadth of science and analysis undertaken throughout defence.

Work completed to date includes: • Investigation into the feasibility of applying genetic algorithms to Strategic Balance of Investment decisions; • Development of visual maps of the Unmanned Air Systems Enterprise, to show the operational impact of changing activities; • Development of a toolset to aid Balance of Investment to support Army HQ; • Helping the Armed Services understand Balance of Investment decisions following defence reform.

IMPACT | SPRING 2015

29

In the maritime domain, CORDA has had a long relationship with the Type 45 destroyer programme. Some of the early work was with the MoD’s Type 45 project team in the Defence Equipment and Support (DE&S) agency, back in the early 2000s, looking at the whole life costs of the programme. The recent work has been with the BAE Systems Type 45 programme managers in Portsmouth.

The recent emphasis of CORDA’s work has been with the “Future output management” team whose focus is on bringing in new approaches that will help manage availability across classes of ships. These approaches include a wider range of data on the ship’s performance and the analysis to exploit that information – CORDA’s focus is on the latter of these.

TYPE 45 DESTROYER

30

IMPACT | SPRING 2015

Recent work includes helping the managers understand the issues around power and propulsion on the ships. A mixed team from BAE Systems Fleet Services and CORDA, analysed the power failure reports from the ships plus the more traditional OpDef (operational deficiency) returns. This analysis focused on looking for patterns in the data and using those patterns to predict the future volume of incidents based on the planned use of the class.

This work has helped the Type 45 programme in a number of ways; it: • provides some objectivity to help focus decision making and debunk myths within the Type 45 community; • provides a way to build the benefits case for candidate interventions; • encouraged DE&S to start collecting slightly broader data for further analysis.

This, and other work that CORDA is doing, has helped strengthen the links between BAE Systems and DE&S and started to highlight the value that analysis can bring to both organisations.

HOW TO CROSS A BURNING DITCH

In the land domain, CORDA supported the British Army in developing its understanding of the future requirement for its battlefield obstacle breaching capability, and how it would meet this need. This work looked over the next 20 years to identify emerging capability gaps and ways of filling them. This provided both Dstl and the Army engineers with a long term vision of the focus of research, actions to take with training, logistics and doctrine, and ideas of where short term procurement funding would be best spent. BAE Systems provided a combined team of analysts from CORDA and engineers from Combat Vehicles (UK) who devised a novel method for identifying capability gaps which displayed the multiple relevant factors on one simple visualisation.

From this visualisation, the team could provide the evidence required by Dstl.

Both Dstl and the Army were engaged throughout the project to ensure that their views were fully accounted for. The results (a prioritised list of capability gaps and solutions – both simple and far-field) were presented at Major General level. This work has the potential for follow on work and gives BAE Systems the visibility of where breaching capability may go over the next 20 years.

Obstacles that may require breaching could range from ditches to parked buses to minefields/IEDs

Image © BAE Systems 2015

CORDA CONTINUES SUPPORTING TYPE 45

Image © BAE Systems 2015

(Improvised Explosive Device) to electromagnetic radiation emitters.

Typically these obstacles would be tackled with Army engineer vehicles (and attachments) like: Trojan, Terrier or Titan. The framework developed by the team provided a transparent, auditable and highly visual tool that captured both objective data and judgemental information to rate the difficulty of the range of obstacles likely to be faced by the British Army.

Applying the nominated approach was new to this customer - previously no such approach had been used with the Royal Engineers. The framework also included the effectiveness of potential solutions in dealing with obstacles, both current and in years to come – future proofing the Army engineers’ capability. The information contained was built up through a series of stakeholder engagements, and formed the basis for the final workshop in which options were prioritised. The final output visualisation displayed all of the gathered and analysed information in a simple grid format that allowed the user to see quickly where capability gaps were.

Additionally, the visualisation was set up with layers of information. The user can drill-down quickly to see the supporting data in how the final outcome was decided. In total there were three layers, each displaying a finer level of granularity.

HELPING SERVICE PERSONNEL GET ON THE HOUSING LADDER

Besides provision of front line capability, CORDA also supports decision making on enabling services.

For example, it is currently supporting

TROJAN AT WORK

MoD head office with their personnel and training policies.

Increasing house prices and frequent moves throughout their careers means that it is challenging for Service Personnel to enter the housing market.

In order to help them with this, former Defence Secretary Philip Hammond launched the Armed Forces Help to Buy scheme, where Service Personnel are able to take out an interest-free equity loan up to a maximum of £25,000 or 50% of their salary.

The £200m, 3 year pilot scheme has been introduced as part of the New Employment Model (NEM) review in order to help members of the Armed Forces set down roots, providing more stability for themselves and their families.

Working as part of the Defence Human Capability Science & Technology Centre (DHCSTC) for Dstl, CORDA supported the MoD

by providing analysis and quantitative evidence, enabling the MoD to finalise details of the scheme and produce a strong business case to be put forward.

The DHCSTC is a centre of excellence for defence human capability research, delivering across a broad range of themes including Humans in Systems, Personnel, Influence Activities and Outreach, Health and Well-Being, and Tri-Service Training and Education. They are supported by a broad chain of suppliers including large defence organisations, Small and Medium Enterprises and academia.

CORDA ABROAD

In addition to supporting decision makers in the UK, CORDA also works abroad. It has supported BAE Systems’ business in the USA, Australia, Saudi Arabia and India. It also works with NATO, in particular the NATO

IMPACT | SPRING 2015

31

Communications and Information Agency (NCI Agency – formerly NC3A) in The Hague, Netherlands: a relationship that dates back to the 1990s.

CORPORATE MEMBER

CORDA is one of the first corporate partner of the OR Society. O.R. is the application of analytical methods to help make better decisions and the corporate partnership with the professional body of the field demonstrates the strength of expertise within the company.

To welcome CORDA to the fold, Gavin Blackett, the Secretary & General Manager of the OR Society, came to CORDA to give a lunch-time speech.

As well as talking about the benefits of membership to the OR Society, he also talked about changes that it has seen over the last 30 years, in recognition that 2014 was CORDA’s 30th year.

Judith Rawle, Head of O.R. at CORDA, believes that the partnership will bring many benefits to CORDA: “I am delighted that CORDA and the OR Society have agreed to become corporate partners. CORDA will benefit from access to the resources of the society to help with the development of its technical capability to achieve national standing in O.R. and Analytics. It will also show our customers that we are committed to the continuous development of our skills in supporting decision making.”

FUTURE CHALLENGES

As has been shown above, over the last 30 years O.R. and decision support

32

IMPACT | SPRING 2015

(at least as practiced by CORDA) has adopted and adapted new techniques to remain able to deal with the questions its customers need to address. It is likely that in the coming 30 years there will continue to be increasing consideration of a much broader spectrum of operations, especially those where the military is subordinate to other Government levers of power, such as economic or diplomatic considerations. This will be coupled with more emphasis on the ability to coerce, deter or deceive an opponent rather than physically defeating him.

The modelling of military operations in the current strategic environment presents many intellectual and practical challenges, but lies at the heart of most applications of O.R. in the defence domain.

There is an increasing emphasis on considering the peacetime management of defence

From an industry perspective, the concept of the Total Support Force will gain emphasis; this concept aims to achieve an enhanced capability through better exploitation of industrial expertise and capacity much nearer the front line than has been traditional in the UK military. All of this will be supported by an increasing understanding of human factors associated with decision making - the information that is used by decision makers, and how they interpret and exploit it.

The cost of delivering military capability will continue to be a core consideration, but not just the cost in pure financial terms. The unacceptability of casualties will drive strategic decision making towards greater use of unmanned assets, and the decreasing availability of fossil fuels will increase the focus on energy efficient systems.

CORDA has seen an increasing emphasis on considering the peacetime management of defence – items such as personnel planning, training policy, and vehicle fleet management.

All are topics that CORDA has been exploring, but need further development and exploitation with the appropriate level of decision maker to realise their full benefits.

Big data is very fashionable. The analysis of real operational data could hold the answers to some puzzles, and provide the key to understanding the drivers of perception and behaviour, whilst allowing the validation of models of irregular warfare and stabilisation operations.

To meet these challenges, CORDA will require method enhancements, increasing the interdisciplinarity of its study teams to include a greater range of softer skills. This will be coupled with increasing use of multimethodologies, for example using “softer” facilitation and problem structuring methods in concert with “hard”, numerical methods such as simulation or mathematical optimisation.

Noel Corrigan is a principal consultant for CORDA

Image © VILevi/iStock/Thinkstock

A H E A LT H Y O U T LO O K NEIL ROBINSON

SUCCESS OR FAILURE in a business setting often depends on understanding how all kinds of factors develop and interact – not just at any given moment but over months, years or even decades. As demonstrated by a recent review of doctor training in the UK, system dynamics can prove a powerful and cost-effective resource when tackling such issues.

Imagine a giant factory that produces a highly specialised and sophisticated product – say, for the sake of argument, a state-of-the-art, hand-crafted, luxury car that is painstakingly assembled at enormous cost.

This factory boasts the world’s longest production line. The rudimentary shell of the vehicle begins the journey at one end, and more than 15 years later, after countless additions and tweaks and

enhancements, the finished product arrives at the other.

The problem is that the line seldom changes speed. It might slow or quicken a tad every now and then, but on the whole it rumbles along at a uniform pace. This is unfortunate, because nobody really knows what the demand for such a car might be in 15 years’ time. Sometimes customers are left wanting; sometimes these beautiful, best-in-class products reach the end of the line and have nowhere to go.

Such a state of affairs sounds fanciful enough when applied to manufacturing, but what if it were applied to humans? We might think it inconceivable, but this sort of situation is all too common in the sphere of workforce planning.

For example, it can take more than 15 years and over half a million pounds in government funding to train

IMPACT © THE OR SOCIETY

33

a single specialist doctor in the UK. Thereafter the lifetime salary can exceed £2m. The sheer scale of these temporal and financial considerations poses major challenges, particularly when the notion that more should be done with less has become firmly rooted in economic reality.

As with our car factory, undersupply and oversupply are constant and unwelcome threats. The corollaries of training too few doctors – including negative impacts on the general health and wellbeing of the population – are hard to quantify but unmistakably significant. The principal consequence of training too many doctors – millions of pounds in unwarranted investment – is painfully simple to calculate.

In 2011 the Department of Health requested the help of the operational research community in addressing these risks. The answers it sought lay in the field of system dynamics, a cuttingedge approach to understanding and refining the behaviour of complex systems over time. The result was a

completely new method of workforce planning – one that is now expected to deliver substantial savings for the NHS.

FAIL TO PLAN...

“Effective workforce planning is all about ensuring the right people with the right skills are in the right place at the right time,” says Dr Graham Willis, head of research at the Centre for Workforce Intelligence (CfWI).

“This is an especially tough challenge in healthcare, given the mix of staff and their disparate functions, the large geographic areas involved and the changing policies that might affect demand and supply. That’s why we needed a solution that was unprecedented.”

CfWI is a key contributor to the planning of future workforce requirements for health, public health and social care in England. It is commissioned by the Department of Health, as well as Health Education England and Public Health England, to

look at specific workforce groups and pathways, provide materials, tools and resources and inform policy decisions.

The issue of training provision for doctors and dentists necessitated a comprehensive analysis of intakes at medical and dental schools and an in-depth understanding of the factors likely to mould them over the course of many years.

Poor workforce planning can put patients’ lives at risk and be wasteful

To appreciate the size and nature of the task Dr Willis and his team faced it is first essential to grasp several basics.

The medical and dental workforce is both extremely big and highly qualified. Trainees have more than 70 specialties from which to choose. The period between starting university and obtaining full qualifications lasts at least a decade for most and longer still

SYSTEM DYNAMICS The field of system dynamics emerged from the work of Professor Jay W Forrester. In 1956 Forrester accepted a professorship at the Massachusetts Institute of Technology’s new Sloan School of Management, where he set out to use his background in science and engineering to tackle issues of corporate success and failure.

Much of his formative research was carried out with General Electric, whose managers were concerned by the dominance of a three-year employment cycle at the company’s appliance plants in Kentucky. Forrester analysed factors such as GE’s decision-making processes for hiring and layoffs and was able to show the problem could be traced to solvable internal issues – not, as managers had previously thought, the business cycle or other external influences.

Forrester described system dynamics as “the investigation of the information-feedback characteristics of systems and the use of models for the design of improved organisational form and guiding policy”. It has since come to be acknowledged as a powerful means of studying the world around us and understanding how objects and factors develop and interact over time. It can be applied to anything from business models to social systems, from climate change to public policy – anything, indeed, whose drivers and behaviour we would like to better understand and further refine.

34

IMPACT | SPRING 2015

Image © UK Stock Images Ltd / Alamy

for those whose progress is interrupted by research or other commitments. The routes through the training process are myriad and frequently entwined.

Little wonder, then, that workforce planning of this magnitude and intricacy can be difficult. Historically, the lengthy timescales and profusion of influences have seriously hampered attempts to gauge the effect of policy shifts and to implement corrective responses. It is easy to see how inertia and delay, although not tolerated per se, could come to be regarded as inherent idiosyncrasies of the system.

Yet such idiosyncrasies can prove costly. “Poor workforce planning can put patients’ lives at risk and be wasteful,” says Dr Willis.

“Understaffing can leave employees stressed, while overstaffing can endanger livelihoods if jobs aren’t available. To guard against these possibilities you need foresight of the key issues and, just as importantly, the flexibility to adapt when necessary.”

Foresight is one thing; clairvoyance is quite another. Nobody can genuinely presage the future: the best we can do is prepare for it. Recognising as much, Dr Willis and his colleagues set about creating a scenario-based model that would permit much more informed long-term decision-making.

It was named the Robust Workforce Planning Framework, and nothing of its kind, as far as anyone was aware, had ever before been used in the healthcare arena.

NEW HORIZONS

In 2002, ruminating on the lack of evidence linking the Iraqi government to the supply of weapons of mass destruction to terrorist groups, the then US Secretary of Defence, Donald Rumsfeld, delivered his infamous

remarks on “known knowns”, “known unknowns” and “unknown unknowns”.

The spiel attracted widespread ridicule and eventually won the Plain English Campaign’s Foot in Mouth Award.

In truth, as a succession of academics and commentators have since pointed out, there was something to be said for Rumsfeld’s seemingly warped logic. It is no coincidence that the tale has become an amusing but instructive touchstone in the discipline of risk assessment. Risk analyst Nassim Nicholas Taleb, who had given a presentation to the Department of Defence shortly before the speech, even contends that “unknown unknowns” are usually responsible for the greatest societal change.

Similarly, it was vital that the Robust Workforce Planning Framework should incorporate everything from near-inevitabilities to apparent imponderables. The objective was to combine accepted facts, reasonable assumptions, controllable parameters and intrinsic uncertainties.

This required the ability to explore and examine prospective technological, economic, environmental, political, social and ethical influences on demand and supply – some of them predetermined, as in the case of an

ageing population, and some of them comparatively indeterminate, as in the case of technical advances.

“We resolved at an early stage that stakeholder engagement would be absolutely central to the framework,” says Dr Willis, “because it’s imperative to arrive at a shared view of future challenges when formulating policy decisions. This is particularly so in relation to what we call ‘horizon scanning’, which involves capturing and synthesising a broad array of expert opinions concerning the potential factors that should be taken into account. In essence, horizon scanning is about learning from the past, which is what we know, and surveying the future, which is what we would like to know.”

A dedicated website, www. horizonscanning.org.uk, was launched to encourage stakeholders to contribute their insights. Visitors were not only asked to submit ideas about the drivers that might come to shape medical and dental school intakes: they were also asked to provide quantitative interpretations of their suggestions – for instance, whether the likely impact would be high or low. Several hundred stakeholders offered their thoughts. This gave the team a rich source of material

IMPACT | SPRING 2015

35

and fed directly into the model’s second phase, scenario generation.

“Because it’s not possible to predict the long-term future accurately, scenarios are crucial for workforce planning,” says Dr Willis. “The aim is to envisage futures that are challenging but consistent, and to do that we have to establish contexts and set parameters. What’s needed is a range of plausible futures – including a ‘business as usual’ baseline – that can be used to project demand and supply. Any plans can then be tested against the different scenarios for robustness.

“A unique aspect of our design was the use of what’s known as a Delphi process. This is a forecasting method in which each member of a panel of experts states and explains his or her forecasts over a number of ‘rounds’, gradually revising them in light of fellow members’ responses. The intention is to converge towards an appropriate degree of consensus and, by extension, a ‘right’ answer. This was very useful in helping us come up with credible scenarios.”

With a wealth of expertise digested and distilled, the uncertainties realistically reduced and the resultant scenarios in place, workforce modelling – the simulation of demand and supply across the range of futures created – could commence in earnest. This would be phase three. It was time to harness the full power of system dynamics.

FAITH IN THE FUTURE

There are numerous reasons why the system dynamics methodology is well suited to workforce modelling. One is the relative ease with which it can integrate complicated schemata and datasets; another is that it is based on a graphical approach, which tends to save stakeholders from being hopelessly

36

IMPACT | SPRING 2015

overwhelmed by the depth and complexity of the information that confronts them.

“The first stages of developing the model revolved around mapping out the relevant processes,” says Dr Willis.

“These maps were actually printed out and shared with stakeholders. They were also presented at a series of national roadshows that we hosted, which allowed other interested parties to comment on and amend them. The Department of Health, the British Medical Association, the General Medical Council, the University and Colleges Admissions Service and many others – this was the standard of stakeholder buy-in that guided our modelling.”

When one bears in mind that the team set out to forecast demand and supply until 2040, to segment the doctor and dentist workforce by age and gender, to delineate every element of the training pipeline and every facet of the career paths that might follow and to merge sizeable datasets from the NHS, the Delphi discussions and elsewhere – and, for good measure, to achieve all of this while enabling rapid execution and analysis – the value of such an enviable level of stakeholder input becomes plain.

So, too, does the efficiency of system dynamics. Developed using Vensim and Excel and equipped with an Excel interface, one of the models Dr Willis and his colleagues produced contained 15 separate influence diagrams, almost a thousand distinct variables and more than 900,000 items of data upon initialisation; and yet simulation took place within 10 seconds.

Such flexibility and rapidity would be fundamental to the fourth and final phase, policy analysis. This was the crux of the matter: the capacity to stress-test decisions and identify those liable to prove robust against future

uncertainties. It was this facility, above all, that decision-makers had previously lacked.

“It’s all about revealing vulnerabilities and trade-offs,” says Dr Willis. “By evaluating the effectiveness of prospective policies across different future scenarios we can find which are the most desirable and which are the most difficult. We can also discern the signals that a favourable or unfavourable future might be unfolding, which means we can take mitigating action if needed. Ultimately, that’s what workforce planners want: not just to be able to react with speed but, better still, to be able to stay ahead of the game.”

IMMEDIATE IMPACT

Dr Willis and his team carried out their work between winter 2011 and autumn 2012. Shortly afterwards the findings were used to inform a major review by the Health and Education National Strategic Exchange (HENSE), which was set up by the Department of Health and the Higher Education Funding Council for England specifically to investigate medical and dental school intakes.

In its subsequent report, published in December 2012, HENSE noted: “The review group acknowledged the challenging nature of this work. With so many possible variables impacting on the potential demand for and supply of doctors and dentists over the next few decades, any such review [was] inevitably going to be difficult.

“Whilst [it is] clearly not possible for any model to be relied upon accurately to predict the future, the CfWI developed a sophisticated model that the review group believes could best inform its deliberations and those of future review groups looking at the same issue… The

SCENARIO GENERATION We know that the future is complex and uncertain. How long will doctors work in future? When might they retire? What will be the future health needs of the population? What will be the shape of the health system that drives the need for doctors?

Although we have historical data, forecasting the future by extrapolating the past can often lead to absurd results. Intuition and luck may lead to good forecasts, but there will be more bad ones, and we cannot recognise good from bad until the future has happened.

Success comes from understanding the dynamics of the system and what drives change. Our approach is based not on trying to forecast the future but on producing a range of plausible, challenging and yet consistent scenarios.

A scenario has two parts. The first is a narrative explaining how driving forces, events and responses may unfold over time to lead to a future. The second is a set of quantified parameters that define this future and feed into our system dynamics model.

We use these scenarios to test prospective workforce policies. Some policies may work well in all these futures, in which case we would say they are robust against future uncertainty. Others may work badly, and we might reject them.

Where the future is uncertain we can look at how it is unfolding and make gentle adjustments in response.

Dr Graham Willis

review group would like particularly to thank the team at the CfWI for their contribution to this project and for the development of new medical and dental models that should also be invaluable in informing future reviews.”

Underpinned by the Robust Workforce Planning Framework, HENSE’s recommendations included a 2% reduction in medical school intakes from 2013. This should equate to a considerable saving. The model also highlighted concerns over the data used to regulate dental school intakes.

HENSE recommended further reviews be undertaken every three years. Unlike the production line in our imaginary car factory, the system is finally capable of anticipating and responding to circumstance.

In February 2013 CfWI and its partner on the project, Decision

Analysis Services Ltd (DAS), won the prestigious Steer Davies Gleave Prize for System Dynamics. Dr Willis and Dr Andrew Woodward, CfWI’s lead modeller, accepted the award with DAS’s Dr Sion Cave. The competition promotes the use of system dynamics to tackle “real-world problems of significant public interest”.

The framework has since been used in more than 20 reviews of health and social care professions. Its underlying structure remains the same – horizon scanning, scenario generation, workforce modeling, policy analysis – but refinements and revisions are ongoing.

The Delphi process, for example, has been superseded by a state-ofthe-art elicitation method in which a small group of experts synthesise their knowledge to produce not a single

number – as was the case with Delphi – but a probability distribution representing the spread of uncertainty. This method is known as the Sheffield Elicitation Framework – SHELF. “The fact that our original design is still substantially unchanged is a testament to its simplicity and ease of use, and we’re very proud of that,” says Dr Willis. “But it was always our intention to keep improving it. As HENSE has noted, part of its appeal is that it can continue to be developed in the future – and the future, after all, is what it’s all about.”

Neil Robinson is the managing editor of Bulletin Academic, a communications consultancy that specialises in helping academic research have the greatest economic, cultural or social impact.

IMPACT | SPRING 2015

37

Picture: Stabilisation Study, Crown Copyright/Dstl 2011

S H A P I N G P E AC E S U P P O R T O P E R AT I O N S I N A F G H A N I S TA N ( 2 0 1 1 ) BRIAN CLEGG

A SOPHISTICATED WARGAMING technique has enabled an Operational Analysis team to give the best possible support to operational decisionmaking.

It is ironic that the military, the very field that led to the formation of Operational Research, or Operations Research in its US guise, is the only major user of the discipline that doesn’t call it that. O.R. began in the late 1930s when scientists (mostly physicists), including A. P. Rowe, Patrick Blackett and Freeman Dyson,

38

IMPACT Š THE OR SOCIETY

were brought in to apply scientific and mathematical techniques to improve the success of military operations. A classic example would be devising the pattern of depth charges most likely to knock out a submarine.

Numerous military O.R. groups were formed during the Second World War and the discipline rapidly established a permanent place for itself as part of the scientific support available to defence planners. However, the military is rife with acronyms, and in the mid-1960s, the Ministry of Defence (MoD) decided to rename

Picture: Stabilisation Study, Crown Copyright/Dstl 2011

THE PEACE SUPPORT OPERATIONS MODEL

the activity Operational Analysis (OA), to avoid confusion with another O.R., Operational Requirements.

After a string of reorganisations and amalgamations, the work now comes under the OA wing of the MoD’s Defence Science and Technology Laboratory (DSTL), based at Portsdown West and Porton Down.

From the point of view of Colin Marston, Principal Analyst at DSTL, OA is the application of scientific techniques to aid decision-making. Such evidencebased processes can inform complex military problems involving people, machines, materials or money. And, as Marston comments, ‘For any work I am involved in, my main goal is that it is “useful” and “used”!’

Marston joined DSTL in 2005 after completing a degree in physics with astrophysics and serving in the Territorial Army for three years. His main areas of work have been in support to the operations environment and the defence policy arena, with a recent focus on cyber and wargame related projects. Working with DSTL has provided Marston with the opportunity to be involved in numerous international research collaborations with partners such as NATO, and nations such as the US, Canada, Australia, Sweden, Netherlands and Japan.

Marston’s most complex DSTL task to date has been the development of the Peace Support Operations Model (PSOM). PSOM (pronounced ‘Possum’) was initially designed to assist decision-making relating to how best to structure forces, what balance there should be between military and civilian organizations, what training might be required etc. But it has also proved to be invaluable for giving insights into the planning of operational campaigns and assessing the impact of possible courses of action.

PSOM is more than just a computer simulation - it is a human-in-the-loop, computer-assisted wargame

PSOM is a computer simulation, which supports decision-making for particular types of complex operation. Such operations include those that involve stabilising a dangerous situation, counter insurgencies and ‘irregular warfare’ (a blanket term to describe the kinds of situation we are all too familiar with from nightly news bulletins). But PSOM is more than just a computer simulation - it is a human-in-the-loop, computer-assisted war-game.

‘PSOM is used as part of a wider war-game to seek solutions to customer problems. It is both the PSOM model and the human players that drive the direction that the game takes,’ Marston observed. A pure simulation could be run in isolation on a computer once the initial conditions are set up, but a wargame is very much about facilitating human decisions.

GAMES OF WAR

Traditionally, war-games were manual simulations of real operations, but after the Second World War, with the increasing availability of computers and the growing use of OA, some aspects of the process could be digitally simulated. Sometimes the gaming was at a highly strategic level, most notably in the application of game theory to Cold War concerns.

This was responsible for producing the ‘Mutually Assured Destruction’ doctrine that suggested that when both sides in a conflict have the ability to annihilate their opponent, the outcome should be deterrence. In other cases, the use of war-games was more at the level of practical modelling – a more sophisticated equivalent of the popular computer games like Command & Conquer, Total War and Civilization, which allow players to try out different tactics in battles and social environments.

For a time, following the end of the Cold War, it seemed that the use of conventional war-games might be

COLIN MARSTON RECEIVING THE O.R. SOCIETY'S PRESIDENT'S MEDAL FOR THIS WORK, FROM PRESIDENT GEOFF ROYSTON

IMPACT | SPRING 2015

39

in decline. With only one superpower, the approach taken by earlier forms of war-gaming seemed less appropriate.

However, the defence arena has moved on once more. Today’s melting pot of influences that need to be understood and interpreted in many hotspots produces a very different environment.

It is now common for the military to be faced with a situation that is not a straightforward battle where one side wins or loses. The machinesupported human process at the heart of a war-game has come back into the ascendancy.

In principle, the kind of war-game undertaken using PSOM could be run without support from the model, but in practice the situation is so complex, involving so many variables and influences, that the software is essential to make the process effective. The computer program provides structure to the process, tracks what is happening (and what caused what) and acts as a virtual environment that constrains the players to what might be actually possible, giving them feedback on the potential outcome of their decisions.

This allows for further iterations to consider new possibilities.

PUTTING THE EXPERT INTO THE SYSTEM

The method used by PSOM seems at first sight to have a resemblance to the approach taken in the ‘expert systems’ that were put forward as universal panaceas by computing academics during the 1990s. The idea was that the knowledge and skills of subject matter experts could be captured and codified. It was then hoped that the computer model could replace the experts in diagnostic processes and planning, freeing up expert time for more complex requirements, or even

40

IMPACT | SPRING 2015

displacing expensive experts entirely with tireless computer systems that could work 24/7 at far less cost.

In reality, many of the muchvaunted expert systems came to nothing. It proved extremely difficult to elicit the knowledge from the experts and to translate this complex information into simple enough rules to work in the system. After many years of development the only applications that have emerged into real world use have tended to be simple diagnostic tools, like the ones used in a call centre when a customer rings up with a problem with his or her broadband.

The experts have hung onto their well-paid positions and have not been replaced by computer models.

What happens in PSOM does indeed have some similarities to the original vision of expert systems – but employed in a very different fashion.

In a complex scenario there may be a thousand events taking place each month. These come together to form a run of the game which could cover six months or more development of the situation. A subject matter expert would be able to apply his or her judgement to any single event, but dealing with them all in a manageable timeframe would be impossible.

PSOM captures the judgements of the experts and applies them in the form of a series of algorithms at high speed in a consistent fashion across the many events to give an overall outcome. Unlike an expert system, the expertise is not codified and static, but is developed to fit the scenario, ensuring the latest thinking and most appropriate expertise is applied.

War-games using PSOM will usually be complex affairs with a wide range of personnel involved. A typical small game might involve four ‘players’, and a pair of support staff, interacting with

the system over two days. For a largescale game, there could be over 100 players with 16 support staff and the process can easily take up to five days to complete.

RUNNING THE PSOM WAR-GAME

A PSOM run consists of two parts. The first is the Strategic Interaction Process.

This is a structured role-play exercise to establish the key political forces and other influences outside the operational theatre. This is then used as part of the input to the Operational Game. Here the PSOM software allows the players to implement plans by placing units on a computerised map and assigning tasks to them. The software then simulates the outcome of these tasks, given the environment and range of influences.

Part of the complexity of the model is the need to cover a whole range of activities outside of the traditional military role. The units can be tasked with building infrastructure and providing aid to civilians, for instance.

Broad economic factors are taken into account, such as employment and the availability of illicit sectors, such as drug trafficking. And the outputs are far more than levels of casualties and any concept of ‘winning’ or ‘losing’.

For example, the model will judge the degree of popular support that is given to the various factions represented in the model, the level of security (or lack of it) experienced by the civilian population and the level of threat they face.

To give a feel for the sheer volume of factions that have been considered in war-games using PSOM, Marston lists ‘Other Government Departments (OGDs), International Organisations (IOs), Non-Governmental Organisations (NGOs), the ‘unaligned’

Picture: Stabilisation Study, Crown Copyright/Dstl 2011

civilian population, formed military units, police, militia groups, insurgent organisations, private security companies, organised crime groups, narcotics traffickers… and so on.’ This is not a clean, relatively simple decision of the kind often faced by business, but an extremely messy environment with many conflicting influences.

INTO AFGHANISTAN

The original idea for PSOM, devised to support future UK strategic defence planning, dates back to 2004, with development over a number of years including an intense verification and validation process in 2008. After its first major deployment in the NATO ARRCADE FUSION exercise in 2009, PSOM had a very significant role in March 2011, when 16 members of the DSTL team flew out to Kabul to support a week-long conference at the International Airport in which more than 150 military and civilian personnel deployed in Afghanistan would come together to use the wargaming process to test out their plans for the following year.

This is the most effective tool for war-gaming at the higher levels I have ever experienced

The conference, and follow-up event in November, were held at the request of the International Security Assistance Force (ISAF) Joint Command, which was responsible for the combined Coalition and Afghan military campaign across the country. This was a critical time for ISAF. It was a result of the process that consensus was reached on the way forward for the campaign.

GENERAL KARIMI, CHIEF OF THE AFGHAN NATIONAL ARMY, AND THE SENIOR US COMMANDER, LT GENERAL SCAPARROTTI, AT A DAILY WAR-GAME BRIEFING SESSION, BEFORE DETERMINING THE STRATEGIC DIRECTION TO ALL THE PARTICIPANTS FOR THE NEXT DAY OF ACTIVITY

Brigadier (then Colonel) Gary Deakin representing the British Army commented: ‘The use of the war-gaming tool PSOM enabled Commanders and their planning staffs to objectively visualize the likely outcomes of the transition campaign plan for Afghanistan. Use of the tool, and critically the supporting team of civilian staff who made it work, enabled the planners to think at the strategic, operational and tactical levels identifying and developing understanding of the risks to the plan and how they should be mitigated.

The tool challenged planners to think laterally, attacked group-think and challenged cognitive dissonance.

The outcomes directly informed Commanders decision-making processes.

‘Almost 3 years on, and having been in involved directly or indirectly in the Afghanistan campaign since, whilst working in NATO and now in US CENTCOM, I have frequently observed events and trends which were identified as key risks to the plan in the war-gaming. This is the most effective

tool for war-gaming at the higher levels I have experienced.’

Clearly, with such a complex application, it is important that the model is validated as much as is possible. In the case of PSOM this has been done both by examination of the output by academic and military subject area experts, and by comparison of the output against historical data. As far as can be judged, the model’s output fits experience, though inevitably what happens in a war-game is strongly reliant on the actions taken by the players.

Marston points out the importance of being aware of the limitations of any such approach. ‘It would be wrong to say that PSOM could ever be 100% valid. It is at this point that George Box’s quote comes to mind: “Essentially, all models are wrong, but some are useful.” And this is something we need to be mindful of when we use OA to support customer problems and questions. Essentially for me, it is all about the “fitness-for-purpose” – so is it right to use a particular model in a particular situation?’

IMPACT | SPRING 2015

41

PSOM does not attempt to predict the future. Instead, by identifying what is likely to happen if certain decisions are made, PSOM gives both military and civilian decision makers valuable insights into the possible outcomes of different plans. As Marston puts it ‘Suppose an event of interest happens during a PSOM game. The model adds value by identifying the event itself, but also by helping the customer answer questions like “Is this event likely to happen?”, “What does it mean for us?” and “What could we do to mitigate its effects or prevent it from happening at all?”’

DELIVERING OPERATIONAL ANALYSIS

By March 2014 PSOM had been thoroughly debugged and documented and thus was in a position where it could be distributed for use by a

number of interested groups without technical support from the developers, though its use will always involve analysts as well as players.

Delivering technical support to the two PSOM events in Afghanistan was a demanding but highly rewarding experience for the DSTL team. The real reward was the ability to deliver added value, based on robust operational analysis principles, to senior decision makers leading real world operations at a critical time. Precise measurements of success in such a complex and dynamic environment are difficult to ascertain, but a telling quote comes from the ISAF Joint Command summary of the 2011 Afghanistan exercise. ‘You all have raised issues that a coalition and combined team, hundreds of thousands strong, have not thought all the way through to the finish. That early catch will save many lives as well as be critical to the success of the future campaign.’

Brigadier Deakin summed up the wider contribution of OA to the MoD: ‘Operational Analysis enables Commanders to make decisions.

Based on objective rigorous analysis by subject matter experts who are not afraid to challenge conventional thinking, Operational Analysis enables Commanders to better understand the likely outcomes of their decisions and to be able to identify the key risks and opportunities.’

As long as there have been wargames, there have been tools to support them, from diagrams drawn in the sand to the sophisticated computer modelling that sits at the heart of PSOM. By providing the big picture and a safe environment to run multiple scenarios, Operational Analysis is enabling the modern military to undertake their work in the most effective fashion, as the discipline has done since the Second World War.

THE PRIMARY ELEMENTS OF A WAR-GAME EMPLOYING PSOM: Phase 1: Preparation and understanding (takes from six weeks up, though can be significantly reduced if a similar scenario can be used as a starting point)

• Determining the aims and objectives (what is the question?)

• Determining the setting and scenario • Collecting the relevant data • Selecting and setting up the simulation(s) (manual, computer or mixture) • Determining the rules, procedures and adjudication • Identify players required Phase 2: Testing and adjusting

• Conducting testing and rehearsals (including information capture) • List assumptions • Confirm player roles Phase 3: Delivering the event

• Playing the game • Conducting in-game analysis • Conducting an after action review to cover observations, insights and lessons.

• Conducting post-game analysis (if required)

42

IMPACT | SPRING 2015

ENGINEERING HAPPINESS? Geoff Royston

income). In many industrialised countries, although average income has more than doubled in real terms over the last fifty years, average happiness, as measured in surveys, has pretty well flatlined.

A strand of related work has been on the societal impact of inequality, popularised by Kate Pickett and Richard Wilkinson’s influential book The Spirit Level: Why Equality is Better for Everyone. This argued that, for affluent societies, a wide range of social phenonema - life expectancy, health, and happiness - are affected more by how unequal a society is than by its level of wealth and that the bigger the gap between rich and poor the worse it is for everyone including the wealthy!

it is worth focusing on what will bring you most happiness CAN YOU ENGINEER - OR EVEN MEASURE - SOMETHING AS INTANGIBLE AS HAPPINESS?

Until quite recently the suggestion that happiness is something that could – or should - be investigated in any rigorous way was unlikely to be met with much scholarly enthusiasm. Those few academics who chose to study the topic were regarded as eccentric. But maybe not any longer. Work and publications such as Richard Layard’s Happiness: Lessons from A New Science have entered the public consciousness. One country, Bhutan, got into the news for measuring not only its GDP but also its GNH Gross National Happiness and in 2011 the UN General Assembly passed a resolution inviting member countries to measure the happiness of their people and to use this to help guide their public policies (and there is now a regular World Happiness Report). At a more individual level, new sophisticated techniques such as MRI scanning of neural activity are adding to our ability to measure happiness and its correlates. And now a book - Engineering Happiness: A new approach for building a joyful life - by Manel Baucells and Rakesh Sarin has recently won an award in operational research/management science - the INFORMS Decision Analysis Society Publications Award 2014. Time, perhaps, to sit up and take notice?

One stimulus for such interest has been the observation - backed up with extensive research – that societies, once they have achieved a certain level of material wealth, do not then get noticeably happier as they get richer (more exactly, happiness shows exponentially diminishing returns to

While controversy still surrounds such work, others have moved on to considering the next step – assuming some or all of this is true, what can we do about it? At the level of society that is clearly a political question, but what about action that people can take individually? Paul Dolan, professor of behavioural science at the London School of Economics, sets out some ideas in his book Happiness by Design: Change What You Do, Not What You Think, in which he argues that happiness, defined as experiences of pleasure and purpose over time, depends not so much on what happens to you but on how you process those inputs – how much attention you give them. Your attention time and energy is limited; so it is worth focusing it on what will bring you most happiness.

For example, continued focusing on one initially happinessgenerating activity (e.g. eating) will result in diminishing returns of well-being. Dolan argues that this is at least as much an issue of external environment as of internal psychology – and suggests designing your environment so it makes it easier for you to direct time and attention towards what will make you happier and away from what will not (e.g. not leaving tempting food on display when you are trying to diet)– echoes here of Thaler and Sunstein’s Nudge.

Which takes us on the OR/MS contribution and the INFORMS prize-winning book Engineering Happiness.

This seeks to pull together much of the latest thinking and research on happiness. Baucells and Sarin highlight six major principles, which they call the “laws of happiness”, which they claim to be universal, though acknowledging that

IMPACT © THE OR SOCIETY

43

The authors argue that these four laws provide a model of human psychology that is simple but sufficient to make predictions of how much happiness will be produced under different scenarios in which people are making choices. They analyse some scenarios, formulating them as optimisation problems which they then solve by standard OR/MS techniques. For example they consider a commuter with a given budget to spend on bus fares or car rental, and show that, with the given costs and values, the optimal, happiness maximising, strategy is to progressively “trade up” , starting with using the bus and saving renting the most expensive car for last. This avoids too early an increase in expectations and also avoids a later sense of loss that would be caused by splashing out at the start and running out of money.

(It may be unlikely for there to be data, or time, for exact application of such a model in practice, but the main value is, as so often, in demonstrating how a simple modelling exercise can produce useful broad insights).

• prioritising “basic” goods - like food, health, music, sex, friendship - and “cumulative” goods - like learning a skill, or developing relationships - for both of which expectations tend not to outstrip reality (provided they are consumed at a steady rate and not past satiation point); • being cautious ( e.g. adopting strategies like “leave the best to last”) when consuming “adaptive” goods, where expectations keep rising with consumption and there is a risk of not being able to keep up; similarly not allocating too much time and effort to goods for which comparison with other people is important and you can’t compete; • using “reframing” to extract more happiness from the same reality (the “glass half full” versus the “glass half empty” approach) and avoiding the “choice anxiety” that can be created by dwelling on past choices and comparing with what might have been.

So. Can – or should - happiness be engineered? Maybe, or at least landscape gardened. Read some of these books and decide for yourself!

the authors propose a number of practical ways to “engineer” happiness

Adding their fifth law, which concerns how good (or bad) experiences produce less impact on happiness the closer together they occur or the less novel and varied they are, and their sixth, which relates to our tendency to forecast future preferences and emotions as more similar to current ones than they actually will be, allows the authors to propose a number of practical ways to “engineer” happiness, based on the underlying view that happiness depends on choices.

These include:

44

IMPACT | SPRING 2015

Dr Geoff Royston is a recent president of the O.R. Society and a former chair of the UK Government Operational Research Service. He was head of strategic analysis and operational research in the Department of Health for England, where for almost two decades he was the professional lead for a large group of health analysts.

Cover courtesy of University of California Press Cover courtesy of Sustainable Development Solutions Network/Sunghee Kim

they are much less precise than the laws of physics. Their first four laws are, in summary: • Happiness is triggered by comparison of what you get with what you expect • Expectations change (adapting to past events and to what other people get) • Losses are felt more keenly than equivalent gains • Happiness shows diminishing sensitivity (the tenth bite of cake is less delicious than the first)

EURO2015

Glasgow, 12-15 July

Interested in making the ‘real-world’ more effective?

EURO2015 is the premier European conference for Operational Research and Management Science (ORMS), organised by the European Association of Operational Research Societies in conjunction with the OR Society.

EURO2015 takes place in the University of Strathclyde featuring all branches of ORMS. The event is for Consultants, Analysts and Decision Makers – in fact, anyone with a passion for making the ‘real-world’ more effective.

For more information on all aspects of this amazing conference and to register, go to www.euro2015.org

Make optimized decisions.

Even if your data is incomplete.

Robust optimization at both the solver and modeling level.

Now part of FICO® Xpress Optimization Suite.

Missing data and the challenges of harnessing Big Data have introduced a lot of uncertainty into the process of solving complex optimization problems.

FICO has solved that problem.

We’ve enhanced FICO® Xpress Optimization Suite with features to handle the difficulty of uncertainty introduced by predictive analytics data. This robust optimization guarantees feasible solutions in the face of unknowns.

Learn more about the Xpress Optimization Suite: fico.com/xpress

© 2015 Fair Isaac Corporation. All rights reserved.