Solver International - September 2017 by Lionheart Publishing Inc.

SE P T E MBER 2017 WWW.SOLVER-INTERNATIONAL.COM

TUTORIAL

CASE STUDY

EXECUTIVE INTERVIEW

Risk Analysis: A Breath of Better Decisions and Success

Recruiting through Analytics Tapan Patel â&#x20AC;&#x201C; SAS

solver... International

Educating and Empowering the Business Analyst

Solver-International.com Solver-International.com provides immediate information, exclusive articles, and updated news for businesses and academics. The Website also offers daily and weekly access for everyone seeking the latest tools to improve their analytic capabilities.

Welcome to Analytic Solver ® Cloud-based Data and Text Mining that Integrates with Excel

Everything in Predictive and Prescriptive Analytics Everywhere You Want, from Concept to Deployment. The Analytic Solver® suite makes powerful forecasting, data mining and text mining software available in your web browser (cloud-based software as a service), and in Microsoft Excel. And you can easily create models in our RASON® language for server, web and mobile apps.

Full-Power Data Mining and Predictive Analytics. It’s all point-and-click: Text mining, latent semantic analysis, feature selection, principal components and clustering; exponential smoothing and ARIMA for forecasting; multiple regression, logistic regression, k-nearest neighbors, discriminant analysis, naïve Bayes, and ensembles of trees and neural networks for prediction; and association rules for affinity analysis.

distributions, 50 statistics and risk measures, rankorder and copula correlation, distribution fitting, and charts and graphs. And it has full-power, point-and-click optimization, with large-scale linear and mixed-integer programming, nonlinear and simulation optimization, stochastic programming and robust optimization.

Find Out More, Start Your Free Trial Now. In your browser, in Excel, or in Visual Studio, Analytic Solver comes with everything you need: Wizards, Help, User Guides, 90 examples, even online training courses. Visit www.solver.com to learn more or ask questions, and visit analyticsolver.com to register and start a free trial – in the cloud, on your desktop, or both!

Simulation/Risk Analysis, Powerful Optimization. Analytic Solver is also a full-power, point-and-click tool for Monte Carlo simulation and risk analysis, with 50

Tel 775 831 0300 • Fax 775 831 0314 • info@solver.com

S e p te mbe r 2017 • Vo lume 1, Number 2 • ww w . s ol ve r -i n te r n a t i on a l . com

CONTENTS Cover Story

Risk Analysis: A Breath of Better Decisions and Success Uncertainty is the only certainty there is, and knowing how to live with insecurity is the only security.”

Tutorial

Optimization for Better Decisions Optimization requires that we define, in quantitative terms, a model that specifies all the ways, times or places our resources may be allocated, and all the significant constraints on resources and uses that must be met. Here is the way to do that.

Solver International | September 2017

Solver-International.com

Technology

Solver Gets a Facelift – and New Muscle Solver, the optimization software, has myriad uses in industries from manufacturing to distribution and logistics, marketing and sales, finance and investment, and human resources. Now it is even stronger and easier to use.

Case Study Executive Interview

Tapan Patel— SAS Institute Tapan Patel is Principal Product Marketing Manager at SAS.

The University of Oklahoma Recruiting students to a university can be a challenge when the data isn’t handled properly. Meeting the challenge can also be a challenge.

22 COL UMNS & DE PA RT ME N TS 4 Off the Top 6 Impact Analytix 8 Industry News 54 INFORMS Society News 56 Glossary

Educating and Empowering the Business Analyst

September 2017 | Solver International

Off the

TOP

Tom Inglesby, editor tom@solver-international.com

Looking Back In July of 2016, a man who taught how analysis could become second nature passed away. Howard Raiffa was 92 when he died and 48 when I met and worked with him. Raiffa was an economics professor whose mathematical formulas for decision making were applied to the search for a missing nuclear bomb and the siting of a Mexico City airport. Professor Raiffa co-founded the Harvard Kennedy School and taught at Harvard for 37 years. One of his claims to fame was developing and teaching what became known as decision science, a discipline that includes risk analysis, negotiating techniques, conflict resolution, and even game theory. Instead of focusing on theory, he applied his concepts to real-world cases of the three C’s—conflict, cooperation, and compromise. “I learned a lot about the theory and practice of many-party negotiations in the presence of extreme cultural differences,” he was quoted in his New York Times obituary. In an interview, Prof. David E. Bell of the Harvard Business School said: “Howard came up with brand-new theories that helped us understand how we should make decisions in a wide variety of circumstances. These were practical approaches, not ivory tower constructs.” The best practical advice, Raiffa wrote, is “to maximize your expected payoff, which is the sum of all payoffs multiplied by probabilities. The art of compromise centers on the willingness to give up something in order to get something else in return.” According to the Times piece, his students engaged in cutthroat simulated negotiations, prompting The Harvard Crimson to ask him whether the curriculum taught students to lie in actual business dealings. He replied by relating a story about the former president of the University of Chicago. “When, in the 1950s,” he began, “Robert Hutchins was hauled before a congressional committee and asked if it was true that the University of Chicago taught communism, he replied: ‘Yes. And in the medical school we teach cancer.’” “It’s a valid analogy,” Raiffa said. “To deal with a problem, we have to teach about it.” Students of his recall he would close the course with, “When we see we could improve our profit or further maximize our desired result, we might ask, Is this a ‘dirty dollar’ or a ‘clean one’ that we could earn? What would happen if everybody did this? Would we be able to sleep at night if we did this? How would we feel if we had to explain this to our families? “I hope that in answering these questions, you will favor the course of action embracing a higher moral standard.” Raiffa died before the 2016 elections, the results of which might have tested his theories to the limit. RIP, mentor. Si

Solver International | September 2017

SOLVER INTERNATIONAL DIGITAL MAGAZINE A JOINT VENTURE BETWEEN FRONTLINE SYSTEMS, INC. AND LIONHEART PUBLISHING, INC.

SOLVER INTERNATIONAL ADVERTISING AND EDITORIAL OFFICE Send all advertising submissions for Solver International to: Lionheart Publishing Inc. 1635 Old41 Hwy., Suite 112-361, Kennesaw, GA 30152 USA Tel.: 888.303.5639 • Fax: 770.432.6969 Email: lpi@lionhrtpub.com URL: www.lionheartpub.com

PRESIDENT John Llewellyn, ext. 209 llewellyn@lionhrtpub.com Direct: 404.918.3275

EDITOR Tom Inglesby tom@solver-international.com Direct: 760.529.9437

NEWS SUBMISSIONS editor@solver-international.com

ART DIRECTOR Alan Brubaker, ext. 218 albrubaker@lionhrtpub.com

ONLINE PROJECTS MANAGER Patton McGinley, ext. 214 patton@lionhrtpub.com

ASSISTANT ONLINE PROJECTS MANAGER Leslie Proctor, ext. 228 leslie@lionhrtpub.com

ADVERTISING SALES MANAGERS Sharon Baker sharon@lionhrtpub.com Direct: 813.852.9942 Aileen Kronke aileen@lionhrtpub.com Direct: 678.293.5201

REPRINTS & SUBSCRIPTIONS Kelly Millwood, ext. 215 kelly@lionhrtpub.com

FRONTLINE SYSTEMS, INC. P. O. Box 4288, Incline Village, NV 89450 www.solver.com

Solver International is published bimonthly by Lionheart Publishing, Inc. in cooperation with Frontline Systems, Inc. Deadlines for contributions: Manuscripts and news items should arrive no later than three weeks prior to the first day of the month of publication. Address correspondence to: Editor, Solver International, Lionheart Publishing, Inc., 1635 Old 41 Hwy., Suite 112-361, Kennesaw, GA 30152. The opinions expressed in Solver International are those of the authors, and do not necessarily reflect the opinions of Lionheart Publishing Inc., Frontline Systems, Inc. or the editorial staff of Solver International. All rights reserved.

Solver-International.com

Welcome to Analytic Solver ® Cloud-based Simulation Modeling that Integrates with Excel

Everything in Predictive and Prescriptive Analytics Everywhere You Want, from Concept to Deployment. The Analytic Solver® suite makes the fastest Monte Carlo simulation and risk analysis software available in your web browser (cloud-based software as a service), and in Microsoft Excel. And you can easily create models in our RASON® language for server, web and mobile apps.

Comprehensive Risk and Decision Analysis Tools. Use a point-and-click Distribution Wizard, 50 probability distributions, automatic distribution fitting, compound distributions, rank-order correlation and three types of copulas; 50 statistics, risk measures and Six Sigma functions; easy multiple parameterized simulations, decision trees, and a wide array of charts and graphs.

nonlinear optimization, simulation optimization, stochastic programming and robust optimization. And it’s a full-power tool for forecasting, data mining and text mining, from time series methods to classification and regression trees, neural networks and more, with access to SQL databases and Spark Big Data clusters.

Optimization, Forecasting, Data and Text Mining. Analytic Solver is also a full-power, point-and-click tool for conventional and stochastic optimization, with powerful linear and mixed-integer programming,

Tel 775 831 0300 • Fax 775 831 0314 • info@solver.com

Impact

ANALYTIX

Jen Underwood

Prescriptive Analytics on the Rise

hile you were skimming through a plethora of artificial intelligence and deep learning headlines, prescriptive analytics has quietly been on the rise. In 2017, more companies started looking at prescriptive analytics—Gartner predicts this niche market to grow to $1.1 billion by 2019 – 22 percent compound annual growth rate (CAGR) from 2014. What is prescriptive analytics? Simply put, prescriptive analytics provides the best options for given

situations based on the concepts of optimization. Prescriptive analytics uses the insights revealed by predictive analytics and provides a call to action based on what is found. It analyzes current data sets for patterns and evaluates the outcomes of the multiple scenarios that could be enacted, based on decisions that could be made based on the data, providing decision makers with hypotheticals on the impact of each option. Prescriptive analytics lies at the far end of the analytics

Figure 1: The Analytics Continuum

Solver International | September 2017

maturity spectrum that starts with descriptive analytics, progresses to diagnostic analytics, predictive analytics, and finally finishes with prescriptive analytics. Today we are seeing increased investment from data discovery and mainstream analytics vendors in basic prescriptive capabilities such as what-if analysis. Approximately 10 percent of organizations currently use some form of prescriptive analytics, according to Gartner, but this will grow to 35 percent by 2020. 1 RIGHT TIME, RIGHT PLACE Although prescriptive analytics has exceptionally high business impact potential, it can become overwhelming and complex rather quickly. As a result, this area of analytics is often an untapped, truly golden window of opportunity to explore in most organizations. A continued explosion of data combined with vast improvements in prescriptive analytics technology, broader access in mainstream analytics

Solver-International.com

tools, and ease of use should be the perfect formula to develop interest in this domain. The digital era of big data started the momentum for the predictive and prescriptive analytics market. Organizations are reinventing, utilizing new models, and on a mission to improve business outcomes. Prescriptive analytics, simulation of alternatives, and optimization of options can be extremely useful, providing unbiased tools to efficiently solve planning problems and help improve decision making. Simulations allow you to evaluate new ideas before you make a complex business decision. This analysis technique lets you test different parameters, such as pricing and costs, to discover opportunities for improvement in your current operations. Simulation helps provide clarity of what might happen if you take certain actions. Applying these results to your business helps you manage risk and make better choices. Optimization approaches apply linear programming to decision making on problems that can be expressed in terms of a linear objective function and linear constraints on the decision variables. It is a powerful technique for maximizing or minimizing a target variable, such as budget, while satisfying operational constraints.

NEW WAY OF THINKING For many organizations, prescriptive analytics projects introduce a new way of problem solving and thinking. The iterative exercise of developing an accurate prescriptive business model is still a bit of an art that forces you to think through different scenarios and variable combinations. Large to midsize organizations are moving away from â&#x20AC;&#x153;gut-feelâ&#x20AC;? decision making, and instead are using more sophisticated analytics and fact-based decisions to project future trends and optimize business decisions. 2 There are some common misconceptions that prescriptive and advanced analytics in general require a data warehouse or data scientist. Although both a data warehouse and data scientist are incredible resources, there are simple methods that you can use to get started using a basic spreadsheet. Using spreadsheets, you can start learning how to design prescriptive analytics business models with variables, explore what-if scenarios, and perform simulations of what might happen if certain conditions exist. To learn more about optimization and simulation techniques, check out Spreadsheet Modeling and Decision Analysis by Cliff

Educating and Empowering the Business Analyst

Ragsdale and Management Science: The Art of Modeling with Spreadsheets by Powell and Baker. Those books were recommended to me several years ago. I also feel it helps to understand statistics since many algorithms are based on statistical concepts. Si Forecast Snapshot: Prescriptive Analytics, Worldwide, 2016 https://www. 1

gartner.com/doc/3202617/ forecast-snapshot-prescriptiveanalytics-worldwide

Gartner Hype Cycle for Business Intelligence and Analytics, 2016 2

Jen Underwood is Founder and Principle of Impact Analytix, LLC. Impact Analytix is a boutique integrated product research, consulting, technical marketing and creative digital media agency led by experienced handson practitioners. Jen can be tweeted at @idigdata

September 2017 | Solver International

Industry

NEWS

Analytic Solver for Cloud and Desktop Business analysts gain new power and flexibility to build predictive and prescriptive analytic models.

F “We’re making the fruits of their efforts readily accessible to everyone who needs them.”

rontline Systems is shipping a new product line for desktop and cloud, Analytic Solver® 2017-R2 for Excel and enhancements to AnalyticSolver.com, its SaaS (software as a service) equivalent. Building on the February V2017 release that unified and simplified Frontline’s product and introduced Analytic Solver Basic, the V2017-R2 release features major performance improvements in optimization, new distribution and correlation functionality in simulation/ risk analysis, and a new, simple way to convert Excel analytic models into Power BI® custom visuals, for distribution to business intelligence consumers throughout an organization.

Solver International | September 2017

“We’re not only making advanced analytics power tools and learning aids accessible to every business analyst,” said Daniel Fylstra, Frontline’s President and CEO. “We’re making the fruits of their efforts – working analytic models and their insights – readily accessible to everyone who needs them.” Microsoft’s popular Power BI business intelligence and data visualization platform now supports custom visuals, originally conceived as a way to broaden the range of chart types available in Power BI dashboards. Independent developers, using an array of special Web development tools such as TypeScript (a strongly typed JavaScript programming language) and the Node.js framework, can create custom visuals

Solver-International.com

to be used in Power BI. But for Excel-literate business analysts, creating their own Power BI custom visuals has been far out of reach. All that has changed with Analytic Solver V2017-R2. Working in Excel with this tool, a business analyst user can turn his or her Excel-based optimization or simulation model into a Power BI Custom Visual, with just a few mouse clicks. Users simply select rows or columns of data to serve as changeable parameters, then choose Create App – Power BI, and save a custom visual file created by Analytic Solver. They click the Load Custom Visual icon in Power BI, and select the file just saved. The result isn’t just a chart – it’s a fully-functional optimization or simulation model, ready to accept Power BI data, run on demand on the Web, and display visual results in Power BI. Users simply need to drag and drop appropriate Power BI datasets into the “well” of inputs to match model parameters. THE MAGIC OF RASON® The secret behind the ability to turn desktop Excel models into Web-based Power BI custom visuals, without any programming, is Analytic Solver’s ability to translate an Excel model into RASON (RESTful Analytic

Solver Object Notation), Frontline’s powerful modeling language embedded in JSON. While not every model can be automatically translated, a wide array of simulation/ risk analysis and linear and nonlinear optimization models can be. Analytic Solver then “wraps” Power BI custom visual code around the RASON model. The result is a radically simple way to move from traditional Excel to cloud deployment to business intelligence consumers – not available anywhere else.

Frontline has seen speed gains of 10 to 50 times on real customer models.

PERFORMANCE GAINS FOR LARGE MODELS For large optimization models with tens to hundreds of thousands of decision variables, the V2017-R2 release greatly improves the speed of “problem setup,” where a Jacobian matrix is built from many thousands of Excel formulas. Though individual results vary, Frontline has seen speed gains of 10 to 50 times on real customer models. These improvements benefit all of Frontline’s 12 different “Solver Engines” that handle problems ranging from linear programming and nonlinear optimization, to arbitrary models and stochastic optimization. Also part of the V2017-R2 release are new, higher performance Solver Engines

Educating and Empowering the Business Analyst

September 2017 | Solver International

Industry

NEWS

The metalog distributions allow empirical data to “speak for itself” in simulation models.

for optimization based on latest releases of the Gurobi Solver, FICO’s Xpress Solver, and Artelys Knitro Solver, as well as Frontline’s greatly-enhanced, highperformance Evolutionary Solver, which is focused on challenging non-smooth, non-convex models with integer variables. Free trials of the optional large-scale Solver Engines are “built in” and available to users at any time. NEW SIMULATION/ RISK ANALYSIS FUNCTIONALITY The V2017-R2 release supports a new family of probability distributions, called the metalog distributions, even more general than the Pearson distributions – members of this family can be chosen based directly on historical data (even just a few observations), without a distribution fitting process. The metalog distributions allow empirical data to “speak for itself ” in simulation models. Frontline’s V2016-R2 release introduced Gaussian, Student, and Archimedean copulas – popular in quantitative finance – to define correlation among uncertain variables with dissimilar distributions – complementing existing support for rank-order correlation. New in the V2017-R2 release is the ability

Solver International | September 2017

to fit copula parameters to historical data – a complement to automatic distribution fitting sometimes called “correlation fitting.” The new release builds on Frontline’s V2017 release earlier this year, which delivered a wide range of enhancements for data mining and predictive analytics including the ability to create ensembles of any mix of six different classification methods and four different prediction methods. Direct use of categorical variables in many predictive algorithms that, in other analytics software, can use only continuous variables is supported. It also has the ability to save and apply to new data an array of data transformation and unsupervised learning tools, such as PCA and clustering Enhanced neural networks with any number of layers, and better scaling, training, and activation function options and support for industry-standard PMML (Predictive Model Markup Language), plus PMML extensions for text mining are incorporated. NEW RELEASE AVAILABLE FOR DOWNLOAD The new V2017-R2 release is available now at Frontline’s Website, Solver. com, and at its cloud version portal AnalyticSolver.com. In

Solver-International.com

this release, Frontline is making available to all users Analytic Solver Basic, a new version of its Analytic Solver Platform for Education product, formerly offered only for university MBA programs, and used by hundreds of thousands of students. Analytic Solver Basic includes every feature of Frontline’s analytics software, but handles only limited-size models and datasets, suitable for learning. (All 100+ example models and datasets included with the software can be used.) A software license is just $250 per year and includes both the desktop version and access to AnalyticSolver.com. Paired with Solver Academy, Frontline’s new Open edXbased online learning platform, Analytic Solver Basic offers a way for business analysts to gain “advanced analytics ninja” skills. Frontline Systems Inc. (www.solver.com) is the leader in analytics for spreadsheets and the Web, helping managers gain insights and make better decisions for an uncertain future. Its products integrate forecasting and data mining for predictive analytics, Monte Carlo simulation and risk analysis, and conventional and stochastic optimization for prescriptive analytics. Founded in 1987, Frontline is based in Incline Village, Nevada (775-831-0300). Si

Excel® and Power BI® are registered trademarks of Microsoft Corp. Knitro® is a registered trademark of

Artelys/Ziena Optimization. Analytic Solver and RASON are registered trademarks of Frontline Systems, Inc.

Stand Out

from the overcrowded field of Data Scientists. Attend the most comprehensive and concentrated live certificate training available for analytics at the enterprise level. Lead your organization to achieve measurable returns in data science with predictive analytics.

…and watch your career take off! Contact a Training Advisor Now

Educating and Empowering the Business Analyst

+1 (281) 667-4200 Opt 3 the-modeling-agency.com/si

September 2017 | Solver International

TUTORIAL

Risk Analysis:

A BREATH OF BETTER DECISIONS AND

“Uncertainty is the only certainty there is, and knowing how to live with insecurity is the only security.” - John Allen Paulos

BY TIANYANG WANG, FRANCISCO ZAGMUTT, AND HUYBERT GROENENDAAL

Solver-International | September 2017

Solver-International.com

Photo Courtesy of 123rf.com | © basketman23

SUCCESS

Educating and Empowering the Business Analyst

September 2017 | Solver-International

TUTORIAL: RISK ANALYSIS Understanding and managing risks are crucial parts of every business. Conceptually, running a business is like navigating a ship in the ocean: There will always be a variety of risks driven by wind, wave, tide, and storm—or even by unpredictable icebergs that can sink the unsinkable Titanic. As risk is such an integrated part of the journey, there is an inevitable need to analyze, understand, and manage it. Unfortunately, most organizations (and people) spend a lot more time on planning for what they believe is the “most likely future,” than on understanding the uncertainties around their decisions, forecasts, and budgets, and on preparing to confidently navigate or mitigate future risks. Risk analysis by using Monte Carlo simulation (hereafter also referred to as simulation modeling) is a very beneficial tool to help organizations better understand future risks and scenarios, and make informed decisions. Based on the “Monte Carlo tutorial” in the previous issue,1 this article will continue the discussion on the importance of risk management and the perils of ignoring it. We will first make the case for how risk analysis and Monte Carlo simulations can help with making better business (and personal) decisions using a simple model; we will explain how simulation modeling works with a more complex/

Figure 1. Base Budget of Operating Profits

Solver International | September 2017

complete case-study; and finally, we will discuss a range of different applications, as well as several factors that are critical to benefitting from the use of simulation modeling to improve decision making. ANALYZING RISK: START THE DIALOGUE WITH A SIMPLE MODEL The basic idea of Monte Carlo simulation is to (quantitatively) play out many what-if scenarios and to statistically examine how a decision or situation will perform under all possible future (simulated) scenarios. While it is typically too costly or sometimes even impossible to perform an experiment in the “real world,” we can easily simulate it in a Monte Carlo model. There are many professional software applications (such as spreadsheet-based Analytic Solver Simulation, an Excel add-in) that are intended to help develop and use a Monte Carlo model easier and more user friendly. To demonstrate the mechanics of preparing and using a Monte Carlo simulation model, we start with a simple example. Suppose a global

Solver-International.com

TUTORIAL pharmaceutical company is considering its 5-year planning budget, and its base budget of operating profits (OP) is displayed in Figure 1. During its budgeting process, the company carefully sets its â&#x20AC;&#x2DC;base budgetâ&#x20AC;&#x2122; at a level that management thought was reasonable and realistic. However, a critical challenge of setting a base budget is that the companyâ&#x20AC;&#x2122;s budget contains many risks: uncertain commercial success of products, key drug R&D and approval risks, regulatory and legal uncertainties, etc. And even though management knows about all of these individual risks, it is very difficult to determine the aggregate impacts of so many different risks on the budget. Fortunately, developing a wellthought-out risk-based simulation model can help. To build the simulation model, the scope of relevant risks must first be understood. While this step is not covered in this article, its importance should not be underestimated. In fact, a lot of value can often already be obtained from brainstorming and discussing the potential risks with diverse stakeholders (e.g., R&D, marketing, legal, regulator, and/ or finance departments). In the second step, a simulation model can be developed to incorporate the identified risks. While some of the identified inputs may be available from existing data (e.g., past sales and costs), others may require estimates of future revenues and risks that have not yet occurred. If available and relevant, historical data can be used to determine the appropriate distributions for uncertain variables.

When historical data are not available, or there is a reason to believe that the future behavior of a variable will be significantly different from the past, then expert opinion from different business units can be used to estimate the probabilities and potential impacts of future risks. For example, we can identify probability distribution of the growth in annual sales by fitting the historical annual sales-growth data to a Normal distribution (Figure 2). In addition, Figure 3 shows the potential future peak sales (in thousands of

Figure 2. Annual Sales Growth

Figure 3. Future Peak Sale

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: RISK ANALYSIS units) of a new product estimated as a PERT distribution, which is a probability distribution that is often used to model expert opinion. The example distribution shows that the company forecasts sales to be between 10,000 and 25,000, with the most likely estimated number of units to be at 15,000. It should also be noted that, wherever appropriate, relationships (often measured by linear or rank order correlations) should be considered among the different variables within a model. For instance, there typically is a strong relationship between sales and costs. Such relationships can be either estimated from the historical data, or modeled based on expert opinions, and then be incorporated in the simulation model. Including relationships within a simulation model is important, since not considering relationships in a model often results in a significant misestimation of risks. For instance, stock price movements are often correlated, and the correlations can increase significantly in high volatility periods, as we have observed during the stock market correction of 2008. Mistakenly ignoring such correlations will underestimate the risks of investments in the stock market. The next step in using a simulation model involves recalculating (also called simulating) the model thousands of times or more to generate simulated scenarios for model output (e.g., next yearâ&#x20AC;&#x2122;s budget, Net Present Value (NPV), total project costs, etc.). All professional Excel add-ins such as Analytic Solver Simulation provide a user friendly interface for generating

Solver International | September 2017

random samples and graphical and statistical summaries of the simulated data, making it relatively convenient to conduct simulation in spreadsheets. The thousands of simulation scenarios are one of the main outputs of a simulation model, and provide us with an estimate of the possible future scenarios given the different risks included in the analysis. However, instead of showing management the thousands of possible future scenarios, we can use these simulated scenarios in various cases: to construct a frequency distribution of the performance measure; to compute risk measures such as expected value, percentiles and confidence intervals; to estimate the probability of the performance measure to be greater (or less) than the targeted/ expected performance; and to develop potential business scenarios to evaluate. The risky budget, for example, allows senior management to gain insight into questions such as (1) how realistic is their current budget, (2) what is their confidence for meeting or exceeding the budget, and (3) and how much the budget should be set at given all the relevant uncertainties? For the global pharmaceutical company, Figure 4 is a histogram that displays all 10,000 simulations of the 2018 OP considering all the risks and opportunities identified. As the Figure shows, based on the various risks included in the analysis, the 2018 OP could vary from less than $10M to more than $17M. In fact, while the expected (i.e., mean) OP profits are $13.5M, the company would have a 90 percent confidence that the OP would be between $11M and $16M. Figure 4 also shows that most of the

Solver-International.com

TUTORIAL simulation results for the 2018 OP are lower than the “base budget” of $15M for 2018. In fact, Figure 5, the cumulative distribution of the same 10,000 iterations, clearly shows that the company would have less than a 20 percent confidence that it would meet or exceed the $15M base budget that it originally thought was “reasonable.” In this example, the global pharmaceutical company’s focus is a five-year budget, thus we would develop the simulation model to consider risks affecting the OP from 2018 through 2022. As a result, Figure 6 summarizes for each of the five years. The results from the risk adjusted budget could be eye opening for the managers as the probability of achieving the company’s base five-year budget is often (much) less than 50 percent, which suggests that the budget is a stretch goal and may be unrealistic. Based on these results, management may decide to adjust the budget. In addition, the management team now would have a better understanding of how much risk there is around each of the years’ OP. MONITORING RISK: USING SENSITIVITY ANALYSIS Of course, the next natural question is what uncertainties have the greatest possible impact on the budget? And what risks, if any, could be mitigated to reduce their negative impacts on the budget? If we can identify the most influential risk drivers that are driving the overall risks in, for example, next year’s OP, the company would be able to potentially work on reducing or mitigating them (e.g., spend more resources on a product to expedite the launch, or hedge the foreign exchange

Figure 4. Histogram of Simulations

Figure 5. Cumulative Distribution of Simulations

Figure 6. Risk Adjusted Budget vs. Base Budget

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: RISK ANALYSIS risk). Sensitivity analysis addresses such questions by examining how sensitive the output results are to various individual risks. One of the most frequently used sensitivity analyses in simulation modeling is a Tornado chart that visually displays the importance of each of the individual risks (e.g., product risks, regulatory risks, legal risks) to the overall risk in a company’s key metrics, such as its next year’s OP. For instance, based on the previous example, the Tornado chart2 in Figure 7 shows two main results explaining the relative importance of each risk on the 2018 OP of the company worldwide: (1) which risks are the greatest drivers of the overall 2018 OP and (2) how much effect each of the risk drivers has on 2018 OP. The Figure clearly shows that the main four risk drivers are “regulatory risk #1,” “commercial risk #8,” “legal risk #7,” and “R&D risk #6.” In addition, it shows that “regulatory risk #1” alone could cause the 2018 OP to swing from $10.64 M to $15.25M. Having a better understanding of and potentially mitigating this regulatory risk could therefore have a considerable effect on the 2018 OP. The results of the analysis provide management and

Figure 7. Tornado Chart

Solver International | September 2017

the board of directors with a better understanding of what the most important risk drivers are, and can direct management’s attention to the risks that deserve most of their focus. In summary, simulation and sensitivity analysis provide a business with valuable insights into the company’s long-term risks and opportunities, and serve as a powerful tool in identifying and focusing attention on the risks and opportunities that affect budgets most significantly. MANAGING RISK: WEIGH YOUR OPTIONS After discussing the basics, we now introduce a second example that touches more advanced topics involving options and decisions. Suppose a multinational company with specific technologies and expertise (e.g., to sell products B2B that require upfront R&D) has an ongoing challenge to find the right balance of risk and reward in agreements with business partners. Agreements can be kept as simple as a supply agreement where the supplier is paid for cost plus a markup or made more complex where a supplier takes on a greater share of the cost and risk, and in return, shares the profit margin of the final product. Simulation can be a powerful decision-supporting tool to understand the risk-reward picture of alternative partnership agreements: product/ business development, private equity/ venture capital, joint ventures, etc. Even though the way partnership agreements are structured can greatly influence a business’ risk/reward profile, often businesses only quantify the “most

Solver-International.com

TUTORIAL likely” rewards (e.g., NPV, IRR, sales), and not the risks. In comparison, using Monte Carlo simulation, we can develop risk assessment tools and simulation models to analyze both the risks and potential rewards of pursuing a variety of different partnership agreements and business models. Such quantitative tools would transparently and consistently quantify the risk/ reward trade-off, and provide clarity and insight into alternative partnership structures and business models. This will greatly assist the company in deciding which opportunities to pursue, and what partnership agreements to negotiate. As we discussed previously, first we must gain a good understanding of the main risks and uncertainties that impact the financial risks and returns of the opportunity in question. In this example, risks can be wide-ranging but often include R&D uncertainties, market size, competition and market share, legal landscape, costs, and timelines. The next step is to build a risk-based financial profit and loss (P&L) and a cash flow model that incorporates all identified risks and uncertainties, with all relationships between the risks, and importantly, does not over-complicate the models. Once the financial returns and risks are incorporated in the P&L model, the model then “overlays” the various relevant partnership agreement structures or business models, resulting in a full simulation risk analysis of each alternative “business partnership structure.” A great strength of this approach is that the results include risk and decision

Figure 8. Different Partnership Structure

Figure 9. Risked NPV under Each Partnership

insights on the individual strategic opportunity as well as the various agreement structures. Figure 8 shows an example of some of the results of the analysis for this company under three different agreement structures. In Figure 9, the dots represent the expected “risked NPV” and the error bars show the amount of risk under each structure. The appeal of each option depends on how risk averse you and your potential partner are, what the individual risks are and how they are shared with your partner, and how much risk the rest of your product portfolio has. In other words, to make an informed decision on the strategy of one product, it is important to understand the risks in the rest of the organization’s portfolio. Similarly, while not shown here since this analysis is based on a simulation model, a Tornado chart can show the

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: RISK ANALYSIS main risk drivers and help the company in focusing on how to best potentially understand and mitigate risks in the partnership agreement. MANY APPLICATIONS The authors have successfully applied risk analysis methods in hundreds of projects in fields ranging from pharmaceuticals, oil and gas, finance, manufacturing, and mining to food and beverages, health and food safety. We used and gained our analytical expertise in helping private and public institutions worldwide to make decisions in the presence of uncertainty. The following are several additional applications of Monte Carlo simulation to improve decision making. A large international firm wishes to evaluate the risks and rewards of a potential large business-development deal. The complicating factor is that the proposed M&A contract allows either of the two parties involved to change the business terms, after the deal is signed (i.e., “flip” the terms). A Monte Carlo simulation model can capture the operational, commercial, and financial risks of the deal, as well as the “optionality” within the potential M&A deal-terms, and allows the client to take a more favorable negotiation strategy that can result in a successful licensing deal. A venture capital firm is in the process of structuring a new fund. The firm needs to determine the optimal distribution structure of the different future cash flows to various stakeholders (entrepreneurs, investors, and general partners), taking risks into account. A

Solver International | September 2017

comprehensive Monte Carlo model that simulates the investment performance, and relates various cash flow streams, will help the managing director better understand the risks to each of the stakeholders involved under the different fund structures and scenarios, support the decisions on how to structure the fund, and give investors a quantitative view of risk and return. A gas transmission and storage company manages the large cost of several projects’ costs and schedules. The managers wonder if there is a better way to understand and manage risks. Monte Carlo simulation would provide the company with better insight about the project and its risks. USING SIMULATION SUCCESSFULLY While Monte Carlo simulation can have great advantages in making better informed decisions, there are still many organizations that have not utilized it to its advantage. In our experience with many diverse companies and organizations worldwide, we have found the following three factors to be critical for the successful and continuous use of simulation modeling: • Support from senior management: Senior management’s interest in, and support for the use of simulation modeling to improve decision making is critically important. While there are multiple ways of obtaining and maintaining such support, an effective approach we’ve frequently employed is with one or a couple of important (visible) example projects in which

Solver-International.com

TUTORIAL the benefits of simulation model can be effectively shown. • Making simulation modeling an integral part of the decision making process: To benefit from the advantages of Monte Carlo simulation, its usage should be made part of the regular decisionmaking process (as stated in #1, its use should be demanded by management). This means it should, of course, not be a substitute for business professionals learning with and from real business contexts but instead as a way of enhancing your decision-making. For decisions that are made regularly (e.g., investment decisions), the use of templates can often facilitate and speed this up. One of the companies we’ve collaborated with has made a Monte Carlo analysis mandatory for the review/approval of every capex project over $5M. • Developing and gaining expertise: To achieve credible simulation results, the model must closely resemble the reality, while also keeping the model conceptually tractable. The decision for the right level of detail, a suitable model structure, and valid inputs all require expert experience with the implementation of simulation models. This means that obtaining the right expertise and skills to develop and use simulation model is critically important.

SUMMARY We’ve discussed the importance of risk analysis and risk management using Monte Carlo simulation. In business, risk is everywhere. “Man is a deterministic device thrown into a probabilistic Universe. In this match, surprises are expected.”3 Simulation modeling can help organizations better navigate the business oceans with unexpected icebergs. In the end, what separates successful businesses from those that fail is the attention to risk and the capability to manage it. Si RESOURCES ModelAssist: A free and comprehensive quantitative risk analysis training and reference software. http://www. epixanalytics.com/ModelAssist.html. Practical Spreadsheet Risk Modeling for Management, Lehman, D, Groenendaal, H and G. Nolder. Chapman and Hall/CRC, September 1, 2011. Hardback textbook, 284 pages. FOOTNOTES 1 See Solver International’s premier issue, July 2017, which has a great introductory guide on modeling risks with Monte Carlo simulation. 2 There are various versions of Tornado charts available, but all are focused on the question of what the main risk drivers are towards a certain objective such as a 2018 OP, total costs of a project, NPV, etc. 3 Quote by Amos Tversky, in “The Undoing Project” by Michael Lewis

Organizations that do get the right support, processes and skills in place can expect a true competitive advantage though in making more informed decisions.

Educating and Empowering the Business Analyst

September 2017 | Solver International

CASE STUDY: UNIVERSITY OF OKLAHOMA

RECRUITING THROUGH

ANALYTICS Recruiting students to a university can be a challenge when the data isnâ&#x20AC;&#x2122;t handled properly. Meeting the challenge can also be a challenge.

Photo courtesy of Jana Carson

LISA MOORE

Data Scientist Institutional Research and Reporting University of Oklahoma

The Institutional Research and Reporting (IRR) office of the University of Oklahoma, Norman, Okla., is charged with official reporting for the university and aiding in helping the university make a variety of decisions about operations and giving guidance and providing research. As part of the administration side of the university, they react to issues that affect the total school. 22

Solver International | September 2017

Solver-International.com

CASE STUDY One of the problems that came to the IRR was directly from the Office of the President, David L. Boren. As Lisa Moore, data scientist at IRR, explains, “The director of recruitment came to me because the president told him he needed to increase both the quality and the number of freshmen we had incoming. The director was concerned because there wasn’t a budget to increase the number of recruiters or to hand out many new scholarships to get more students to enroll. We needed to figure out another way to do this.” Moore went to a conference where she talked to other institutional researchers. She recalls, “They said, ‘SAS has this nifty new tool called Enterprise Miner. You need to go to their booth and check it out.’ I stopped by and hung out with the SAS people so much that they should have issued a restraining order against me.” Being a pest can pay off. Moore says, “I got to play around with the program and saw how quickly it analyzed and mined data that I had been trying to do by hand. It was taking me weeks just to do basic regressions.” Moore’s boss happened to be at the same conference and they sat down with the SAS staff. The result was a single desktop license to start working on the recruiting project. “And then we had the fun of trying to figure out how to collect the data and exactly how we would do this project,” Moore admits. Moore had looked at other products before getting the SAS license but felt they were not as robust. She notes, “With one product you’re limited to a handful

The trend in academic preparedness for applicants to the University of Oklahoma—number of students with a high school GPA of 4.0 and the number of students ranked first in the graduating high school class. There was a large jump from Fall 2013 to Fall 2014. The university had to do many things to attain that goal, some of which were unsustainable.

of different types of regressions. You can’t do the neural networks through decision trees, and it’s not quite as statistically user friendly as I wanted. With SAS, you’re still able to manipulate and set things, especially targets, the way you would want to, whereas other systems don’t allow that sort of manipulation.” In their evaluation process, Moore ran tests of the two programs using the data she had obtained earlier. “I ran them against what I had done by hand, and Enterprise Miner won hands down, every time. We also liked Visual Analytics to simplify our reporting,

Educating and Empowering the Business Analyst

September 2017 | Solver International

CASE STUDY: UNIVERSITY OF OKLAHOMA because we were doing what my boss likes to call ‘artisanal Excel reports.’ They were carefully handcrafted and we were hand entering a lot of data. Using the visual reporting, we could create interactive graphs that made it a lot easier for people to see the information. In the end, we purchased Visual Analytics, too, and have started implementing it.” Implementation of a singleuser desktop application should go easily and Moore acknowledges that it did, “Everything installed very easily, and then we set up the Oracle database connections. One of our programmers helped me do that, and that was all we had to do.” The project was started in 2015. “There was a large jump in enrollment from Fall 2013 to Fall 2014, but we had to do a lot of things to attain that goal, many of which were unsustainable. The enrollment goal for 2015 was 4,200 incoming freshmen, and we nailed it right on the head,” Moore says. “We weren’t able to use predictive analysis for fall 2016 and we didn’t quite meet our goal, but in 2017 we are well on our way to surpassing the enrollment goal.” What type of data are being using? “There are a lot of different pieces that we’re combining,” Moore explains. “There are pieces from the applications—that are supplied by the students—and we also get some information from the recruiters. When they contact students, they enter feedback that they’ve received from the students into their recruiting application. We also use data that we purchase from ACT and we add

Solver International | September 2017

data that comes from attendance at recruiting events, especially campus tours.” While Moore is the one doing the predictive analytics there’s a programmer who has helped in the collection of the data. And there is coordination with recruitment and admissions staffs to make certain all the data is there and available. As is often the case, both in business and academia, success leads to increased notice and increased interest by others. “We’re definitely expanding,” Moore admits. “I’ve coordinated with other groups and formed a very close relationship with them. Recently we used information to help set goals for each recruitment region, so they know how many applications and admissions, by area, that are needed for 2018 to hit our overall enrollment target. We’ve also been working to see what they can do to increase the number of applications. We look at prospective student information and see what makes the student ultimately apply to the university. We’ve been implementing a lot of changes based on the findings.” Being in a learning environment doesn’t mean you immediately know everything. Moore has found the SAS world is filled with helpers. “If I have a lot of questions, I can search the SAS online communities to see if somebody else has already dealt with and answered my problem. If not, I can post there and, within 24 hours, I’ll have eight different ways of solving it.” Si Lisa Moore is a data scientist at the Institutional Research and Reporting department of the University of Oklahoma, Norman, Okla.

Solver-International.com

us Busines nuo s Im i t n o C PEOP LE

Management is Commited

BeneĮts are tracked and managed

TEC H

Business Objectives

ic Busine s s Al nam ig Dy OGY OL N

Users / Analytsts are skilled and moƟvated

ent em v o pr 1

Aligned AnalyƟcs ApplicaƟon Porƞolio

E R A TI O N

AnalyƟcs are embedded in processes

Reliable IT/ Cloud Infrast.

Reliable Data

ent nm

Jobs and Org. Structure Adapted

People are sharing insight

Mgmt. QuesƟons / Decisions are deĮned

TUTORIAL

OPTIMIZATION

FOR BETTER DECISIONS

Optimization requires that we define, in quantitative terms, a model that specifies all the ways, times or places our resources may be allocated, and all the significant constraints on resources and uses that must be met. Here is the way to do that. BY DAN FYLSTRA, CEO, Frontline Systems

Solver International | September 2017

Solver-International.com

challenge is multiplied many times over: How to best schedule every hour for a staff of 30 people in a call center? How to load packages on a fleet of 100 trucks, and which routes they should drive to make deliveries in the least time? How to assign crews and aircraft to 1,000 airline flights, as they move across the country throughout a day?

Educating and Empowering the Business Analyst

Photo Courtesy of 123rf.com | © Denis Putilov

Every day, in business, government, and even our personal lives, we make decisions about how to best use the resources – such as time and money – available to us. It is challenging enough for us to decide which items to buy with our available funds, or which of several priorities we should tackle this morning. For even medium-size organizations, this

September 2017 | Solver International

TUTORIAL: OPTIMIZATION These decisions – how to allocate (usually limited) resources to different uses, when there are so many options, with so many interrelationships – are prime candidates for optimization. At leading firms, all the foregoing decisions are routinely made with the aid of optimization. To use optimization, we need to define, in quantitative terms, a model that specifies all the ways, times or places our resources may be allocated, and all the significant constraints on resources and uses that must be met. Then a solver searches for and finds the best resource allocation decisions. DECISION VARIABLES, OBJECTIVE AND CONSTRAINTS Our quantified decision variables are the amount of resources allocated to each individual use – for example, the number of call center employees working on each shift, or the number of packages of a given size loaded onto each truck. To determine what “best” means, we must define a quantity called the objective that we can calculate from the values of decision variables— for example, costs that we’d like to minimize or profits to be maximized. To complete the model, we must define each constraint, or limit on the ways resources may be allocated, that reflects the real-world situation. We usually have both simple constraint limits such as “up to 100 trucks,” and constraints calculated from the decision variables, such as “our beginning inventory, plus units received minus units shipped, must equal our ending inventory.” Let’s consider a very simple example of a call center employee scheduling

Solver International | September 2017

problem, shown in Figure 1 (this example Excel model is included with Frontline’s Analytic Solver software). Our problem is to schedule enough employees to work each day of the week (decision variables) to handle our predicted call volume (a constraint), while minimizing total payroll cost (our objective). Our personnel policy, that employees should work five consecutive days and have two consecutive days off, determines the possible ways that resources can be used: There are seven possible weekly schedules, each one starting on a different day of the week. These are labeled A, B, C through G in the Excel model. For example, employees on Schedule A have Sunday and Monday off, then they work Tuesday through Saturday. Our decision variables – the number of employees working on each schedule – are in cells D15:D21; they are summed in cell D22. In this simple model, all employees are paid at the same rate, $40 per day at cell N15. Our objective, payroll costs to be minimized, is just =D22*N15*5 (5 working days per week). We must meet a constraint that the number of employees working each day of the week is greater than or equal to the “Minimum Required Per Day” figures in row 25. We assume here that we actually know these numbers – in many real-world call centers, the call volumes per day are uncertain, and we have only a range or probability distribution for each one. In this tutorial, we’re covering only conventional or deterministic optimization; in a future tutorial, we’ll describe stochastic optimization,

Solver-International.com

TUTORIAL where we must allocate resources under conditions of uncertainty in the objective and/or constraints (something that Frontline’s software can handle very well). The 1s and 0s in the middle of the worksheet help us calculate the number of employees we’ll have in the call center on each day of the week. For example, on Sunday we’ll have the employees on Schedules B through F, but those on Schedules A and G will have the day off. So the number of employees working Sunday is just =SUMPRODUCT(D15:D21,F15:F21) – and similarly for the other days of the week. Our SUMPRODUCT formulas are in row 24, and we want each value

to be greater than or equal to the corresponding “minimum required” number in row 25. We can express this as F24:L24 >= F25:L25, or using Excel defined names, as “Employees per day >= Required per day.” You can see the Solver model taking shape in the righthand Task Pane in Figure 1. In this model – as in many others – we must be careful to define all the limits on resources, including “nonnegativity.” We cannot have a negative number of employees on any schedule. This may be obvious to us, but Solver does allow negative values for decision variables unless we say otherwise – so we include a constraint D15:D21 >= 0 (or with defined names, “Employees per schedule >= 0”).

Figure 1: Optimization at work: A Simple Call Center Scheduling Example

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: OPTIMIZATION There’s one more constraint we haven’t yet discussed: Solver allows any whole number or fractional value for a decision variable, but we can’t actually assign one-half or two-thirds of an employee to a schedule. Indeed, the optimal solution without this integer constraint assigns fractional values to four weekly schedules, such as 2.67 employees for Schedule A, and 6.67 employees for Schedule C – minimizing payroll cost to $4,933. We complete the Solver model by adding a constraint “Employees per schedule = integer.” (In some optimization software, this is treated as a property of the decision variables, but since it limits the possible solutions, Solver treats these integer requirements as constraints.) We can now solve the model by clicking the Optimize button on the Ribbon, or the green arrow on

the Task Pane. The optimal solution ($5,000 payroll cost) is shown in Figure 2: Note that the integer constraint “cost us” something – indeed, additional constraints always yield a “same or worse” objective. Even more important than the objective value are the decisions that will realize this outcome. These are amounts of resources to be allocated to each use – in this case, the number of employees to be assigned to each of the seven possibly weekly schedules: 2, 5, 7, 4, 6 and 1 for Schedules A, B, C, D, E and F. Because our call volumes on weekends are so high, we assign no employees to Schedule G. OPTIMIZATION: EXPLOITING STRUCTURE How did Solver find this solution, and how do we know that it is the best

Figure 2: Optimal Solution of the Call Center Scheduling Example

Solver International | September 2017

Solver-International.com

TUTORIAL possible solution? You can play “whatif ” with this model, trying different values in cells D15 through D21, searching for a good combination of values. (You’ll find that there is no better combination of values that satisfies all the constraints.) But to really answer this question, we must probe more deeply into “how Solver works,” and talk about some key ideas: optimality conditions, linearity and convexity. Some math and geometry follows(!) – but if you make it all the way through this tutorial, you’ll be rewarded with a much deeper understanding of Solver and optimization. Solver does try different values for the decision variables, searching for the best solution – indeed all optimization algorithms work this way – but the search is much more sophisticated than randomly chosen “what-if ” trials. By computing partial derivatives and testing for satisfaction of the KKT (Karush-Kuhn-Tucker) conditions, Solver can determine that it has found the “top of a peak” or “bottom of a valley” – there are no better solutions “nearby” (a locally optimal solution) – and if the model is convex (discussed later), there are no better solutions anywhere (a globally optimal solution). This yields the message “Solver found a solution. All constraints and optimality conditions are satisfied.” In our simple Call Center example, Solver had to search a seven-dimensional “space” of possible values for the decision variables (one dimension for each variable), for better objective values, while ensuring that the “Minimum

Required” and the non-negativity and integer constraints were satisfied. In a problem with 200 decision variables, this becomes a 200-dimensional search! And Frontline’s enhanced Solvers are routinely used to optimize models with thousands to millions of decision variables. How? Solver can do this by exploiting the (algebraic) structure of the model. In this case, the objective (recall it is =D22*N15*5, which equals SUM(D15:D21)*40*5) is a linear function of the decision variables. Each of the seven constraints is of the form =SUMPRODUCT(D15:D21, constants), also a linear function of the decision variables. Without the integer constraint, this is a linear programming problem, the easiest type of optimization problem to solve, and one that always yields a globally optimal solution. When we add the integer constraint, the model becomes a linear mixed-integer programming problem, or LP/MIP as shown in the “Model Diagnosis” area at the bottom of the Task Pane. These problems are significantly harder to solve, but there are sophisticated search algorithms available for LP/MIP problems, and Frontline’s Solvers use them. LINEARITY AND CONVEXITY: THE KEYS TO SOLVABILITY A linear function, such as SUM or SUMPRODUCT, or any chain of formulas where decision variables are only multiplied by constants and the result added or subtracted, can be plotted as a straight line. In full Analytic Solver software, you can

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: OPTIMIZATION create such a plot (“slicing through” N-dimensional space) with two mouse clicks, Decisions – Plot, as shown for the objective in Figure 3. The constraints in our Call Center model also plot as straight lines (in seven-dimensional space). Since they must all be satisfied at the same time, Solver can limit its search to points (i.e., combinations of values for the decision variables) in the intersection of these linear constraints – forming the feasible region. In many

Figure 3: Plot of the linear objective function, Total Payroll Cost.

Figure 4: Graphical depiction of a two-variable linear programming problem.

Solver International | September 2017

dimensions, the intersection of linear constraints is a geometric form called a polytope, but in two dimensions this would be a polygon, as shown (for a different two-variable maximization problem) in Figure 4. The objective – shown in red – is also a straight line that slides up and to the right as it is maximized; the optimal solution is at point D. But the key point is that the polygon or N-dimensional polytope – the feasible region – can be efficiently searched. From the viewpoint of optimization as a search process, the straight lines in Figure 4 are less important than the overall shape of the feasible region, which is convex. (An intersection of linear constraints is always convex.) A circle, formed by a formula such as x^2 + y^2, is also convex, and is also easy to search; but a non-convex region becomes exponentially harder to search as its dimensionality increases. Figure 5 shows simple examples of convex and non-convex polygons, in two dimensions. Notice that in the non-convex region, the straight line (called a chord) goes into and out of the feasible region multiple times. A non-convex region has “nooks and crannies,” which take more and more time to search as the dimensionality of the region increases. Imagine, for example, searching a 200-dimensional version of this figure. When an optimization problem’s objective and constraints are both convex – as is always true in a linear programming problem – the problem will have one optimal solution, which is globally optimal. But a non-convex problem may have many locally optimal solutions.

Solver-International.com

TUTORIAL THE CONVEXITY KILLERS In Excel models, it is common to use IF functions to make “eitheror” choices, and to use CHOOSE or LOOKUP functions to select among multiple values. These functions are very useful, and it’s fine to use them in Solver models, as long as their arguments do not depend on the decision variables. But if a model has IF, CHOOSE, or LOOKUP functions that do depend on the decision variables, this will quickly make the model both non-convex and non-smooth. Figure 6 shows the type of plot we get for a formula such as =IF(D15<2,F24,3*F24) in our Call Center model. Compare the “kinks” in this plot to the non-convex region in Figure 5. To make matters worse, the “kinks” make the function non-smooth, which means that Solver cannot reliably compute derivatives at those points – another way of saying that Solver cannot reliably follow the “rate of change” in the function. Again, as the number of dimensions (decision variables) and the number of constraints increases, the time needed to search for an optimal solution increases exponentially. What can you do, if you need to make “either-or” choices, or select among multiple values in your model? There is a better way: You can express the same conditions using integer variables and linear constraints. Frontline’s Solver User Guide explains how to do this, and our full Analytic Solver software, as part of its model diagnosis, can detect IF, CHOOSE, and LOOKUP functions and automatically replace them with

Figure 5: Convex and non-convex regions.

Figure 6: Plot of IF Function that Depends on Decision Variables.

equivalent integer variables and linear constraints, up to a certain level of complexity. It is worth noting that introducing integer variables into an optimization model actually makes the model non-convex. In Figure 5, because only integer-valued points (which would appear as dots) are feasible, the chords will go “in and out of the feasible region” multiple times. But if integer variables are the only source of non-convexity in the model, then powerful algorithms for handling these integer variables can be applied to greatly cut down on search time. The Gurobi Solver and XPRESS

Educating and Empowering the Business Analyst

September 2017 | Solver International

TUTORIAL: OPTIMIZATION

points are in dark red and dark blue. Solver in Excel includes basic facilities for global optimization, using either the “Multistart option” for the GRG Nonlinear Solver, or the Evolutionary Solver – and Frontline’s enhanced Solver products offer more powerful methods Figure 7: A global optimization problem, with just two variables. for global optimization, such as the Interval Solver, available as optional plug-in Global Solver, OptQuest Solver, and Solver Engines for any of Frontline’s Frontline’s hybrid Evolutionary Solver Solver products, are highly effective – an engine that combines classical at solving even large linear mixed(linear and nonlinear optimization) integer and quadratic mixed-integer methods with genetic algorithm, problems. scatter search, local search, and heuristic methods. GLOBAL OPTIMIZATION If you’ve followed this tutorial If your model simply cannot be to its conclusion, you now know a expressed as a linear programming lot more than most people about or linear mixed-integer problem, you optimization! While we’ve illustrated can still use optimization. In most these concepts with Excel models and cases, this means you’ll have to accept simple plots, the ideas of linearity an approximate globally optimal and convexity are fundamental, and solution, a locally optimal solution, applicable to any kind of optimization or (for a non-convex, non-smooth problem, solution algorithm, or model) just a “good” solution – better software. Hopefully, you also realize than what you were doing before (this how optimization problems can can still yield a great business payoff). become very difficult to solve, but Figure 7 is a plot of a smooth but how powerful software is available to non-convex objective function. You help you find good solutions, even for can see that it has multiple “peaks” the most challenging problems. Si and “valleys” – the KKT conditions Dan Fylstra is President and CEO of would be satisfied at each peak Frontline Systems, Incline Village, Nev. (when maximizing) or valley (when minimizing), but the globally optimal

Solver International | September 2017

Solver-International.com

solver International

the bi-monthly digital publication covering prescriptive and predictive analytics, will notify you of coming events, news and views from industry experts, and so much more.

WWW.SOLVER- INTE RNA TION AL.C

TUTORIAL

Monte Carlo Simulation

JULY 2017

CASE STUDY

Transportation

TECHNOLOGY

Analytics in the Cloud

SUBSCRIBE TODAY to get notices by e-mail of information you can use to boost your career and improve the success of your company or academic institution.

VISIT OUR WEBSITE TO SIGN UP!

Solver-International.com

EXECUTIVE INTERVIEW

TAPAN PATEL — SAS INSTITUTE

Tapan Patel is Principal Product Marketing Manager at SAS. With more than 17 years in the enterprise software market, he currently leads global marketing efforts at SAS for Business Intelligence and Predictive Analytics topics and SAS Visual Analytics and SAS Visual Statistics products. He works closely with customers, analysts, press and media, and thought leaders.

Solver Internatio nal: What functions in a company are needed to support or develop business intelligence?

Tapan Patel: Typically, business intelligence involves a variety of functional areas. First and foremost, when it comes to establishing the objective, or defining the problem and how you’re going to measure whether you’re successful or not, you need to have the sponsorship, the support, and the guidance from the business unit that is interested in using the business intelligence technology. Secondly, traditionally, IT has been involved in creating, supporting, and maintaining

Solver International | September 2017

Solver-International.com

Educating and Empowering the Business Analyst

September 2017 | Solver International

EXECUTIVE INTERVIEW: TAPAN PATEL the business intelligence assets, content, and data which are needed to create either the reports, dashboards, models or explorations. Now, over the course of the last four or five years, the role of IT in that context has shifted towards enablement. They are there to make sure that governance is maintained, make sure that the proper data are provided for the users who are building the content. Rather than a producer, they have become more an enabler. And then, finally, there are the users or content creators. I like to say that all consumers want to become producers. All of them are creating, either building interactive reports or updating our dashboards, or creating visualizations or explorations to find important relationships. For certain a class of users, like data scientists, for example, or data mining specialists, they are building predictive, prescriptive and diagnostic models. I see three key roles that are important for any business intelligence or analytics initiative. Organizations need business executives and their management to champion and sponsor the project. The role of IT has been more in terms of ensuring governance, making sure that they have the right scalable architecture to serve the needs of a variety of different users and to grow systems in an incremental fashion. And then there are the users, who are all producing and sharing content or insights, depending on the types of problems they’re trying to solve. SI: With the development of dashboards and the simplification of

Solver International | September 2017

the user environment, it seems that people don’t even care what’s going on behind the data. They can still develop an ability to do analytics.

Patel: When organizations need to create predictive or prescriptive models, they are doing so to solve specific sets to problems (e.g., risks), finding new opportunities, or optimizing costs. Not all business analysts, and certainly not all business users, are going to have the skill sets to build and interpret the analytical models. There’s a certain class of user who is going to use analytical tools, techniques and methodologies to build and deploy those models. Given that those skill sets are scarce, more and more analytics software is becoming automated, smarter and easier to use. The goal is: “How can I reduce the time needed to build the analytical models, and quickly go through the entire analytic lifecycle? What are some of the steps that could be automated as part of the analytical life cycle?” For example, can I make use of (self-service) data preparation tools to join my tables or suggest hierarchies? Can I automate some of the feature engineering aspects of the model building? Or can I smartly identify which are the most important drivers for building my analytical model? These will make life easier and more productive for data scientists or information analysts who are hard to find, have enormous workloads, and are expensive, in building analytical models. SI: Business intelligence is a growing investment for large companies, and smaller companies are starting to look at it as well. How

Solver-International.com

EXECUTIVE INTERVIEW can a small company benefit from developing BI?

Patel: Irrespective of whether it’s a large organization, a small, or a midsized organization, our businesses, our public institutions, and our economy, everybody is influenced and affected by the data we are collecting and everybody is interested in getting value out of using BI and analytics. When it comes to small and mid-sized organizations, they have tight budgets, a shortage of IT skills and talent, and often rank low on the scale of analytics maturity. They may not have enough people to prepare, integrate and clean the data—which is absolutely critical for BI and analytics initiatives. It is also critical to identify high priority objectives, business problems or customer issues to achieve and solve with your business intelligence or analytic strategy. Next, for each business objective, identify metrics that are specific and measurable. Monitor the performance and measure the business outcomes from their BI and analytics initiative. Start small, but even with a small project, measure it, continuously measure it, to see whether it is achieving the goals and expectations from your BI and analytics program. Identify where you are and what is your current level of analytics maturity. Are those skill sets available internally within the organization? Do you need to develop those skill sets or train people, or can you outsource it or can you bring in external talent? Can you bring in help from local universities to fill in those critical gaps and achieve a level of maturity such that it will help you move from Point A to Point B?

And finally, since they don’t have big or mature IT organizations, they should look into investing in cloudbased BI or analytics programs. With cloud-based BI, more users in small and mid-sized organizations can quickly start creating content and reduce time-to-insights. IT can focus on content governance, privacy and data preparation tasks associated with the cloud rather than selecting and maintaining hardware. SI: We seem to have taken the approach that we’ve got to teach the ability to use the tools as opposed to developing the tools. That is, the MBA program isn’t an IT program. They want to have the capability to understand how to use the tools that are available, and some tools are very difficult. How does SAS provide a program approach for MBA students that will make it easy for them to learn how to create and use business intelligence?

Patel: In any MBA program, especially one focused on a career in analytics and BI, we have a few programs successfully using SAS products in their courses. Focusing on the right type of user, and providing the right tool, is key to quick adoption and success. SAS Visual Analytics gives the student an ability to do BI and analytics in a self-service and interactive manner. They can prepare data that are needed for creating interactive dashboards or reports, and identify why something has happened. They can intuitively create explorations to find outliers, or find which segments of a customer population they need to target, or they can find the relationships between

Educating and Empowering the Business Analyst

September 2017 | Solver International

EXECUTIVE INTERVIEW: TAPAN PATEL different variables that are important for future outcomes. For some students who are interested in predictive analytics but who do not want to build something from scratch, we have the capability to provide a basic set of self-service analytics built into the product. For example, if they want to do forecasting, or if they want to build word clouds to do text analytics, or if they want to do scenario analysis, these features are available to them. The software picks the analytical algorithms for you and gives you the results and explains those results to you in a quick, easy, and simple fashion. That’s an example of how universities, offering MBA programs, are using SAS Visual Analytics for their BI and analytics curriculum. SI: When you were in your MBA program at North Carolina State, did you have this type of software and capabilities available to you, or was it still pencil and paper?

Patel: That’s an interesting question that brings me back to the good days. We did have software to use! At NC State we were using JMP software, which was a data visualization and statistics software. It was a desktopbased tool and we were learning different statistics-based visualization techniques for data analysis. We were also using different statistical techniques to identify what was important, what were the key drivers, and predicting future outcomes. SI: Let’s talk about business analysts in a company, looking at the ways of developing business in-

Solver International | September 2017

telligence; where are they going? What’s going to happen in the next, say, five years so they can plan and be ready for the next change in the way BI is applied?

Patel: Based on discussions with customers, what many organizations are looking forward to is streamlining the data discovery and deployment aspects of BI and analytics. They want more efficient and effective ways to get value out of different data and analytics initiatives. Besides technology, they want to understand the role people and processes play in success from data and analytics initiatives. What we are seeing from a data standpoint is the rise of self-service data preparation. This is the concept where more and more companies are looking to put data preparation aspects in the hands of analysts and business users, rather than limiting it to just IT. Identifying how quickly and easily they can access data, prepare data, and transform it so the data is readily available for downstream BI and analytic needs. IT does not need to be sidelined. In fact, business and IT will have to work together to understand how self-service data preparation capabilities can affect each other’s roles and responsibilities. IT is definitely getting involved in the hardcore data cleansing and data management tasks, but more often, the business analysts, data analysts, or data engineer roles are getting elevated and staffed to do data preparation tasks in a self-service manner. They can access, blend and prepare data quickly to shrink the time needed for downstream BI and analytics content creation and sharing.

Solver-International.com

EXECUTIVE INTERVIEW When it comes to the data discovery part of the life cycle, how will the next generation of capability available to business users, analysts, and data scientists make them more productive and effective? The software can automatically find and visualize relevant findings, correlations or exceptions, segments or predictions. And once they automatically find it, with help from natural language generation, how can you explain the insights to the business users and the analysts to assist in making the best decisions? And, finally, when it comes to the deployment aspect, I think more and more companies are asking the questions, “We have found these insights. We have developed the best champion model. How can I quickly put it in the hands of more people such that they can take advantage of it? How can business users, the sales teams, and the frontline people make use of these insights to make better decisions? How can I integrate the modeling results into my operational systems? And once I get new data or as my business scenarios change, how can the data discovery deployment cycle be quickly turned around so I’m not missing opportunities for revenue generation or getting new customers? SI: You mentioned cloud computing as a benefit for a company that didn’t have or didn’t want to develop a large IT infrastructure. What other technologies are coming that are going to make a change in the way BI is done?

Patel: The interest and adoption of cloud computing for BI and

analytics is growing fast. It is suitable for organizations that are interested in quickly bringing results to the table when it comes to BI and analytics. It is becoming important for companies who have their data readily available in the cloud. It is also important, as I mentioned, for small and midsized companies that don’t have the fullblown IT or support staff that is needed for traditional, on-premises deployment. Even so, the infrastructure part of BI and analytics is an important aspect. Without the right infrastructure or architecture, you are not going to grow from 10 users to 100 to 1,000. Today you might be doing structured data; tomorrow you might want to combine structured and unstructured data. Today, you might only have simple workloads, involving basic reporting and dashboards, but in the near future you would like to handle complex and diverse workloads involving data discovery and analytics. Having the right infrastructure and architecture is equally important as you grow your footprint when it comes to BI and analytics. The role of IT should not be minimized. Cloud computing, for example, brings up concerns of privacy, regulation, data preparation. For example, accessing and integrating on-premises data and cloud data is a fundamental question for organizations and IT to solve as they embark on their journey to cloud computing. The cloud is suitable and relevant for many customer-facing BI and analytics applications, and hence the interest in cloud computing

Educating and Empowering the Business Analyst

September 2017 | Solver International

EXECUTIVE INTERVIEW: TAPAN PATEL is driven more by business units. Business also has the upper hand to start their new projects or move their existing one to the cloud because they have the budget and power to do so. Another trend of interest to IT is embedded BI/analytics. Once the insights are created, they need to be shared through different channels, like mobile, Microsoft Office, portals, etc. in a repeatable manner to ensure consistency and governance. When those insights are made available to the frontline staff, sales teams, or executives, especially if they’re working online, their first order of business is, “How am I going to collaborate? How am I going to interact with those insights? If I have concerns, can I ask questions right inside the Microsoft Office application or the mobile application? If I want to ask a question through a comment, can other users also look at that comment and the snapshots of the insight associated with that comment? And can I quickly interact with other staff such that all of us come together and collectively take a decision?” Another area where we are seeing interest from IT is around visualization and querying data from streaming sources (sensors, log files, machine data, social streams, weather data, etc.) especially with the growth of the Internet of Things (IoT). Suddenly we are talking about analyzing and visualizing large scale data volumes and complex data in realtime. Viewing data flows in realtime and understanding which signals are important or not within individual data streams is valuable.

Solver International | September 2017

It must help the operational systems to process anomalies and proactively take corrective actions. Some use cases might be interested in dashboards that update themselves as new data comes in and sends alerts for abnormal readings. SI: What would you recommend to people who are, let’s say, high school students or undergraduate students, looking to pick a program to go into the business intelligence and analytics area? What criteria do you feel students should look for in a university or college program?

Patel: Students who are interested in a master’s program in analytics, for example, will find different universities have varied expertise, focus areas, curriculum, resources, and depth and breadth of their faculty. However, I would also suggest students consider how the university collaborates with the local business community to bring them real-world problems to solve. Now, I understand that those opportunities will be much more common in the second year of studies, but are universities participating with diverse sets of companies that can help you understand the context of the problems they’re bringing to the table or the specialty you want your future career to focus on? Second, business intelligence and analytics tools, are a commodity nowadays. You can either take the path to learn open source software, commercial software or both. Hence finding a program that offers you the flexibility to choose is important. You

Solver-International.com

EXECUTIVE INTERVIEW also need a faculty and a program that explains how the data is structured and prepared, and how you can understand the data relationships. Data management is an important topic to look at for any analytics initiative and is more than half the battle students will fight in their analytics career. Look at whether the program gives you exposure to different data management principles. Third, the program needs to focus on teaching how to frame the objectives and the problems that need to be solved. How you can identify the problem in many different dimensions is also a valuable skill when it comes to analytics. It’s not just about analytical techniques and methods, but which ones to use to solve the problem and that requires an understanding of the problem and the objectives. An understanding of how to measure success is equally important – in terms of specific quantitative and qualitative goals to prove whether you have achieved the laid objective or not. Fourth, after you have built your best analytical model and come up with relevant insights, you will need to learn the communication skills to explain the results to the business teams and point out the predicted outcomes once the model is put into production. Learning the soft skills, in addition to

your passion, creativity, and curiosity, will put you over the top among your classroom peers now and potentially your work peers in the future. Select the program that gives you the chance to bring these experiences, either through projects, through real-world experiments, or curriculum. And some of the other things which are also important: does the program take you outside of the classroom and allow you to participate in hackathons or in the Data for Good initiatives within the local community? The latter programs will help bring you a perspective and experience that you will not get in the classroom. SI: BI is a big, growing industry so it’s important to make the right choices early on and not wait until you’re in your MBA program to find out you made the wrong choice.

Patel: Yes, that’s true. That is very true. Because the industry is moving fast, there are new perspectives being brought in. There are new ways to use data and the insights you’re getting when you put it in the hands of different types of users with different types of problems, whether you are incrementally improving an existing problem or whether you’re trying to find or chase a new problem. It’s a challenge, no question. Si

Educating and Empowering the Business Analyst

September 2017 | Solver International

TECHNOLOGY

Solver-International | September 2017

Solver-International.com

SOLVER GETS A FACELIFT – AND NEW MUSCLE Solver, the optimization software, has myriad uses in industries from manufacturing to distribution and logistics, marketing and sales, finance and investment, and human resources. Now it is even stronger and easier to use.

Photo Courtesy of www.shutterstock.com

BY DAN FYLSTRA, CEO, Frontline Systems

Solver – the basic optimization tool in Excel that Frontline Systems developed for Microsoft, probably the longest lived (27 years) and most widely used (with over a billion Excel users) advanced analytics software ever – is getting a “facelift” in its user interface, and new “muscle” to solve larger scale, more challenging optimization problems. Though it solves the same kinds of problems, Solver has been re-imagined for an era of touch-sensitive screens, mobile devices, and cloud computing. All you need to get it—for free—is a web browser, an Office 365 account, and a visit to the Microsoft Office Store.

Educating and Empowering the Business Analyst

September 2017 | Solver International

TECHNOLOGY: SOLVER GETS A FACELIFT If you’re reading this magazine, there’s an excellent chance that you’ve used Solver in desktop Excel, and a decent chance that you’ve used Solver in Excel Online (which has well over 100,000 users). But if you’re new, Solver provides an easy way to turn a spreadsheet model – with input parameters and various calculated results – into an optimization model, that automatically finds the best possible values for chosen input parameters (called decision variables), to maximize or minimize some calculated value (called the objective), while not exceeding

bounds on other calculated values (called constraints). Solver has myriad uses, from manufacturing to distribution and logistics, to marketing and sales, finance and investment, and human resources, in almost every industry. A brief list of examples included with Frontline’s enhanced Solvers in shown in Figure 1. HOW SOLVER GOT STARTED Turn back the calendar to 1989: It was a very different world. IBM-compatible personal computers were widely used in business, but iPhones and iPads didn’t exist yet. Most users were running

Figure 1: Example Solver optimization models in different fields.

Solver International | September 2017

Solver-International.com

TECHNOLOGY MS-DOS on character-mode (non-graphical) displays. Lotus 1-2-3 was “king of the hill” in spreadsheets; Borland Quattro Pro was its challenger; Microsoft Excel was popular on early Macintosh computers, but only perhaps 10 percent of PC users were running it, under an early “Windows 2.0” add-on to MS-DOS. IBM was promoting OS/2 as the new, better alternative to MS-DOS, and Lotus was supporting OS/2. Frontline Systems had just been formed, and had developed a Lotus 1-2-3 addin called “What-If Solver” that could solve optimization problems. At meetings with Lotus, we found their engineers were very interested in what we were doing with What-If Solver, but they never explained why. In the summer of 1989, Lotus introduced 1-23/G, their first graphical spreadsheet, to great fanfare. Besides slick graphics, 1-23/G included a new, internally Figure 2: Excerpt from original Solver project specs and schedule. developed Solver tool that Lotus emphasized as a big advance Bill Gates, made the deal with Frontline over other spreadsheets. And 1-2-3/G – a few years later he called it a “home ran only on OS/2 – not on Windows run for both parties” – and product or MS-DOS. This move by the market manager Tim McGuire shepherded the leader prompted Microsoft (and effort at Microsoft. We worked night and Borland) to hunt for a quick way day throughout 1990 to create Solver for to respond to the challenge of 1-2both Windows and Macintosh. Figure 2 3/G and its Solver. And at Frontline is a brief excerpt from our development Systems, we had an answer. project spec and schedule. After evaluating other developers Solver made its debut as part of from around the world, Microsoft chose Excel 3.0, along with Windows 3.0 to work with Frontline. Pete Higgins, a (a major advance from Windows Microsoft executive reporting directly to 2.0) on May 22, 1990. This was a

Educating and Empowering the Business Analyst

September 2017 | Solver International

TECHNOLOGY: SOLVER GETS A FACELIFT watershed moment: Thereafter, Microsoft Excel steadily gained market share in spreadsheets – exceeding Lotus 1-2-3’s market share by late 1992 or early 1993. We continued working with Microsoft, adapting Solver for each new version of Excel – twelve in all, through Excel 2016. In 1991, Borland signed an OEM agreement with Frontline Systems to develop a Solver for Quattro Pro, and in 1996 Lotus signed an agreement with Frontline to create a new Solver for 1-2-3. STAY THE SAME AND KEEP CHANGING A major reason for Solver’s longevity is that it has been re-invented, in terms

of technology, several times, while maintaining its simple user interface over many years, and maintaining compatibility for Solver models across new versions and new platforms. It’s still possible to open an Excel 3.0 Solver model in Excel 2016, click the Solve button, and get an optimal solution. Figure 3 shows an example Product Mix model in Excel 2016, with the standard Solver modal dialog box open. The 1990 version of Solver used a 16-bit computational engine, a user interface written in the Excel 3.0 macro language, DDE (Dynamic Data exchange) to communicate with Excel for Windows, and AppleScript to communicate with Excel for

Figure 3: Product Mix model with standard Solver dialog in Excel 2016.

Solver International | September 2017

Solver-International.com

TECHNOLOGY Macintosh. Over time, Solver’s computational engine was upgraded to 32-bit and 64-bit versions, the optimization algorithms were improved, the user interface was rewritten using VBA (Visual Basic for Applications), and XLL and COM interfaces were used to communicate with Excel. The new Solver version uses a computational engine running “in the cloud” on Microsoft Azure, a user interface written in HTML, CSS and JavaScript, and new JavaScript APIs to communicate with Excel Online and Excel 2016. Solver has always been quite popular with Excel users – to the extent that, when Excel 2008 for Mac was released without VBA or Solver, an outcry from users led Microsoft to work with Frontline to quickly “bring back Solver” for Excel 2008, in a form that didn’t depend on VBA. Twenty years after its first release, the INFORMS professional society cited the “the overwhelming success and impact of Solver” when awarding Frontline the 2010 INFORMS Impact Prize. SOLVER FOR EXCEL ONLINE The new Solver, which works in both Excel 2016 and Excel Online, is fundamentally different from the standard Solver that’s included with desktop Excel: It is “built for the cloud,” and requires a live Internet connection. (The main reason to use the standard Excel Solver today is that it solves the problem on your own PC and doesn’t require an Internet connection.) The new Solver doesn’t have to be “installed,”

just selected from the Office Store – its user interface is always “served” from a Frontline website, much like a web page. When you click the Solve button, your Excel model is sent (securely) to Frontline’s RASON® server cluster on Microsoft Azure, where the optimization is run; results are then returned to you in Excel 2016 or Excel Online. You’re using “Software as a Service” (SaaS), and Frontline is paying for the compute resources needed to solve the problem – for small models, Frontline absorbs this cost as a marketing expense. The new Solver is in its third generation – and over 100,000 users have used earlier versions. Since Solver is “served from the web” each time it is opened, these users get the latest version automatically. As Office Online evolved, the earliest V1.0 JavaScript APIs for “apps” – later renamed “add-ins” – became available to partners like Frontline Systems at the beginning of 2013, and in July 2013 Frontline released its first-generation Solver app. When new APIs made it possible to create add-ins for Excel for iPad, Frontline developed its secondgeneration Solver add-in, released in December 2015. Now that newer APIs have made it possible to simplify and improve the user experience in all three—Excel 2016, Excel for iPad, and Excel Online—Frontline has released its third-generation Solver. TOUCH-FRIENDLY USER INTERFACE The standard Solver dialog was designed long before touch screens

Educating and Empowering the Business Analyst

September 2017 | Solver International

TECHNOLOGY: SOLVER GETS A FACELIFT and mobile devices became popular. In that version, it is still possible, but not easy, to make selections by touch. The new Solver UI, however, is designed to be easy to use with either a mouse or a fingertip. It also follows UI conventions common on mobile devices. Figure 4 shows the same Product Mix model, still in Excel 2016, but with the new Solver’s “modeless Task Pane” open. AUTOMATIC MODEL DIAGNOSIS AND SOLVER ENGINE SELECTION The standard Solver in Excel handles optimization models of every type – linear programming and nonlinear

optimization, and “arbitrary” models solved via genetic and evolutionary algorithms. But – as many readers have probably experienced – you must determine your problem type yourself, and select the right “solving method.” If you’re a bit rusty on algebra, or if you first built a model for other purposes in Excel and then decided to apply Solver to it, you might not be sure whether your model is linear, nonlinear, or non-smooth. The new Solver determines the problem type for you: It does its own algebraic “diagnosis” of the Excel formulas you’ve used to relate the objective and constraint cells to the decision variable cells, to decide

Figure 4: Product Mix model with new Solver Task Pane in Excel 2016.

Solver International | September 2017

Solver-International.com

TECHNOLOGY the problem type; then it selects the best “solving method” (better called a “Solver Engine”) to handle your problem type – and size. MORE POWERFUL SOLVER ENGINES The new Solver for Excel Online and Excel 2016, like the standard Solver included in Excel 2016, handles models of limited size for free; for years, Frontline has offered compatible Solver upgrade products to scale from hundreds to millions of decision variables and constraints. But the new Solver makes it much easier to “scale up” your model: If you’re using the free version and you try to solve a

model that’s too large, Solver will solve it anyway (subject to some CPU time limits on Frontline’s cloud servers), report the solution status and final objective, and offer an easy upgrade to get full solution results. Once you have a license for one of Frontline’s upgrade products, you can simply login with your Solver.com credentials, and immediately use your license to solve larger models – using the same new Solver in Excel 2016, Excel Online or Excel for iPad. Depending on your license, you can use some or all Solver Engines. You can also use the same license in our Analytic Solver add-in for desktop Excel 2007 through 2016, and on our cloud platform, AnalyticSolver.com

Figure 5: Product Mix model with Analytic Solver Ribbon and Task Pane in Excel 2016.

Educating and Empowering the Business Analyst

September 2017 | Solver International

TECHNOLOGY: SOLVER GETS A FACELIFT – with these products, you can create and run Solver models, Monte Carlo simulation and risk analysis models, decision tree models, and a whole array of forecasting, data mining, and machine learning models. Figure 5 shows the now-familiar Product Mix model with the Analytic Solver Ribbon and Task Pane open in Excel 2016. WHAT IT MEANS FOR YOU Earlier we explained that “When you click the Solve button, your Excel model is sent (securely) to Frontline’s RASON server cluster on Microsoft Azure, where the optimization is run; results are then returned to you in Excel 2016 or Excel Online.” What’s RASON? Indeed, there’s more to this story: RASON is key to getting more than you ever expected out of your Excel Solver model. RASON is an acronym for RESTful Analytic Solver® Object Notation. It is a high-level modeling language that contains the entire Excel formula language, but is embedded in JSON (JavaScript Object Notation) and supports a REST API (Application Programming Interface). Frontline’s RASON Server is a cluster of (virtual) computers providing optimization, simulation and data mining as a service. It is used by making requests over the Web (REST API calls), where the JSON body of each request is a RASON model. Without getting into all the technology details, RASON is a bridge between the world inside Excel, and the world outside – including corporate servers, web and cloud services, and mobile devices. RASON can be a “common language” between business analysts who work in Excel, and

Solver International | September 2017

software developers who work in C++, C#, Java, R, or Python. FROM SOLVER MODELS TO OPERATIONAL ANALYTICS In Figure 5 which depicts Analytic Solver in desktop Excel, you’ll notice a button “Create App” on the Ribbon. This button enables you to translate your Excel Solver model into RASON – and that means your model can run outside Excel on desktops, servers, or in the cloud. Among other things, it means you can build a model in Excel, then easily publish it for use by a wide range of “BI consumers.” These are people using tools like Tableau and Power BI, who may not have the skills to build an analytic model, but who can use your model to help make operational decisions. Used in “point and click,” without any programming in Tableau or Power BI, your model in RASON can connect to a wide variety of data sources to update parameters, re-solve (independently from Excel), and deliver results in tabular or chart form on “BI dashboards.” Used with programming, your model can be easily embedded in desktop, server, web or mobile applications to do almost anything. We believe this is key for organizations that want to realize business value from data and analytic models. And we believe that Solver, Analytic Solver, and RASON offer the easiest and fastest way to reach that goal of business value. If you want to know more, just visit www.solver.com or contact Frontline Systems at support@ solver.com. Si Dan Fylstra is President and CEO of Frontline Systems, Incline Village, Nev.

Solver-International.com

Welcome to Analytic Solver ® Cloud-based Optimization Modeling that Integrates with Excel

Everything in Predictive and Prescriptive Analytics Everywhere You Want, from Concept to Deployment. The Analytic Solver® suite makes the world’s best optimization software available in your web browser (cloud-based software as a service), and in Microsoft Excel. And you can easily create models in our RASON® language for your server, web and mobile apps.

Linear Programming to Stochastic Optimization. It’s all point-and-click: Fast, large-scale linear, quadratic and mixed-integer programming, conic, nonlinear, non-smooth and global optimization. Easily incorporate uncertainty and solve with simulation optimization, stochastic programming, and robust optimization.

And it’s a full-power, point-and-click tool for forecasting, data mining and text mining, from time series methods to classification and regression trees, neural networks and association rules – complete with access to SQL databases, Apache Spark Big Data clusters, and text document folders.

Monte Carlo Simulation, Data and Text Mining. Analytic Solver is also a full-power, point-and-click tool for Monte Carlo simulation and decision analysis, with a Distribution Wizard, 50 distributions, 50 statistics and risk measures, and a wide array of charts and graphs.

Tel 775 831 0300 • Fax 775 831 0314 • info@solver.com

Society News

INFORMS

Bill Griffin

Improving “Softer Skills” in Analytics Helping analytics professionals fine-tune their skills and effectively communicate with non-technical decision makers.

hile being able to communicate and deliver data-driven insights to a non-technical individual—what I call “softer skills”—is essential for data and analytics professionals, it is not often formally addressed in traditional academic curriculums. Regardless how great the results of analytical research, the data isn’t nearly as valuable if you lack the skills to convey conclusions that translate into real business decisions or actions for key decision-makers in your organization. Well-rounded analytics professionals are highly skilled in both worlds, understanding and communicating data, and are constantly improving their analytical and communication techniques. Whether you’re an operations research (O.R.) or analytics professional on a career trajectory, or a team leader at an organization incorporating continuing education into your employee development programs,

Communications: Often the Missing Link

enhancing skills and introducing new concepts through education is vital to data scientists. INFORMS, the leading international association for operations research and analytics professionals, offers its Essential Practice Skills for High-Impact Analytics Projects workshop taught by Dr. Patrick Noonan, former professor of analytics and associate dean at Emory

Solver International | September 2017

University’s Goizueta Business School. This intensive handson, workshop explores the concept of “structured problem solving” and provides participants with practical frameworks to go from ISSUE to ACTION. It combines approaches for analytical and creative problem solving, strategies for communicating and persuading, and processes for managing projects and working in teams.

Solver-International.com

By the conclusion of the course, participants will be able to better apply their subject matter expertise to complex, real-world problems, and effectively communicate their findings. Participants will also learn best practice techniques for analytical and creative problem solving, strategies for communicating and persuading, and processes for managing projects and working in teams. “Essential Practice Skills provided a two-day course in ‘softer skills’ not commonly taught in academic programs: client interaction, problemstructuring, problem decomposition, qualitative methodology, developing work plans, managing work, validating findings, and sculpting communication of results” says Thomas W. Chesnutt, CAP, of A & N Technical Services, Inc. “I found deep insights and structured processes that can be directly applied to my current practice. Highly recommended.” WHO SHOULD ATTEND? This course is ideal for anyone looking to improve and advance their analytics career – participants learn and hone the skills they need to turn analytical insights into organizational actions that drive innovation, growth, and efficiency. Past participants have come from a wide

variety of industries including manufacturing, healthcare, finance, transportation, and have titles ranging from CFOs and Data Scientists, Research Analysts, Business Intelligence Professionals, Academic Program Directors and more. It is offered several times throughout the year around the United States and can be delivered on-site for the benefit of an entire organization. “Irrespective of your level of experience, this is an excellent course that will improve your ability to target the right problems, effectively manage an analytics team, and successfully communicate to a client. Well done and valueadded!” says Dr. Chuck Noon of the University of Tennessee, Knoxville. INFORMS’ Essential Practice Skills for High-Impact Analytics Projects workshop helps provide best practice techniques and practical frameworks for addressing real-world problems. The next public course will be announced here as well as on the INFORMS Website. Additionally, INFORMS will bring the course to an organization that wishes to provide this workshop to an entire team through its Enterprise On-site program. Several large Fortune 100 companies have chosen this option, which saves time and travel costs for large groups. Si

Educating and Empowering the Business Analyst

Participants learn best practice techniques for communicating and persuading.

Bill Griffin is the Continuing Education Program Manager at INFORMS with 20 years’ experience working in academic testing, industry certification, and higher education management.

September 2017 | Solver International

Te r m s o f

THE TRADE

Glossary

Ambari

A web interface for managing Hadoop services and components

Apache Kafka

a distributed streaming platform for building real-time data pipelines and streaming apps.

Apache Spark

Open-source cluster computing framework with highly performant in-memory analytics and a growing number of related projects

Cassandra

A distributed database system

Cubes

A Hadoop job scheduler

Software for streaming data into HDFS

Python

Google BigQuery

BigQuery is Google's fully managed, petabyte scale, enterprise data warehouse for analytics. BigQuery is serverless; there is no infrastructure to manage or a database administrator.

Hadoop

The Apache Hadoop software library is a framework that allows the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

Hadoop Distributed File System (HDFS)

R is a language and environment for statistical computing and graphics. It is a GNU project similar to the S language and environment, The S language is often the vehicle of choice for research in statistical methodology, and R provides an Open Source route to participation in that activity.

Solr

A scalable search tool

Sqoop

Moves data between Hadoop and relational databases

the scalable system that stores data across multiple machines without prior organization.

HBase

A non-relational, distributed database that runs on top of Hadoop

Pig

A platform for manipulating data stored in HDFS

Python is a high-level programming language for generalpurpose programming. Python emphasizes code readability and a syntax which allows programmers to express concepts in fewer lines of code than might be used in languages such as C++ or Java.

Flume

ODBC

ODBC stands for Open Data Base Connectivity, a connection method to data sources.

Oozie

A cube is a set of related measures and dimensions that is used to analyze data. • A measure is a transactional value or measurement that a user may want to aggregate. Measures are sourced from columns in one or more source tables, and are grouped into measure groups. • A dimension is a group of attributes that represent an area of interest related to the measures in the cube, and which are used to analyze the measures in the cube. The attributes within each dimension can be organized into hierarchies to provide paths for analysis.

MapReduce

a parallel processing software framework that takes inputs, partitions them into smaller problems and distributes them to worker nodes

HCatalog

Welch’s Test

Welch’s Test for Unequal Variances (also called Welch’s t-test, Welch’s adjusted T or unequal variances t-test) is used to see if two sample means are significantly different. The null hypothesis for the test is that the means are equal. The alternate hypothesis for the test is that means are not equal.

YARN

(Yet Another Resource Negotiator) provides resource management for the processes running on Hadoop

A table and storage management layer

Hive

A data warehousing and SQL-like query language

Solver International | September 2017

Zookeeper

An application that coordinates distributed processing

Solver-International.com

CERTIFIED ANALYTICS PROFESSIONAL ®

Analyze What CAP Can Do For You