The Discipline of Business Experimentation
Innovations Don’t Always Pay Off. Sometimes They Fail Spectacularly.
Big Data Has It’s Limits It can’t always predict the future.
Innovations Need Rigorous Testing A more reliable way to evaluate new initiatives.
A RIGOROUS EXPERIMENT • • • •
Relies on proven scientific and statistical methods. Has a sample size that will yield valid results. Tests one independent variable against a dependent variable while holding all other variables constant. Incorporates careful observation and analysis.
Five Keys to Good Experiments
Purpose
Buy-In
Feasibility
Reliability
Value
#1
Does the Experiment Have a Clear Purpose? • Start with a Strong Hypothesis • Avoid if the Hypothesis results are weak
#2 Have Stakeholders Agreed to Abide by the Results? • Weigh All the Findings • Walk Away if They’re Negative
#3 Is the Experiment Doable? • Potential Roadblocks
• Too Much Complexity • Costly Sample Size • Operational Disruptions
#4 How Can We Ensure Reliable Results?
• Randomized Field Trials • Blind Tests • Big Data
#5 Have We Gotten the Most Value out of the Experiment? • Invest in areas where the ROI will be highest.
• Determine which components have a positive return.
It All Comes Down to Rigor
Purpose
Buy-In
Feasibility
Reliability
Value
Experiment Checklist Purpose • What specific management action are we considering?
• What do we hope to learn? Buy-In • What specific changes will we make on the basis of the results? • How will we ensure that the results aren’t ignored?
Experiment Checklist Feasibility • Do we have a testable prediction? • What sample size do we need? • Can we avoid disrupting operations at the test locations? Reliability • What measures will we take to counteract bias? • Would others conducting the same test obtain similar results?
Experiment Checklist Value • Can we do a targeted rollout focusing on areas where the payback is highest? • Have we implemented only the components with the highest returns? • Do we understand which variables are causing which effects?
Thank You You can contact us at 00966 56 100 4748 info@dtechsystems.co www.dtechsystems.co