THE EVOLUTION OF COMPETITIVE INTELLIGENCEÂ The Past, Present & Future of Intelligence Programs
EXECUTIVE SUMMARY BY
LIZ
FARQUHAR
Looking back at the evolution of competitive intelligence in the business world, it's clear that the most effective strategies are those that integrate primary and secondary research, using internal resources with external partners and tools. This is especially critical now, and in the future, as more tools and resources become available to intelligence teams, marketers, product teams and business leaders. As with everything else in the business world, the invention of the internet, as well as more recent advancements in artificial intelligence and machine learning, have completely transformed CI/MI programs. What used to be an extremely manual process for gathering and delivering intelligence briefings, can now be largely automated. Thus, creating an opportunity for deeper human analysis and smarter decision making at every level of an organization. Of course, as with all things, there are always going to be people and organizations that resist change, refusing to adopt new tools and techniques. History has proven that this type of defensive approach is not only shortsighted, but also unsustainable in the business world. Not only that, but as we’re about to show you, early adopters - and those who have been most willing to evolve with new innovations - are often the most likely to succeed. And in this day and age relevant, actionable intelligence is critical as it becomes more difficult for businesses to remain relevant and sustain growth in an increasingly crowded global marketplace. By looking back through the history of CI and MI, we've found what differentiates companies with the highest return on their intelligence investment is the degree to which they use outsourced services and technology to support in-house efforts.
CI AS A BUSINESS TOOL The origins of competitive intelligence for businesses in the
By the mid-80s, national corporations like Motorola,
United States are rooted in marketing. During the 1960s
Exxon Mobil, Procter & Gamble and Johnson & Johnson
and much of the 1970s, competitive analysis was conducted
had built formal CI organizations internally to gain a
as a market research function through the lens of sales,
competitive advantage in their respective markets. These
marketing and advertising campaigns.
programs were strategically designed to support marketing and product development decisions on a
In the mid-to-late 70s, business leaders in the US expanded
tactical level, in addition to guiding high-level executive
CI activities to inform high-level decision-making in a more
decisions for the business.
formal capacity. In addition to gathering primary research, companies began to invest more time and resources in
And of course, by the late 80s, intelligence gathering,
secondary research methods. For many companies, these
especially of the secondary variety, had already come a
new research projects were spearheaded by members of
long way. For instance, Bell Atlantic Directories took
the marketing team who would gather and compile
advantage of the growing need for market intelligence to
intelligence briefings for executives and department heads
better position their business in a competitive phonebook
from newspaper clippings, as well as radio and television
market. The company restructured its business model and
transcripts.
primary product lines to focus on market research directories and buying guides for B2B and B2C purposes.
The first “tool” for CI came in 1980 with Michael Porter's book "Competitive Strategy: Techniques for Analyzing
Unfortunately, even as advancements were being made in
Industries and Competitors.” In his book, Porter introduced
tools and resources for curating intelligence, the process
the five factor analysis, which helped businesses analyze
was still very manual for practitioners, with the
competitors' activities and plan strategic responses. To this
management and distribution of intelligence remaining
day, Porter’s methodology is widely viewed as the
inconsistent and difficult to formalize.
foundation of modern competitive intelligence.
INVENTION OF THE INTERNET When the "World Wide Web" was first launched in 1990, it quickly transformed the way businesses collected and disseminated intelligence. Up to that point, competitor
"THE INTERNET IS THE
analysis was largely restricted to advertising spend, highlevel R&D activities and new hire announcements. The internet connected the dots between primary and
FIRST THING THAT
secondary information sources. HUMANITY HAS BUILT
Company websites were especially valuable when competitors were unknowingly leaking classified
THAT HUMANITY DOESN'T
information on a regular basis. Of course, once companies realized the amount of sensitive information they were giving away, they got smarter about what they published
UNDERSTAND, THE
on their websites. This practice became especially important to keep up with when search engines and news
LARGEST EXPERIMENT IN
monitoring tools came on the scene in the mid-1990s. SEARCH ENGINES & WEB SYNDICATION
ANARCHY THAT WE HAVE
By 1999, there were already more than 300 search engines and database tools available to consumers and
EVER HAD."
businesses alike.
- ERIC SCHMIDT, GOOGLE
Tools like Deja News (now known as Google Groups) indexed content and information from forums and groups on the internet, while others like DialogWeb were built for research professionals and combined technology with human analysis. In addition to internet search engines, more and more news monitoring services began to hit the market. NewsLink and NewsWork were two of the first tools to focus solely on aggregating news sources on the local and national level. Top CI professionals would use these tools to uncover local news stories about a competitor's headquarters and for CEO profiles. At the turn of the century, developers made significant advancements in website indexing and internet research tools. In addition to search engines like Google and Yahoo! entering the market, "push" technologies like RSS Feeds and information services with email briefings were increasingly more available to corporate researchers for automated alerts on competitor activity and market conditions. However, despite these advancements in intelligence gathering, many companies were still struggling with storing and sharing intelligence. The amount of data available through the internet made knowledge management and data storage more important than ever, and even more difficult to achieve.Â
COMPETITIVE INTELLIGENCE IN THE DIGITAL AGE The financial crisis of 2007-2009 forced many companies to apply leaner business models. These models relied heavily on digital innovations to drive cost efficiencies without negatively impacting customer satisfaction and employee engagement. Of course, artificial intelligence and machine learning emerged as a solution to these business challenges, and continue to play a big role in achieving these goals, especially when it comes to market research and competitor monitoring for CI units, product teams, marketers and executives. Two particularly important AI trends for modern CI success are Intelligent Process Automation (IPA) and human-in-the-loop applications that combine human intelligence with machine learning.Â
INTELLIGENT PROCESS AUTOMATION In the past few years, IPA has become a vital innovation among product and marketing teams who rely on it to save time on intelligence gathering and curation. Many companies are experimenting with IPA and through these new tools are able to automate up to 70% of repetitive tasks. For product managers and product marketers, this type of process automation is vital for productivity – and sanity in many cases. Tracking even just 10 competitors and market conditions, along with other day-to-day responsibilities is difficult for individual contributors, and largely impossible for small teams with limited resources and time constraints.
HUMAN-IN-THE-LOOP HUMAN-IN-THE-LOOP Although AI has come a long way, it still has a long
Human Analysis 20%
way to go if it’s ever going to learn to replace human intelligence. To address limitations with machine learning and automation, business leaders are adopting a sort of “80:20” approach to technology solutions. Right now, 80% of manual, repetitive activities can be effectively automated/handled by machine intelligence. For the remaining 20%, there’s a human put in the loop to tie up the loose ends and do other AI/Automation 80%
tasks that machines are not yet able to effectively learn or perform as well.
In fact, CI Radar is an example of this business model. Our algorithms and proprietary technology produce outputs of curated data based on key intelligence topics that are defined by human analysts. Our platform does the repetitive, manual intelligence gathering activities, pulling from thousands of online sources 24/7. This enables our analysts to focus on the daily review and analysis of information to remove irrelevant findings, continually optimize search algorithms and identify gold nuggets of intelligence to share with clients through alerts and email briefings.
BENEFITS OF IPA & HUMAN-IN-THE-LOOP The biggest benefit that companies have realize when it comes to IPA and HITL applications is the stackability of these technologies. By using a technology stack, every team from marketing and product to CI and senior management, can streamline and automate up to 80% of repetitive, low-value and time-consuming tasks and drastically increase time spent on critical, high-value tasks that inform corporatelevel decision making.
THE NEW ORGANIZATIONAL STRUCTURE FOR CI While it may be tempting to assume that AI is the key to solving every business challenge you have, it will only create different problems without the right organizational structure to support it. To successfully implement HITL and IPA
Think of it like this, would you rather spend a
systems, structural changes have to be made to
few thousand dollars upfront on an accurate
support ongoing management, optimization
and reliable CI solution that cuts research
and integration.
time in half or manage information between 5-10 different free tools and spend 5-6 hours
For instance, there are plenty of free tools, like
a week on basic data filtering in place of high
Google Alerts, that seem to save time by
value analysis?
automating portions of intelligence gathering. However, more often than not, these same
Although AI has come a long way in the past
tools create more work by aggregating
few years, it still has a long way to go before
unfiltered data that still requires careful review
any robot can effectively replace a human
from subscribers.
analyst for intelligence gathering and data analysis.
When it comes down to it, the cost of using 1020 different free tools is exponentially more in
Instead of focusing on how robots can or will
the long-term than a smart, upfront investment
replace jobs eventually, our time would be
in building a well-structured technology stack
much better spent on discussions of how
and optimized workflow.
they can support and improve people's jobs.
WHAT'S NEXT? THE FUTURE OF CI & NEXT GENERATION TOOLS Until AI and machine learning can replicate the human brain (if that day ever comes), it will be important for companies and emerging industry leaders to continue finding the balance between artificial and human intelligence. Until the entire data collection, analysis and reporting process can be automated by robots, it is important for business leaders to find ways to take the robot out of the human by automating the time-consuming, manual tasks that have plagued the field since its inception. Of course, there’s no way to know exactly what’s coming next or what mind-blowing innovations are headed down the pipeline. The only thing we know for sure is that in this new highly competitive world, technology is going to play a crucial role in continued business growth and long-term success for market intelligence and product development.Â