What was hot and what was not? From what I can remember, in-memory analytics, mobile BI, OLAP cubes and drag-and-drop analytics tools got all the buzz. Most customers I worked with simply wanted data warehousing and reporting projects.
Top tier enterprise business intelligence solutions included Oracle, MicroStrategy, SAP Business Objects, IBM Cognos, Information Builders, SAS, and Microsoft. Unlike today, Microsoft, Tableau, and Qlik didn’t get invited to enterprise BI vendor selection projects. Most often I ran into IBM Cognos, SAP Business Objects, Oracle, and MicroStrategy.
In 2010, MicroStrategy, Tableau, and Qlik started to gain significant market traction. Qlik had grown to 13,000 customers and completed an IPO. Tableau doubled revenue that year. In the first six months, Tableau Public visualizations were viewed 4.5 million times.
Meanwhile, Microsoft Excel PowerPivot and SQL Server Reporting Services Power View for SharePoint launched. As an early adopter, I built several PowerPivot case study projects highlighted in that event. Shocked Excel enthusiasts globally couldn’t believe 100 million rows of data could be analyzed. Neither could I. Historically, the Excel worksheet row limit was one million. The powerful in-memory engine called Excel PowerPivot terrified database managers, IT professionals and analytics leaders.
The bundling of BI offerings into SQL Server, SharePoint, and Excel helped Microsoft secure market share via their non-BI related offerings. This would be a recurring theme we continue to see today.
In addition to PowerPivot, Excel was jam-packed with nice BI, OLAP, and data mining features. We got conditional formatting, cool sparklines, slicers, cube write-back and so much more.
In the data science space, SPSS and SAS were the defacto industry standard. Kaggle, a website that serves as a platform for machine learning competitions, launched. Although I proposed numerous predictive analytics projects, no one bought them. Few groups understood the untapped potential in data.
What about cloud? At that time, no one asked about it. Mobile and hosted yes. Cloud…crickets. Quietly in the background, early cloud and SaaS BI apps entered the market thanks to the Salesforce model of cloud app success.
Ah, 2011… I fondly recall 2011 as a joyous year for me professionally. It would end up being my happiest year ever in tech.
In 2011, I excelled in a Microsoft BI technical presales role helping customers design enterprise BI programs. I started this blog to supplement my work. At Microsoft, I loved applying my domain knowledge and technical skills to plan projects and solve complex problems. I also fed my brain continuously with boundless training resources.
Qlik and Tableau started the fundamental market shift to data discovery.
That year, I ran into Microstrategy, Qlik, and Tableau most often. MicroStrategy released a stellar native mobile BI suite. Their beautiful mobile dashboards mesmerized executive buyers. MicroStrategy inspired Tableau, Qlik, and other vendors to prioritize mobile BI. They also influenced me to buy an iPad. I spent countless weekends coding mobile-friendly BI apps to try to emulate them.
Qlik’s in-memory, proprietary technology, and associative data model eliminated the need for complex OLAP cube builds. Both vendors offered much faster, easier builds. Where traditional BI solutions took months, Qlik and Tableau could be deployed within a couple of days or weeks.
In 2011, Qlik skyrocketed to over $200 million in revenue. Green and grey seized enough customers to rank 7th in the overall BI market share. Qlik’s rollercoaster stock valuation changes didn’t slow down sales. They amassed over 15,000 customers in over 100 countries. Qlik’s community grew to 43,000 users. Qlik’s freemium land and expand approach disrupted the frumpy, expensive, slow traditional BI market. Frantic Microsoft sales reps booked me on back-to-back calls for up to six weeks to showcase demos. By the time I could get to a sales demo, Qlik already won the deal and delivered a complete solution.
Tableau also continued to pop up in my accounts. Their 6.1 release included lovely mobile BI support for iPad. My peers in the industry began raving about Tableau’s visualizations and data exploration capabilities.
Spotfire also thrived in 2011. They grew over 50% in license revenue for the year and doubled over the past two years according to earnings calls.
In enterprise accounts that year, I saw a lot of SAP, Oracle, and IBM.
Let’s move on to 2012. In 2012, SQL Server 2012 was released along with SharePoint 2013 BI previews. The enhancements across both offerings were in exceptionally high demand. Technical marketing content development, demos, and POC builds took up most of my time that year. Few people within Microsoft could get the complicated, buggy SharePoint BI installs working.
Although SharePoint was prevalent, the BI tools were brand new. I tried my best to build inspiring, dazzling sales demos. However, Microsoft’s PerformancePoint, Excel and Silverlight versions of Power View paled in comparison to Tableau and Qlik.
That year, Tableau came up in almost every BI sales deal. They became a momentous market force for Microsoft sales to reckon with. Top Microsoft BI customers, partners, and talent ended up going to Tableau in droves. Meanwhile, Microsoft BI execs remained focused on Excel.
Qlik and Tableau changed the BI market in a good way.
Qlik continued to grow that year but word started to get out about Qlik’s load script build learning curves. QlikView seemed to still require IT or BI professionals to develop dashboards. At the same time, Tableau no-code excitement rose in the market.
After I saw Tableau’s TCC 2012 keynote, I knew it was time for me to leave Microsoft. I no longer believed in what I was marketing. Tableau’s elegant drag-and-drop experience continued to get better and more powerful with Tableau v8 – the Kraken. The visualizations were absolutely stunning. It was obvious to me Tableau listened to customers.
In 2012, all the old school BI vendors started losing market share. If you loved no-code visual analytics, you couldn’t resist Tableau. If you liked to sling a little code, then you probably bought Qlik. TIBCO Spotfire also gained steam in research-oriented industries.
My departure from Microsoft felt like a death. It forever changed me. I mourned that loss over the entire year literally weeping myself to sleep at times. Why didn’t executives listen to me? The Microsoft BI community started to wander over to Tableau and Qlik too.
In February 2013, Gartner revealed Tableau ranked the highest in the Leader quadrant. That report merely confirmed what I already knew. I publicly shared my views on the state of the BI market and that report.
During the first half of the year, I pitched combining the best of Microsoft and Tableau. In May, I launched Impact Analytix, LLC. Rather than be tied down to one vendor, I decided to explore all of them again.
No more non-compete stress.
I showcased best practice data visualization. I taught groups about predictive analytics. I developed predictive analytics and surface my results in visualization tools. I indulged in open source R, SAS, SPSS, Alteryx, Microsoft SQL Server and Excel Data Mining, Oracle Data Mining, Oracle SQL Extensions, SAP HANA, KXEN, Sybase, and many other solutions.
During that time, big data hype dominated headlines. The cool new big data analytics tools back then that I reviewed included Splunk, Platfora, Datameer, and Hadapt.
In 2014, I continued to enjoy my freedom exploring the world of analytics. I didn’t make much money but I totally loved the work! I began looking into automated machine learning with SAP KXEN. I also stumbled on a tiny startup in Boston called DataRobot. I ended up stalking them for the next three years.
TIBCO released a fantastic 6.0 update to Spotfire. They were the first data discovery vendor to add predictive algorithms directly within the data discovery UX. I played with that release and publicly shared my findings.
DOMO! I couldn’t talk about 2014 and forget DOMO. Oh, dear DOMO. In 2014, DOMO flooded LinkedIn and the media channels with expensive advertising and outrageous claims. DOMO’s marketing tactics got them market attention yet they remained strangely secretive. At top industry events, DOMO would show up with tents and require NDAs for anyone wanting to see a demo.
All that DOMO noise combined with several client inquiries prompted me to investigate. That’s when I wrote my DOMO review article. As I predicted, DOMO would struggle in the upcoming years. What I didn’t know then was what they’d reveal in their 2018 IPO filing.
In what became an annual tradition, I shared my take on Gartner’s 2014 BI Magic Quadrant. Not much had changed. Tableau continued rapidly growing. Alteryx rose to shining star status by riding on Tableau’s success. Qlik slowed a bit that year even though they unveilied a solid Qlik Sense desktop app. Microstrategy, Oracle, and IBM all lost more market share. Microsoft’s bundling strategy kept them alive and thriving despite a failure with the first Excel-focused version of Power BI.
Speaking of Excel Power BI, it was in July 2014 that year I got the call from Redmond to advise the new GM and product team. I flew to my old employer’s offices to deliver the difficult, much-needed feedback along with ideas and a high-level go to market outline. That guidance helped them get started rebuilding the version of Power BI that you see in the market today. After my Microsoft product team engagement, I wrote my first industry consolidation article on October 13, 2014.
For better or worse, we were headed towards industry consolidation again.
In October 2014, I would rejoin the Microsoft Power BI product team. We all worked 18 to 20-hour days, seven days a week to get a preview in production by December 31. It was intense but we did it. The first public of new Power BI was released the week before Christmas. Exhausted and frightened by our minimal viable product (MVP) offering, I published my first article on it.
The crowded “analytics for everyone” market entered a mid-life crisis. After enduring several years of land and expand self-service chaos, enterprise BI leaders shifted back to IT-led, governed approaches. New solutions IBM Watson Analytics, Salesforce Analytics Cloud “Wave”, ZoomData, Looker, DataHero and DataRPM and established market players Tableau, Qlik, Spotfire, DataWatch, Sisense, GoodData, Birst, YellowFin, DOMO, Datameer, Platfora, Logi Analytics, Antivia, Targit, SAP, SAS, Oracle Cloud Analytics and Microstrategy all scrambled to stand out in the sea of look alike, sound alike offerings.
At that time, the Microsoft Power BI threat was not truly understood yet. From here until the end of the decade, niche analytics vendors would be in for a rough ride. Many stand-alone solutions would end up getting acquired.
Cloud changes the game
Born via data migrations to AWS, Microsoft and Google cloud, a new world of data emerged. Data virtualization and hybrid BI solutions became the new industry hot topics. Cloud data warehouses and data lake technology became simple, fast and cost-effective to spin up and scale in minutes. On-premises Hadoop vendors would soon begin to suffer from the nicer cloud vendor alternatives.
After the launch of innovative IBM Watson Analytics, automated analytics started getting more market attention. It was slick and insightful. Basically, IBM influenced existing BI vendors to quickly add automated exploratory analysis with that launch.
In late 2015, Microsoft finally woke up and realized on-premises BI investment was a necessity to win market share. Most organizations were not ready for the cloud. Thus, the Microsoft Power BI threat to niche BI vendors grew far more intense. In January, I wrote about that key change in a Star Wars themed BI Wars article. Ironically three years later, Scott Galloway wrote an article about the big tech death stars of our economy using a similar theme.
One month later, Thursday, February 4, 2016, rocked the business intelligence and analytics world. That day the release of Gartner’s 2016 Magic Quadrant for BI and Analytics report came out. Unlike prior year reviews where there were modest vendor movements, there were significant vendor shifts. As the lead for Microsoft’s technical RFI response to the Gartner 2016 Magic Quadrant for BI and Analytics report, I was stunned and delighted with Microsoft’s Leader results and highest vision placement.
Gartner’s evaluation criteria changed.
Gartner’s criteria change shifted parts of mega-vendor offerings into a new, separate Market Guide for Traditional Enterprise Reporting Platforms. The changes impacted the placement of several market-leading BI vendors. Several traditional players were moved entirely into that report. New players that made a debut in 2016 included ClearStory, DOMO, Platfora, Salesforce, and Sisense.
Tableau’s investor community responded instantaneously to that report. OMG! DATA stock dropped ~ 50% overnight. My heart sank for my friends over at Tableau. Regardless of where I worked, I still loved Tableau and adored their product team.
From that point on, Power BI take share motions accelerated with top-down, C-level enterprise sales discounts, partner-led land tactics, and community promotion. Free, cheap, good enough Power BI offerings at that time were also bundled into Microsoft Office. Power BI began replacing competing vendor installations at a much quicker pace.
The BI vendors in the market rallied back that year with impressive updates and faster release cycles. We saw Tableau 10, Qlik Sense 3.0, Spotfire 7.6, Sisense, and many other players speed up the development of unique innovations to counter the “good enough” Power BI threat.
With prices being pressured down by Microsoft, the niche analytics vendors would not be able to sustain long-term. Few vendors can compete with Microsoft if they want to copy and kill you.
This is an ongoing theme before, during and after this decade.
On a much happier note, data prep solutions got reinvented in 2016 with Paxata and Trifacta leading the way forward. The awesome visual approaches to solving painful data-wrangling problems enchanted me.
In 2017, we continued to see the ongoing impact of Microsoft Power BI on market consolidation. The 2017 Gartner BI Magic Quadrant didn’t change much that year. Oracle reappeared. Tableau remained the gold standard. Qlik retained their Leadership placement. Spotfire continued to not get ranked as highly as it should score. Pyramid Analytics was undermined while both AWS QuickSight and Looker were not included. Ironically, many vendors that fared well in 2017’s report did not make it through the next two years.
While Tableau focused on Hyper integration and bought ClearGraph for semantic search. Qlik went private to evolve. Both companies became a bit more secretive.
The BI war casualties lingered. Progress bought DataRPM. TIBCO snatched up Jaspersoft. Hitachi acquired Pentaho. Infor seized Birst. Workday picked up Platfora. I’m sure there were more acquisition activities that I’m missing.
Amazon continued to neglect AWS QuickSight even though they touted over 1000 daily innovations and Microsoft’s Power BI sold many cloud deals. With little to show for AWS QuickSight enhancements, where did they invest?
At re:Invent Amazon AWS unloaded a zillion announcements on us including but not limited to stellar cloud infrastructure improvements, database offerings, EMR, Redshift, Aurora, RDS, DynamoDB, ElastiCache and data pipeline services. They also added Glue for ETL, Kinesis for streaming data, Athena, introduced machine learning and a plethora of analytics and artificial intelligence offerings. Finally, Amazon AWS committed to analytics.
Google didn’t do much with Google Data Studio. However, Google did steal massive enterprise data platform deals with Google BigQuery. In 2017, Google made waves with news of a super cool, globally distributed cloud relational database service called Cloud Spanner. Google also embedded Trifacta, previewed Cloud Dataflow, Google Cloud Datalab, and several other machine learning resources.
In 2017, machine learning and artificial intelligence became the new overly-hyped tech. You rarely heard about Bi and analytics anymore. It got drowned out by all the artificial intelligence noise.
Bots, natural language, search, automated analytics, and automated machine learning gem DataRobot started gaining serious market momentum. Other data science vendors H2O.ai, Dataiku, Databricks, Anaconda, and Algorithmia became globally known brands.
Moving on to 2018, Gartner’s BI Magic Quadrant opened the year like a rerun episode of the previous two years. BI market consolidation continued. Salesforce acquired Datorama. Lavastorm was acquired by Infogix.
Europe’s GDPR enforcement fears fueled data catalog adoption. Alation, Collibra, Waterline, Informatica, Datawatch, Attivio, Podium, etc.…anyone and everyone appeared to be building or buying a data catalog. Related master data management, metadata management, and privacy topics also elevated in importance. A myriad of global policy proposals and privacy changes that year further accelerated our progression into the digital surveillance and censorship state we live in today.
Automation and natural language enhancements got added to many analytics and data science offerings. I wrote articles about YellowFin, Zoho Reports, Stratifyd, IBM Watson Studio, DataRobot, and other vendors.
Several months after sharing any new idea that I’d find in the market, big tech vendors seemed to announce or introduce something eerily similar. I knew Microsoft monitored me. I didn’t even realize AWS and Google were also watching me closely. In addition to helping my clients increase market awareness, I felt like I was expediting big tech copy kill death sentences.
In the database market, AWS and Microsoft dominated with more than 75% of the growth.
In a Gartner update called, The Future of Database Management Systems Is Cloud, the market shifts for databases were discussed. While hundreds of database vendors tussled over market breadcrumbs, newcomer Snowflake rose to become a respected top cloud player.
In mid-2018, Qlik impressed me with a fantastic new CEO, better licensing, and compelling future roadmap plans. Qlik’s private equity strategy appeared to help them develop a solid, much wider portfolio of APIs, cloud and data-ecosystem solutions.
Tableau’s passionate community continued to rally for them. Despite extreme competitive pressure, DATA’s stock price recovered. In 2018, Tableau bought Empirical Systems. They also rolled out new pricing and Tableau Prep.
TIBCO delivered a strong Spotfire X release. Spotfire X included automated data wrangling, AI-powered analytics, natural language explanations, real-time streaming, and a sleek new UX.
Fun and creative Sisense continued to grow in the mid-market and embedded OEM space. Their annual event doubled in size.
Last but not least, let’s discuss DOMO again. One of the biggest stories in 2018 was DOMO’s IPO S-1 confessions and warnings. The overly hyped startup appeared to overestimate their CEO’s prior startup experience and underestimate the competition.
DOMO’s S-1 revealed strange spending habits by a CEO who was given too much control. That would be a theme we continue to see today with startups like WeWork. Bloomberg’s article, We’ve Seen the WeWork Saga Before. It’s Called Domo. pretty much summed it up. “An IPO is not a magic bullet that will fix a private company’s shortcomings.” Thus, venture capital investors start pivoting to back to business fundamentals. Profits become crucial for startups. Eccentric personalities telling good stories will begin to struggle to get heavily funded without profits.
To wrap up the past decade in analytics, let’s start with the annual February Gartner 2019 Magic Quadrant for Analytics and BI. This year Gartner placed highly funded, niche BI search vendor ThoughtSpot into the Leader quadrant among Qlik, Tableau, and Microsoft. At that time, ThoughtSpot was mentioned as having over 200 customers. The wide gap between ThoughtSpot’s adoption and the other market leaders’ tens of thousands to Microsoft’s millions of customers seemed peculiar. Several vendors openly questioned BI search as a feature versus a solution.
ThoughtSpot will be one to watch in the evolving analytics market.
ThoughtSpot’s debut as Leader in Gartner’s report significantly boosted market awareness – even more so after one of the Gartner analysts left to join them two months later.
As the year progressed, Qlik’s CEO and ThoughtSpot’s CEO publicly argued with one another. Unlike Oracle and Amazon that constantly tease one another publicly, the arguments we saw were not made for entertainment purposes. Although it was difficult to watch, both CEOs raised industry awareness of controversial non-compete agreements.
Speaking of Qlik, Qlik continued to pleasantly surprise me this past year with excellent data science integrations, extensions and APIs, strong data integration options with Attunity, search enhancements to the Cognitive Engine, and expanding their data literacy program.
In 2019, both the BI and data science markets experienced more consolidation. There are way too many acquisitions to list. These are the highlights. Qlik bought Crunchbot, Crunch Data, and Attunity. TIBCO bought SnappyData. Alteryx acquired Feature Labs and ClearStory Data. DataRobot bought Cursor, ParallelM, and Paxata. ZoomData was bought by Logi Analytics. Google bought Alooma and Looker. Microsoft bought DataSense, Citus Data, and 10 other vendors. Amazon acquired approximately six vendors.
Then came news of Tableau’s acquisition by Salesforce. It sent shockwaves across the market. Salesforce bought Tableau for $15.7 billion in an all-stock deal. Salesforce’s CEO, Marc Benioff, chose Microsoft’s Power BI event keynote day and time to make that announcement. You know that was planned.
I didn’t sleep for several days following Tableau’s acquisition news. While I am thrilled for my friends at Tableau, I’m anxious about what will happen next. For me, seeing Tableau go to a large cloud vendor signaled a cyclical shift back to mega-vendor days.
Cool new technologies talked about in 2019 include time-series databases, streaming databases, graph analytics, and analytics marketplaces. Hybrid distributed computing and serverless architectures went mainstream. Concurrently, algorithms, frameworks, and AutoML solutions rapidly advanced from innovation to commoditization.
In 2019, artificial intelligence and machine learning continued to gain adoption. Finally, organizations got serious about competing in the algorithm economy. Rather sponsor than one-off projects, market-leading companies elevated data science prominence by planning enterprise-wide AI strategies. Unfortunately, while machine learning planning and adoption rates improved, project success eluded most with up to 85% of projects not getting deployed. Meanwhile, mature data science organizations launched ethics, governance and ML ops initiatives.
Until Next Year
With that, I will conclude my marathon reflection blog. I hope you enjoyed the trip down memory lane.
Personally, I’m currently taking some time off to think, visit my family, and reconnect with cherished friends. This past year has been super difficult witnessing my mother’s death and then getting a frightening call for help several weeks later. My father was in the ER. He had an accident.
Both events reminded me that life is short, unpredictable, and precious.
I do plan to share my annual analytics industry predictions with you this coming January. After that, we shall see!