At the end of February, Gartner released the latest 2016 Magic Quadrant for Data Warehouse and Data Management Solutions for Analytics. The biggest take-away this year was the key trends to watch in a rapidly evolving data warehouse market. Much like the 2016 Magic Quadrant for BI and Analytics, there was a bit of vendor movement. For data professionals, the full report is a fantastic read to understand where our industry is headed and the emergence of the logical data warehouse (LDW) in a hybrid data world.
A logical data warehouse is a combination of hardware and software. Much like a traditional data warehouse, a logical data warehouse organizes data by subject and honors the concepts of time intelligence. Topics such as distributed queries, data virtualization and hybrid on-premises/cloud storage are often included in broader logical data warehouse architecture discussions.
As the data world changes to include unstructured data, streaming data, event data and many other forms of data, the data warehouse definition is also changing. Today’s data warehouse is not necessarily a relational database. The 2016 Magic Quadrant results below truly illustrate that point with players like MongoDB, Cloudera, Hortonworks, and MarkLogic in the mix.
April 19, 2018: The Magic Quadrant image and report links have been removed due to a Gartner content take down request. Interestingly, Gartner has not required content take down on hundreds of other personal and business websites that have also not purchased it, Twitter, LinkedIN, Reddit, Pinterest and other social media posts on the public domain including websites that have publicly criticized the reports. If you do want to see the Magic Quadrant graphic and report, run a simple Google search: Gartner Data Warehouse Magic Quadrant.
Notable Market Trends
The war for getting your data into the cloud is heating up amongst mega-vendors Amazon, Oracle, Microsoft, SAP, IBM and others. According to Gartner, more organizations are reviewing cloud-based analytics environments these days. In the past, cloud BI and analytics accounted for ~2% of the BI market. I am pretty sure I know why that trend is changing now.
Cloud is a lucrative, high profit margin business. It is a serious business – quite possibly the most critical business for technology mega-vendors to secure. The vendor that wins most of your cloud data and cloud app usage, most likely wins the cloud war. Right now there are heavily overstaffed, hungry vendor cloud sales forces out there that are eager to win market share. Cloud is all about acquiring more data and more cloud app usage. The more data hosted by a cloud vendor, the more likely increased data gravity alone will pull in additional data in the future. Now add in appealing easy button simplicity of evaluating a highly scalable, cloud data warehouse or analytics app in minutes. Plus you get paid amazing incentives to do it. Why not try it? Unless you can’t explore cloud due to legal or data privacy restrictions, the cloud full court press is on and will not being going away.
As cloud data warehouses become more appealing to try and adopt, analytics appliances and hardware offerings are getting cannibalized. Think about it. Instead of spending millions of dollars on hardware, that funding can now be redistributed to data warehouse implementers, cloud provider subscription models and other projects. Thus we have more reasons why organizations are considering cloud. Analytics and data warehouse service providers are selling it to enjoy a larger wallet share of your budget. Less money on hardware means more money for other areas.
Another trend cited this year was the growing popularity of data lakes. If you are not familiar with data lakes, KD Nuggets has a good article from SAS that compares data lakes with data warehouses. Gartner mentions organizations have succeeded using three analytics use cases for getting value out of big data:
1) data exploration/data science “sandboxes”; 2) offloading of history from the warehouse; and 3) moving transformation support back off the data warehouse platform
Interestingly, Gartner also noted that customers are taking best-of-breed and best-fit approaches because no single product is a complete solution. Having been a services provider and hands-on implementer for many years, I can share that I love the best-of-breed approach myself. When you use it, you sometimes trade-off the ease of integration and single point of contact to call with issues.
Gartner BI Summit
Next week is the annual Gartner BI Summit. If you are attending, please connect with me while you are there. I have huuuuge news to share. If you are not going, I promise to write up a summary with you just like I did last year. Check out the hot topics in 2016:
- Advanced Analytics
- Big Data
- Automated Decisions
- Self Service
- Internet of Things
- Data Lakes
- Modern bi-modal BI
- Chief Analytics Officer
- Algorithmic Business
Personally, I can’t wait to learn more about the advances in automated decisions and the algorithmic business. Things are really getting exciting with predictive analytics now taking center stage in our data loving world.